Repository navigation
[Bug][CPU] QDQ MatMul fusion accepts INT32 activations unsupported by MatMulIntegerToFloat #33437
Description
Problem
INT32 activation dequantization with zero zero-point is legal and produces a FLOAT MatMul. The QDQ MatMul selector creates MatMulIntegerToFloat with an INT32 A input, whose type contract excludes INT32, and rejects the optimized graph.
Minimal reproduction
Install onnx==1.22.0, numpy==2.4.6, and the ONNX Runtime wheel under test. Save this as repro.py and run python repro.py. It constructs the model and fixed inputs directly; no downloaded model, attachment, private repository, or research framework is needed. Source legality and unoptimized execution are checked before the optimized result and pass-disabled control. Expected final line: REPRODUCED.
import copy
import hashlib
import json
import tempfile
from pathlib import Path
import numpy as np
import onnx
import onnxruntime as ort
from onnx import helper as h, numpy_helper as nh
def run(model, feeds, optimize=False, disabled=(), export=None):
options = ort.SessionOptions()
options.graph_optimization_level = (ort.GraphOptimizationLevel.ORT_ENABLE_ALL
if optimize else ort.GraphOptimizationLevel.ORT_DISABLE_ALL)
options.intra_op_num_threads = options.inter_op_num_threads = 1
options.log_severity_level = 4
if export is not None:
options.optimized_model_filepath = str(export)
session = ort.InferenceSession(model.SerializeToString(), options,
providers=["CPUExecutionProvider"], disabled_optimizers=set(disabled))
assert [v.name for v in session.get_outputs()] == [v.name for v in model.graph.output]
return session.run(None, feeds)
def exact(left, right):
return len(left) == len(right) and all(isinstance(a, np.ndarray) and isinstance(b, np.ndarray)
and a.shape == b.shape and a.dtype == b.dtype and np.array_equal(a, b)
for a, b in zip(left, right))
def describe(values):
return [None if a is None else {"shape": list(a.shape), "dtype": str(a.dtype),
"sample": a.reshape(-1)[:8].tolist()} for a in values]
def make_source():
nodes = [
h.make_node('DequantizeLinear', ['a', 's', 'az'], ['af']),
h.make_node('DequantizeLinear', ['b', 's', 'bz'], ['bf']),
h.make_node('MatMul', ['af', 'bf'], ['y']),
]
initializers = [
nh.from_array(np.array(1.0, dtype='float32'), 's'),
nh.from_array(np.array(0, dtype='int32'), 'az'),
nh.from_array(np.array([[3], [4]], dtype='int8'), 'b'),
nh.from_array(np.array(0, dtype='int8'), 'bz'),
]
return h.make_model(h.make_graph(nodes, "repro", [h.make_tensor_value_info('a', 6, [1, 2])],
[h.make_tensor_value_info('y', 1, [1, 1])], initializers),
opset_imports=[h.make_opsetid('', 13)], ir_version=10)
source = make_source()
onnx.checker.check_model(source, full_check=True)
onnx.shape_inference.infer_shapes(source, strict_mode=True)
feeds = {'a': np.array([[1, 2]], dtype='int32')}
expected = [np.array([[11.0]], dtype='float32')]
before = run(source, feeds)
assert exact(before, expected), (describe(before), describe(expected))
guarded = run(source, feeds, optimize=True, disabled=('QDQSelectorActionTransformer',))
assert exact(before, guarded)
try:
run(source, feeds, optimize=True)
except Exception as error:
assert all(token in str(error) for token in ['MatMulIntegerToFloat', 'int32']), str(error)
print(type(error).__name__, str(error).split("Status Message:")[0])
else:
raise AssertionError("The reported optimization failure did not reproduce")
print("source_sha256", hashlib.sha256(source.SerializeToString()).hexdigest())
print("ort", ort.__version__, ort.get_build_info())
print("REPRODUCED")Expected and actual behavior
The minimal graph has scale1, zero points0, activation [1,2] and weight [[3],[4]]. Direct integer arithmetic followed by FLOAT conversion gives MatMul output [[11.0]] exactly.
Observed output:
InvalidGraph [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. Type Error: Type 'tensor(int32)' of input parameter (a) of operator (MatMulIntegerToFloat) in node () is invalid.
source_sha256 b81d9e7ec3ca7623f327ddbdbdc07a49fe6ac05b2f6e10d12ac7bcc4eee474a9
ort 1.29.0 ORT Build Info: git-branch=HEAD, git-commit-id=2e2543fbe9, fp8-kv-cache=1, build type=Release
REPRODUCED
Causal analysis
MatMulNodeGroupSelector checks some INT8,16-bit and4-bit conditions, but its no-output-Q branch can accept unsupported INT32 activations solely because MatMulIntegerToFloat fusion is enabled. The target activation-type precondition must be checked before creating it.
Source reference: onnxruntime/core/optimizer/qdq_transformer/selectors_actions/qdq_selectors.cc. No native patch was built or tested for this report.
Related reports and discovery scope
#33261 is INT32 Conv weights passing ConvNodeGroupSelector and creating QLinearConv. This witness uses a different selector, activation port and target operator. A guard in the Conv selector cannot fix the MatMul no-Q branch.
Targeted open/closed issue and PR searches were refreshed immediately before publication; no exact covering report was identified. This is not a claim of exhaustive global novelty.
the reduced model is a newly constructed three-node witness with an INT32 graph input. the retained generated source/the saved original input preserve the real generated six-node source that obtained INT32 data through Cast. The displayed constructors rebuild the tested graph and input values. Model serialization hashes can differ from the retained original artifacts; no byte-identity claim joins those representations.
Environment
Linux x86_64, Python 3.12.3, ONNX 1.22.0, NumPy 2.4.6, ONNX Runtime 1.29.0, CPUExecutionProvider, one intra-op and inter-op thread. Actual binary build information: ORT Build Info: git-branch=HEAD, git-commit-id=2e2543fbe9, fp8-kv-cache=1, build type=Release. Native module SHA-256: ff54b93f257508c8e32a43f3528382b7eb791f0946730187e4767d7292a290ee. No source tag is claimed to be the build source of this installed binary. Current upstream source was reviewed at c1ea5bffb8f78d6d6b7239eccf66ebfc3e5833b0; this is a static source observation, not execution of a newly built main branch.
Activity
xgbtxy commentedon Oct 20, 2026
I reproduced this on Windows x64 with ONNX Runtime 1.31.0: the checked source model and the pass-disabled control return
[[11.0]], while optimization creates an invalid INT32MatMulIntegerToFloatinput.I'm preparing a selector guard matching the target operator's 8-bit input constraints, with INT32 regression coverage and supported 8-bit fusion controls. This work uses AI assistance.