Skip to content

[Bug][CPU] QDQ MatMul fusion accepts INT32 activations unsupported by MatMulIntegerToFloat #33437

Open

Description

@Yuhx141

Problem

INT32 activation dequantization with zero zero-point is legal and produces a FLOAT MatMul. The QDQ MatMul selector creates MatMulIntegerToFloat with an INT32 A input, whose type contract excludes INT32, and rejects the optimized graph.

Minimal reproduction

Install onnx==1.22.0, numpy==2.4.6, and the ONNX Runtime wheel under test. Save this as repro.py and run python repro.py. It constructs the model and fixed inputs directly; no downloaded model, attachment, private repository, or research framework is needed. Source legality and unoptimized execution are checked before the optimized result and pass-disabled control. Expected final line: REPRODUCED.

import copy
import hashlib
import json
import tempfile
from pathlib import Path
import numpy as np
import onnx
import onnxruntime as ort
from onnx import helper as h, numpy_helper as nh

def run(model, feeds, optimize=False, disabled=(), export=None):
    options = ort.SessionOptions()
    options.graph_optimization_level = (ort.GraphOptimizationLevel.ORT_ENABLE_ALL
        if optimize else ort.GraphOptimizationLevel.ORT_DISABLE_ALL)
    options.intra_op_num_threads = options.inter_op_num_threads = 1
    options.log_severity_level = 4
    if export is not None:
        options.optimized_model_filepath = str(export)
    session = ort.InferenceSession(model.SerializeToString(), options,
        providers=["CPUExecutionProvider"], disabled_optimizers=set(disabled))
    assert [v.name for v in session.get_outputs()] == [v.name for v in model.graph.output]
    return session.run(None, feeds)

def exact(left, right):
    return len(left) == len(right) and all(isinstance(a, np.ndarray) and isinstance(b, np.ndarray)
        and a.shape == b.shape and a.dtype == b.dtype and np.array_equal(a, b)
        for a, b in zip(left, right))

def describe(values):
    return [None if a is None else {"shape": list(a.shape), "dtype": str(a.dtype),
        "sample": a.reshape(-1)[:8].tolist()} for a in values]

def make_source():
    nodes = [
        h.make_node('DequantizeLinear', ['a', 's', 'az'], ['af']),
        h.make_node('DequantizeLinear', ['b', 's', 'bz'], ['bf']),
        h.make_node('MatMul', ['af', 'bf'], ['y']),
    ]
    initializers = [
        nh.from_array(np.array(1.0, dtype='float32'), 's'),
        nh.from_array(np.array(0, dtype='int32'), 'az'),
        nh.from_array(np.array([[3], [4]], dtype='int8'), 'b'),
        nh.from_array(np.array(0, dtype='int8'), 'bz'),
    ]
    return h.make_model(h.make_graph(nodes, "repro", [h.make_tensor_value_info('a', 6, [1, 2])],
        [h.make_tensor_value_info('y', 1, [1, 1])], initializers),
        opset_imports=[h.make_opsetid('', 13)], ir_version=10)

source = make_source()
onnx.checker.check_model(source, full_check=True)
onnx.shape_inference.infer_shapes(source, strict_mode=True)
feeds = {'a': np.array([[1, 2]], dtype='int32')}
expected = [np.array([[11.0]], dtype='float32')]
before = run(source, feeds)
assert exact(before, expected), (describe(before), describe(expected))
guarded = run(source, feeds, optimize=True, disabled=('QDQSelectorActionTransformer',))
assert exact(before, guarded)
try:
    run(source, feeds, optimize=True)
except Exception as error:
    assert all(token in str(error) for token in ['MatMulIntegerToFloat', 'int32']), str(error)
    print(type(error).__name__, str(error).split("Status Message:")[0])
else:
    raise AssertionError("The reported optimization failure did not reproduce")
print("source_sha256", hashlib.sha256(source.SerializeToString()).hexdigest())
print("ort", ort.__version__, ort.get_build_info())
print("REPRODUCED")

Expected and actual behavior

The minimal graph has scale1, zero points0, activation [1,2] and weight [[3],[4]]. Direct integer arithmetic followed by FLOAT conversion gives MatMul output [[11.0]] exactly.

Observed output:

InvalidGraph [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. Type Error: Type 'tensor(int32)' of input parameter (a) of operator (MatMulIntegerToFloat) in node () is invalid.
source_sha256 b81d9e7ec3ca7623f327ddbdbdc07a49fe6ac05b2f6e10d12ac7bcc4eee474a9
ort 1.29.0 ORT Build Info: git-branch=HEAD, git-commit-id=2e2543fbe9, fp8-kv-cache=1, build type=Release
REPRODUCED

Causal analysis

MatMulNodeGroupSelector checks some INT8,16-bit and4-bit conditions, but its no-output-Q branch can accept unsupported INT32 activations solely because MatMulIntegerToFloat fusion is enabled. The target activation-type precondition must be checked before creating it.

Source reference: onnxruntime/core/optimizer/qdq_transformer/selectors_actions/qdq_selectors.cc. No native patch was built or tested for this report.

Related reports and discovery scope

#33261 is INT32 Conv weights passing ConvNodeGroupSelector and creating QLinearConv. This witness uses a different selector, activation port and target operator. A guard in the Conv selector cannot fix the MatMul no-Q branch.

Targeted open/closed issue and PR searches were refreshed immediately before publication; no exact covering report was identified. This is not a claim of exhaustive global novelty.

the reduced model is a newly constructed three-node witness with an INT32 graph input. the retained generated source/the saved original input preserve the real generated six-node source that obtained INT32 data through Cast. The displayed constructors rebuild the tested graph and input values. Model serialization hashes can differ from the retained original artifacts; no byte-identity claim joins those representations.

Environment

Linux x86_64, Python 3.12.3, ONNX 1.22.0, NumPy 2.4.6, ONNX Runtime 1.29.0, CPUExecutionProvider, one intra-op and inter-op thread. Actual binary build information: ORT Build Info: git-branch=HEAD, git-commit-id=2e2543fbe9, fp8-kv-cache=1, build type=Release. Native module SHA-256: ff54b93f257508c8e32a43f3528382b7eb791f0946730187e4767d7292a290ee. No source tag is claimed to be the build source of this installed binary. Current upstream source was reviewed at c1ea5bffb8f78d6d6b7239eccf66ebfc3e5833b0; this is a static source observation, not execution of a newly built main branch.

Activity

  1. xgbtxy commented on Oct 20, 2026

    @xgbtxy

    I reproduced this on Windows x64 with ONNX Runtime 1.31.0: the checked source model and the pass-disabled control return [[11.0]], while optimization creates an invalid INT32 MatMulIntegerToFloat input.

    I'm preparing a selector guard matching the target operator's 8-bit input constraints, with INT32 regression coverage and supported 8-bit fusion controls. This work uses AI assistance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    quantizationissues related to quantization

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions