Fix QMoE SM80 FP4 init order: don't let the tactic seed throw (#32194)

### Description
<!-- Describe your changes. -->

Fix MXFP4 QMoE session initialization on Ampere by applying the SM80
grouped-GEMM routing decision **before** default tactic discovery.

The original failure was:

```text
TMA WS grouped MoE GEMM for SM80 is not compiled, and this QMoE configuration has no SM80 fallback
```

`CutlassMoeFCRunner` previously queried tactics before QMoE could call
`setUseSm80Fp4()`, so an SM80 MXFP4 runner queried the unavailable TMA
family with the provisional `false` flag.

Changes addressing the review:

- Add a defaulted `use_sm80_fp4` constructor argument and initialize
through the existing `setUseSm80Fp4()` helper, which selects both
default GEMM tactics.
- Pass the captured decision from both FP16 and BF16 primary MXFP4 QMoE
construction sites. Standard MoE, dense fallback, and other quantization
callers retain the default `false` behavior.
- Remove the blanket exception catch entirely. Unsupported
configurations and allocation failures propagate from initialization;
there is no suppressed failure or deferred error. With no remaining
`std::exception` reference, an `<exception>` include is unnecessary.
- Add six session-construction regressions covering FP16/BF16 with
`ORT_FP4_SM80_GEMM` unset, explicitly enabled, and disabled. Disable CPU
EP fallback in these regressions.
- Ensure A100 runs both the new initialization tests and the existing
parity/initializer-release tests. After merging current main, preserve
its expanded SM80+ eligibility, including SM120/121.
- Give the shared MXFP4 test graph builder explicit standard and
`com.microsoft` opset imports so the regression models also pass the
ONNX checker.
- Merge current main and resolve the two overlapping QMoE/test edits,
retaining main's new empty-tactic initialization check and expanded
architecture support.

#### Validation of this revision

Run from a neutral directory with the transformers helper directory on
`PYTHONPATH`, using the existing isolated Python environment:

```powershell
python -m pytest C:\Users\gnallasamy\Codes\onnxruntime-pr32194\onnxruntime\test\python\transformers\test_qmoe_fp4_cuda.py -q -rs -k 'TestFP4PackingUtility or TestQMoEFP4Sm80SingleWeightCopy'
```

**Result:** 4 passed, 15 skipped, 19 deselected. The skips are 14 CUDA
cases (including all six new initialization cases) and one unavailable
native packing binding. These skips are **not GPU validation**.

```powershell
lintrunner --take CLANGFORMAT,RUFF,RUFF-FORMAT onnxruntime\contrib_ops\cuda\llm\moe_gemm\moe_kernels.h onnxruntime\contrib_ops\cuda\llm\moe_gemm\moe_kernels.cu onnxruntime\contrib_ops\cuda\moe\moe_quantization.cc onnxruntime\test\python\transformers\test_qmoe_fp4_cuda.py
git diff --check
```

**Result:** both passed.

Additional focused runtime checks passed: Python syntax parsing, ONNX
checker validation of the actual FP16/BF16 regression graphs, acceptance
of SM80/86/89/90/100/119/120/121 and exclusion of SM79 by the regime
gate, and environment override/restoration for all three modes.

**Still required before merge:** build the revised CUDA provider and
execute the targeted tests on an SM80/A100 CUDA 12.8+ FP4-QMoE-enabled
build. This Windows validation environment has an SM86 RTX A1000,
CPU-only Python test packages, and no CUDA toolkit; it cannot compile or
execute the revised CUDA implementation. Also confirm the new-head CI
results and obtain maintainer approval.

#### Historical author-reported validation

The original revision was exercised on A100-80GB (SM80), using
`openai/gpt-oss-20b` MXFP4 with ONNX Runtime GenAI continuous batching.
The reported SM80 path initialized successfully, used 24,149 MiB peak
GPU memory versus approximately 49,800 MiB for the dequant fallback,
loaded in 4.56 s versus approximately 5.9 s, and had exact output
parity. This empirical run predates the constructor-plumbing revision
above and has **not** been repeated for it.

### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

The SM80 MXFP4 grouped-GEMM path is selected by default on supported
GPUs, but tactic discovery during runner construction happened before
that selection was applied. This made the intended Ampere path
unreachable at session initialization.

Passing the decision into construction fixes the ordering at its source
without suppressing unrelated initialization failures in the shared MoE
runner. Regression coverage now includes the affected SM80 architecture
instead of skipping it.

---------

Co-authored-by: Gopalakrishnan Nallasamy <gopalakrishnan.nallasamy@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: 3 people

onnxruntime/contrib_ops/cuda/llm/moe_gemm/moe_gemm_template_dispatch.h

@@ -671,10 +671,8 @@ MoeGemmRunner<T, WeightType, OutputType, ScaleBiasType>::getTmaWarpSpecializedCo
   auto config_type_param = static_cast<CutlassGemmConfig::CandidateConfigTypeParam>(weight_only_flag | simt_only_flag | grouped_gemm_flag | enable_blackwell | enable_hopper | fp8_only_flag | fp4_only_flag);
   ORT_ENFORCE(!(enable_blackwell && enable_hopper), "Blackwell and hopper flags are mutually exclusive");
 
-  // When the SM80 FP4 path is enabled, wfp4a16 uses the Ampere fused-dequant grouped GEMM only;
-  // do not offer any TMA WS configs. The WFP4A16 TMA WS kernel uses sm_90a WGMMA, so it can run
-  // only on SM90, and only when it is compiled (MSVC builds omit it). Return no configs instead of
-  // throwing because the runner constructor queries configs before setUseSm80Fp4() is applied.
+  // SM80 fused-dequant and unsupported architectures/builds have no TMA WS
+  // tactics. WFP4A16 TMA WS requires compiled SM90 WGMMA kernels.
   if constexpr (use_wfp4a16) {
     if (moeUseSm80Fp4(sm, use_sm80_fp4) || sm != 90 || !isTmaWarpSpecializedGroupedGemmCompiledForSm(sm)) {
       ORT_LLM_LOG_DEBUG("wfp4a16 TMA WS grouped MoE GEMM is not used for this SM or build");

onnxruntime/contrib_ops/cuda/llm/moe_gemm/moe_kernels.cu

@@ -2216,16 +2216,14 @@ CutlassMoeFCRunner<T, WeightType, OutputType, InputType, ScaleBiasType, Enable>:
     int sm_version,
     ActivationType activation_type,
     bool normalize_routing_weights,
-    bool use_sparse_mixer)
+    bool use_sparse_mixer,
+    bool use_sm80_fp4)
     : sm_(sm_version),
       activation_type_(activation_type),
       normalize_routing_weights_(normalize_routing_weights),
       use_sparse_mixer_(use_sparse_mixer) {
-  auto tactics = getTactics(sm_);
-  if (!tactics.empty()) {
-    gemm1_config_ = tactics[0];
-    gemm2_config_ = tactics[0];
-  }
+  // Apply the routing decision before querying tactics; a provisional TMA query can fail on Ampere.
+  setUseSm80Fp4(use_sm80_fp4);
 }
 
 template <class T, class WeightType, class OutputType, class InputType, class ScaleBiasType, class Enable>

onnxruntime/contrib_ops/cuda/llm/moe_gemm/moe_kernels.h

@@ -415,7 +415,8 @@ class CutlassMoeFCRunner : public CutlassMoeFCRunnerInterface {
   static_assert(std::is_same_v<OutputType, BackBoneType>, "Scale and bias types must match OutputType");
 
  public:
-  CutlassMoeFCRunner(int sm_version, ActivationType activation_type, bool normalize_routing_weights, bool use_sparse_mixer);
+  CutlassMoeFCRunner(int sm_version, ActivationType activation_type, bool normalize_routing_weights, bool use_sparse_mixer,
+                     bool use_sm80_fp4 = false);
 
   ~CutlassMoeFCRunner() override = default;
 

onnxruntime/contrib_ops/cuda/moe/moe_quantization.cc

@@ -621,10 +621,10 @@ QMoE::QMoE(const OpKernelInfo& op_kernel_info) : CudaKernel(op_kernel_info), MoE
 #if defined(ENABLE_FP4) && defined(USE_FP4_QMOE)
       if (is_fp16) {
         m_moe_runner = std::make_unique<CutlassMoeFCRunner<half, __nv_fp4_e2m1, half>>(
-            sm_, activation_type_, normalize_routing_weights_, use_sparse_mixer_);
+            sm_, activation_type_, normalize_routing_weights_, use_sparse_mixer_, enable_fp4_sm80_gemm_);
       } else {
         m_moe_runner = std::make_unique<CutlassMoeFCRunner<__nv_bfloat16, __nv_fp4_e2m1, __nv_bfloat16>>(
-            sm_, activation_type_, normalize_routing_weights_, use_sparse_mixer_);
+            sm_, activation_type_, normalize_routing_weights_, use_sparse_mixer_, enable_fp4_sm80_gemm_);
       }
       // Dense A16 fallback runner for the large-per-expert-M prefill regime, where dequantizing
       // the MXFP4 weights to FP16/BF16 and running the dense grouped GEMM beats the native FP4
@@ -638,10 +638,6 @@ QMoE::QMoE(const OpKernelInfo& op_kernel_info) : CudaKernel(op_kernel_info), MoE
         m_fp4_dense_fallback_runner_ = std::make_unique<CutlassMoeFCRunner<__nv_bfloat16, __nv_bfloat16, __nv_bfloat16>>(
             sm_, activation_type_, normalize_routing_weights_, use_sparse_mixer_);
       }
-      // Capture the SM80-FP4 routing decision (made above from the environment at op-construction
-      // time) into the runner, so inference-time config/tactic selection does not re-read the
-      // environment (which may have changed since the session was created, e.g. in unit tests).
-      m_moe_runner->setUseSm80Fp4(enable_fp4_sm80_gemm_);
       ORT_ENFORCE(!m_moe_runner->getTactics().empty(),
                   "QMoE MXFP4: no grouped GEMM kernel for SM", sm_,
                   " is available in this build. Unset ORT_ENABLE_FP4_CUTLASS_GEMM to use the default path.");

onnxruntime/test/python/transformers/test_qmoe_fp4_cuda.py

@@ -312,7 +312,9 @@ def create_fp4_moe_onnx_graph(
     ]
 
     graph = helper.make_graph(nodes, "QMoE_FP4_Test", graph_inputs, graph_outputs, initializers)
-    model = helper.make_model(graph)
+    model = helper.make_model(
+        graph, opset_imports=[helper.make_opsetid("", 22), helper.make_opsetid("com.microsoft", 1)]
+    )
     return model.SerializeToString()
 
 
@@ -925,8 +927,10 @@ class TestQMoEFP4Sm80SingleWeightCopy(unittest.TestCase):
     ``gemv_fp4_fc*_reads_sm80_layout_``), and then reports ``is_packed = true`` so ORT releases
     the raw ``[E, K, N/2]`` initializers (``release_fp4_raw_weights_``).
 
-    Two properties have to hold for that to be safe, one test each:
+    Session initialization and two properties of the packed weights are covered:
 
+    * session construction applies the SM80 decision before tactic discovery, for both FP16 and
+      BF16 with the default, enabled, and disabled modes -- ``test_sm80_session_initialization``;
     * every dispatch stays numerically equivalent to ORT_FP4_SM80_GEMM=0, which keeps the raw
       initializers and serves decode from a GEMV-native weight copy and prefill from the
       dequant fallback -- ``test_sm80_parity_vs_dequant_fallback``;
@@ -969,6 +973,29 @@ def _build_model(self, hidden_size, inter_size, num_tokens, onnx_dtype, num_expe
         )
         return model, dequantized
 
+    @parameterized.expand(
+        [
+            ("fp16_default", TensorProto.FLOAT16, None),
+            ("fp16_enabled", TensorProto.FLOAT16, "1"),
+            ("fp16_disabled", TensorProto.FLOAT16, "0"),
+            ("bf16_default", TensorProto.BFLOAT16, None),
+            ("bf16_enabled", TensorProto.BFLOAT16, "1"),
+            ("bf16_disabled", TensorProto.BFLOAT16, "0"),
+        ]
+    )
+    def test_sm80_session_initialization(self, _name, onnx_dtype, sm80_mode):
+        """Constructing an MXFP4 session must apply the SM80 decision before tactic discovery."""
+        self._skip_unless_sm80_regime()
+        model, _ = self._build_model(512, 512, 1, onnx_dtype)
+        opts = onnxruntime.SessionOptions()
+        opts.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
+        opts.add_session_config_entry("session.disable_cpu_ep_fallback", "1")
+        with _env_overrides(ORT_FP4_SM80_GEMM=sm80_mode, ORT_ENABLE_FP4_CUTLASS_GEMM="0"):
+            session = onnxruntime.InferenceSession(
+                model, opts, providers=[resolve_cuda_plugin_ep("CUDAExecutionProvider")]
+            )
+        self.assertIn("CUDAExecutionProvider", session.get_providers())
+
     def _run(self, model, sm80_mode, input_tensor, router_logits, onnx_dtype):
         opts = onnxruntime.SessionOptions()
         opts.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL