Repository navigation
[BUG] FP4 quantization on CPU rounds ties toward zero while Metal rounds to even #4753
Description
☑️ I understand it is strictly prohibited to use AI to write issues.
Describe the bug
This bug was identified by comparing floating point number conversion results across a few open-source implementations. On CPU, mx.quantize to FP4 rounds some ties towards zero and some to even while the Metal kernel rounds all ties to even. The OCP MX spec says that implementations must at least support round ties to even (other rounding modes are optional). The CPU implementation should at least be self-consistent. Feel free to close if this is expected/intended.
fp_quantize (mlx/ops.cpp)
| input | expected | observed |
|---|---|---|
0.75 (0x3F400000) |
0x2 = 1 |
0x1 = 0.5 |
1.75 (0x3FE00000) |
0x4 = 2 |
0x3 = 1.5 |
3.5 (0x40600000) |
0x6 = 4 |
0x5 = 3 |
-0.75 (0xBF400000) |
0xA = -1 |
0x9 = -0.5 |
-1.75 (0xBFE00000) |
0xC = -2 |
0xB = -1.5 |
-3.5 (0xC0600000) |
0xE = -4 |
0xD = -3 |
Note that some of the ties are correctly rounded to even: (0.25, 1.25, 2.5, 5)
To Reproduce
Call fp_quantize with the input values provided
Expected behavior
See expected column in the table above
Desktop (please complete the following information):
- OS Version: macOS 26.6.2 (Apple M5 Pro)
- Version: mlx 0.32.4
Activity