Repository navigation

All pull requests

Labels (17)Milestones (1)New pull request

Open

(97)

Closed

(1,086)

fix(mcq): stop scoring judge failures with a random option
#1802FazeelUsmani opened Oct 23, 2026
fix(unisvg): normalise the prediction when matching categories
#1801rudycelekli opened Oct 23, 2026Contributor
fix(phyx): take the concluding option letter in multiple-choice answers
#1800rudycelekli opened Oct 23, 2026Contributor
fix(anls): divide by the length of the strings the distance was measured on
#1799rudycelekli opened Oct 23, 2026Contributor
fix(mmmu): write the submission JSON for every PRED_FORMAT
#1798rudycelekli opened Oct 23, 2026Contributor
fix(olympiadbench): restore sympy_sub_pi so expression comparison runs
#1797rudycelekli opened Oct 23, 2026Contributor
fix(claude): send the configured temperature in the request payload
#1796rudycelekli opened Oct 23, 2026Contributor
fix(yorn): extract markdown-emphasised yes/no verdicts
#1795rudycelekli opened Oct 23, 2026Contributor
fix(verifier): read word-form judge replies instead of mis-scoring them
#1794rudycelekli opened Oct 23, 2026Contributor
fix(mcq): restore the missing commas in the answer-prefix strip lists
#1793rudycelekli opened Oct 23, 2026Contributor
fix(chartqapro): score Fact Checking and Multi Choice by exact match
#1792rudycelekli opened Oct 23, 2026Contributor
fix(gsm8k-v): extract numbers written with thousands separators
#1791rudycelekli opened Oct 23, 2026Contributor
fix(mme-reasoning): download the TSV file, not its Hugging Face page
#1790David-Wu1119 opened Oct 23, 2026
1
fix(mathverse): report overall accuracy across problem versions
#1789David-Wu1119 opened Oct 23, 2026
1
fix(mme-reasoning): restore imports dropped by the star-import migration
#1788David-Wu1119 opened Oct 23, 2026
1
fix(vqa): restore the official period stripping in answer normalization
#1787David-Wu1119 opened Oct 23, 2026
1
fix(inference): reuse MMBench V11 JSON prediction tables
#1784rudycelekli opened Oct 20, 2026Contributor
[Fix] Keep missing and substring-only MCQ predictions unresolved
#1773rudycelekli opened Oct 16, 2026Contributor
[Benchmark] Add E.T. Bench for event-level time-sensitive video understanding
#1772aliasgerovs opened Oct 15, 2026Contributor
1
[Benchmark] Add QVHighlights moment retrieval and highlight detection
#1770aliasgerovs opened Oct 14, 2026Contributor
1
fix(images): normalize EXIF orientation before conversion
#1764rudycelekli opened Oct 13, 2026Contributor
fix(images): flatten transparent diagrams on white
#1756rudycelekli opened Oct 13, 2026Contributor
[Benchmark] Add Spatial Scene Bench dataset
#1750selfishout opened Oct 10, 2026
fix(images): expand one listed path for multiple images
#1745rudycelekli opened Oct 10, 2026Contributor
Use canonical V2PBench video paths for decoding and cache validation
#1735rudycelekli opened Oct 6, 2026Contributor

Pull requests