Skip to content

Proposal : Benchmarking Improvement - Aligning measure_runtime with time_fn in time_utils.py #4747

Open

Description

@manig1729

☑️ I understand it is strictly prohibited to use AI to write issues.

I want to improve the shared python benchmark timing helper by making measure_runtime sync the returned MLX values like in time_fn

If a callable returns an unevaluated MLX value, timing without mx.eval could measure host-side dispatch rather than the actual completed device work

Configurable warmup and iteration counts would also improve benchmarking by making it a little more modular - I will preserve the current defaults

Please let me know whether this scope would be useful, or if you would prefer a different approach before I begin implementation

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions