Skip to content

Add baseline metrics for remaining execution plan nodes #1019

Closed

Description

@alamb

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

We want basic understanding of where a plan's time is spent and in what operators. See #866 for more details

This information has been added (#866) for intermediate operators but not for low level readers for parquet, json, csv, and avro.

Describe the solution you'd like
Add "baseline" metrics (start time, stop time, output rows, and elapsed_compute) for the data source ExecutionPlans:

  1. ParquetExec
  2. NdJsonExec
  3. NdJsonExec
  4. AvroExec
  5. CsvExec
  6. CrossJoinExec
  7. EmptyExec
  8. MemoryExec

Describe alternatives you've considered
None

Additional context
It should be straightforward to follow the model for adding statistics on the following PRs:

Activity

  1. Dandandan commented on Sep 17, 2021

    @Dandandan
    Contributor

    What about CsvExec, CrossJoinExec, EmptyExec, MemoryExec?

  2. alamb commented on Sep 17, 2021

    @alamb
    ContributorAuthor

    Good eyes @Dandandan -- including them in the description

  3. changed the title [-]Add baseline metrics for ParquetExec, NdJsonExec, NdJsonExec, and AvroExec[/-] [+]Add baseline metrics for remaining execution plan nodes[/+] on Sep 17, 2021
  4. alamb commented on Nov 18, 2022

    @alamb
    ContributorAuthor

    I don't plan any more work on this ticket -- we can add metrics to other operators if there is a specific need

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions