Skip to content

feat: Migrate to PhysicalExprAdapterFactory #2058

Closed

Description

@comphead

What is the problem the feature request solves?

DataFusion is considering to deprecate SchemaAdapter which is intensively used in Comet.

Need to investigate migration to PhysicalExprAdapterFactory

The SchemaAdapter API will be removed from ParquetSource in a future release. Use PhysicalExprAdapterFactory API instead. See apache/datafusion#16800 for discussion and https://datafusion.apache.org/library-user-guide/upgrading.html#datafusion-49-0-0 for upgrade instructions

Describe the potential solution

No response

Additional context

No response

Activity

  1. self-assigned this
    on Aug 1, 2025
  2. mbutrovich commented on Aug 5, 2025

    @mbutrovich
    Contributor

    I've run into some issues with this. In particular, I don't see a way to handle default values that don't have a filter on them:

    https://github.com/apache/datafusion/blob/40015a8345c6c20f439cb3829897d20ebce70cc9/datafusion/datasource-parquet/src/opener.rs#L256

    Even the example says that default values without a filter on them should rely on the SchemaAdapter but we're seeing a ton of WARN messages that the API is going away.

    https://github.com/apache/datafusion/blob/6d0073465a59d743e42e6fdba690e40ced8a792a/datafusion-examples/examples/default_column_values.rs#L67

  3. added this to the 0.10.0 milestone on Aug 5, 2025
  4. modified the milestones: 0.12.0, 0.13.0 on Dec 1, 2025
  5. modified the milestones: 0.13.0, 0.14.0 on Jan 30, 2026
  6. andygrove commented on Mar 12, 2026

    @andygrove
    Member

    @comphead @mbutrovich can this issue be closed now?

  7. mbutrovich commented on Mar 12, 2026

    @mbutrovich
    Contributor

    @comphead @mbutrovich can this issue be closed now?

    Yep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions