Skip to content

refactor: derive IcebergDataSource column names from its schema - #112

Merged
gabotechs merged 2 commits into
apache:mainfrom
NoahKusaba:refactor/data-source-cleanups
Nov 18, 2026
Merged

gabotechs merged 2 commits into
apache:mainfrom
NoahKusaba:refactor/data-source-cleanups

Conversation

@NoahKusaba

@NoahKusaba NoahKusaba commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

IcebergDataSource stores its column names in a projection field, but they are always the field names of its schema. Two copies of the same state can drift apart.

What changes are included in this PR?

  • Removes the projection field and the projection() accessor; the column names come from schema().

Are there any user-facing changes?

Yes: use schema() instead of projection().

Are these changes tested?

Yes, by the existing scan tests, updated to check the schema instead of projection().

…p projection

- Rename the fallible IcebergDataSourceBuilder::build to try_build.
- Remove IcebergDataSource's projection field and accessor. They always
  held the field names of its schema, which the source now derives when
  it needs them.

@mbutrovich mbutrovich left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NoahKusaba! Deriving the column names from schema removes state that could drift, and the change to the scan and the tests looks right to me.

Comment thread crates/datafusion/src/read/scan.rs Resolved
A builder whose only terminal method can fail calls it build() and
returns Result, as DataFusion's and iceberg's builders do.
@NoahKusaba NoahKusaba changed the title refactor: rename IcebergDataSourceBuilder::build to try_build and drop projection refactor: derive IcebergDataSource column names from its schema Nov 18, 2026

@gabotechs gabotechs left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks @NoahKusaba!

@gabotechs
gabotechs merged commit a7e5f55 into apache:main Nov 18, 2026
5 checks passed
@NoahKusaba
NoahKusaba deleted the refactor/data-source-cleanups branch November 18, 2026 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants