Skip to content

Core: Allow limiting upsert update detection to selected columns - #4454

Open
Anurag-S1ngh wants to merge 1 commit into
apache:mainfrom
Anurag-S1ngh:feat/3598-upsert-difference-cols
Open

Anurag-S1ngh wants to merge 1 commit into
apache:mainfrom
Anurag-S1ngh:feat/3598-upsert-difference-cols

Conversation

@Anurag-S1ngh

Copy link
Copy Markdown
Contributor

Rationale for this change

Upsert compared every non-key column to decide whether a matched row changed. For wide or nested rows this is expensive, and callers often already know which column signals a change, such as a hash column.

Add an optional difference_cols parameter to Table.upsert, Transaction.upsert, and get_rows_to_update. When set, only those columns are compared. Unknown columns raise a ValueError, and the selection must contain at least one non-key column.

The parameter is added at the end of the signature to keep positional calls working.

Are these changes tested?

Yes. Unit tests cover comparing only the selected columns, unchanged selected columns, unknown columns, and key-only selections. An end-to-end upsert test confirms that a change in an unselected column does not update the row. tests/table passes.

Are there any user-facing changes?

Yes. upsert accepts a new optional difference_cols argument.

Upsert compared every non-key column to decide whether a matched row
changed, which is expensive for wide or nested rows. Add an optional
difference_cols argument to get_rows_to_update and both upsert entry
points so callers can compare only the columns that signal a change, such
as a hash column.

The argument is appended to the signatures to keep positional calls
working. Unknown columns and key-only selections raise a ValueError.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant