You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
update_spec: removing a partition field and adding one with the same name raises "Cannot add duplicate partition field name" #4306
Replacing a partition field while keeping its name, for example to change the number of buckets, fails:
frompyiceberg.catalog.sqlimportSqlCatalogfrompyiceberg.schemaimportSchemafrompyiceberg.transformsimportBucketTransformfrompyiceberg.typesimportLongType, NestedFieldcatalog=SqlCatalog("default", uri="sqlite:///:memory:", warehouse="file:///tmp/warehouse")
catalog.create_namespace("db")
schema=Schema(NestedField(1, "x", LongType()), NestedField(2, "y", LongType()))
table=catalog.create_table("db.events", schema)
withtable.update_spec() asupdate:
update.add_field("y", BucketTransform(16), "y_shard")
# Change the number of buckets but keep the partition namewithtable.update_spec() asupdate:
update.remove_field("y_shard")
update.add_field("y", BucketTransform(32), "y_shard")
ValueError: Cannot add duplicate partition field name: y_shard
UpdateSpec.add_field checks whether the id of the new field is in the set of removed fields, instead of the existing field with that name, so the pending removal is never taken into account. Java's BaseUpdatePartitionSpec.addField checks the existing field, allows the add when that field is being deleted, and renames the deleted field to <name>_<fieldId>.
Activity