Repository navigation
Conversation
| table = _open_schema_table(args) | ||
| try: | ||
| status = table.file_io.get_file_status(args.file) | ||
| if status.type != pafs.FileType.File: |
There was a problem hiding this comment.
[Med]
list-files and manifest emit _metadata_file_path(table, entry), which preserves file: schemes and percent-escaped partition directories (e.g., file:/warehouse/db.db/t/pt=a%2Fb/bucket-0/data-0.parquet). Passing args.file directly to table.file_io causes LocalFileIO._to_file() to URL-decode %2F into / and fail with FileNotFoundError.
We should convert the path with to_file_io_path(args.file) before calling file_io (matching DataFileMeta.physical_path()):
from pypaimon.utils.path import to_file_io_path
io_path = to_file_io_path(args.file)
status = table.file_io.get_file_status(io_path)
file_format = _data_file_format(basename(status.path)).lower()
if file_format != 'parquet':
raise ValueError(
f"Unsupported data file format '{file_format}' for '{args.file}'; only parquet is supported.")
with table.file_io.new_input_stream(io_path) as stream:There was a problem hiding this comment.
@Akash3121 Thanks for reviewing! Agreed. I've added a local change to preserve the emitted metadata path at both FileIO calls, with regression tests for the public list-files/manifest-to-file round trip on escaped partitions and for ordinary encoded file URIs.
| rollback_target = rollback_parser.add_mutually_exclusive_group(required=True) | ||
| rollback_target.add_argument( | ||
| '--snapshot', type=int, default=None, | ||
| '--snapshot-id', type=int, default=None, |
There was a problem hiding this comment.
Unifying this option name with --snapshot-id used by get, snapshot, list-manifests, and list-files makes the CLI much more consistent, and argparse sets args.snapshot_id automatically.
Two minor suggestions here:
- Keep
--snapshotas a backward-compatible alias ('--snapshot-id', '--snapshot') so existing scripts usingpaimon table rollback ... --snapshot <id>don't break. - Could this also use
type=_positive_intlike the--snapshot-idoption onget(line 1471),snapshot(line 1501),list-manifests(line 1531), andlist-files(line 1564)?
| '--snapshot-id', type=int, default=None, | |
| '--snapshot-id', '--snapshot', type=int, default=None, |
There was a problem hiding this comment.
@Akash3121 Thanks for reviewing! I've added --snapshot as an explicit alias and switched to _positive_int.
JingsongLi
left a comment
There was a problem hiding this comment.
Requirement fit: SUPPORTED. Implementation: FINDINGS.
I independently confirmed the existing path round-trip finding on bdd9b2a142 with a real filesystem catalog and committed Parquet data: warehouse configured as a file URI, partition value a/b, then public list-files --format json followed by file --file <exact emitted file_path>. The latter exits 1 because the literal pt=a%2Fb directory is URL-decoded; inspecting the same file using its physical local path succeeds. This breaks the documented diagnostic workflow. Please preserve the metadata path semantics at both FileIO calls and add this public command-to-command regression, while retaining support for ordinary encoded filesystem URIs.
90 CLI tests passed (108 subtests); project flake8 and git diff --check passed. The explicit legacy rollback --snapshot 1 command also succeeded: argparse currently accepts it as an unambiguous abbreviation, so I did not find a compatibility failure there. No additional independent code finding.
|
@JingsongLi Thanks for reviewing! Agreed. I’ve updated both FileIO calls to preserve the metadata path semantics, with regression tests for the public |
Purpose
Implements the file commands proposed in #11073 (https://github.com/apache/paimon/issues/11073).
Adds read-only file inspection to the PyPaimon CLI:
These commands inspect metadata without scanning data records or modifying the table. The file command does not query snapshots or manifests. CLI documentation and output examples are included.
Also renames table rollback --snapshot to --snapshot-id for consistency with the other table commands.
Tests