You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Raincloud conformance oracle: add repeated/list column support (FixedSizeList, List) #261
RaincloudConformanceIntegrationTest's Parquet oracle (writeOracleCsv/oracleCell) aborts rather than formats any column with maxRepetitionLevel > 0 — i.e. any LIST/repeated Parquet column, which is how Vortex FixedSizeList and List columns round-trip through Parquet. This blocks conformance verification for any slug that stores columns as lists, even though vortex-java's own CsvExporter now renders them correctly (#257).
Affected slugs (currently gap:257 in the matrix, but that issue is closed — the real blocker is this one)
glove-6b-50d, glove-6b-100d, glove-6b-200d — vector column is FixedSizeList[F32, n]
osm-germany-relations — members/tags columns are List[Struct]
Confirmed via
Flipping glove-6b-50d to ok and running the conformance test (after #259 fixed the harness hang) reproduces:
TestAbortedException: oracle cannot format repeated list column: vector
from writeOracleCsv (RaincloudConformanceIntegrationTest.java).
Expected behavior
The oracle should recursively read repeated/nested Parquet groups (hardwood's RowReader/ColumnSchema repetition/definition levels) and render them as JSON arrays with the same rules CsvExporter.jsonValue/jsonArray already use, so the two sides can be compared value-for-value:
FixedSizeList[F32, 50] → [0.1,0.2,...,0.05]
List[Struct] → [{"ref":1,"role":""},...]
Raincloud matrix
Flip the 4 slugs above from gap:257 to gap:<this issue> in the same change that adds this test infra, then to ok once oracle output is verified to actually match CsvExporter's output.
Activity
test: repoint glove/osm-germany-relations matrix entries from gap:257…
fix: Raincloud oracle gains repeated/list column support (#261)