Repository navigation
Open
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers
Description
vortex-jdbc only has JdbcImporter/JdbcImportOptions; csv and parquet each have an importer and an exporter. Add JdbcExporter + JdbcExportOptions for symmetry: read a Vortex file and move its rows into a table in a JDBC database over a caller-supplied Connection.
API sketch
JdbcExportOptions opts = JdbcExportOptions.defaults()
.withBatchSize(5_000)
.withBinder(JdbcBinder.standard()
.with(DType.Extension.class, (ps, i, value) -> ps.setObject(i, value)));
JdbcExporter.export(vortexPath, connection, "ohlc", opts);- Options are a final class with a private constructor,
defaults()andwithXxxcopies, as everywhere else. The target table is a required argument ofexport, not an option (confirm thatCsvExporter/ParquetExportertake their target the same way). - Binder (callback): a small functional interface that sets one value on a
PreparedStatement.JdbcBinder.standard()covers primitives, strings, decimals, booleans and dates. A user overrides only the types their database needs special handling for (Postgres arrays,setObjectfor timestamps). - DDL helper, not automatic: a
DTypeto SQL type function pluscreateTableSql(name, dtype)returns aCREATE TABLEstring. The exporter never creates or alters a table on its own; creation is opt-in.
Behavior
- Appends to an existing table with
INSERT INTO t (cols...) VALUES (?, ...)through aPreparedStatement. Nulls follow the validity mask. - One transaction for the whole export: remember the connection's auto-commit setting,
setAutoCommit(false),addBatch()per row,executeBatch()everybatchSizerows,commit()at the end,rollback()and rethrow on failure, and restore auto-commit in afinally. The connection belongs to the caller. A failed export leaves nothing behind. ProgressListener, as in the other exporters.
Open design questions
- Reverse type mapping
DTypeto SQL type (the inverse ofSqlTypeToDType); it varies by database. - Create-if-missing vs truncate vs fail-if-exists: all opt-in, append is the default.
- A
withCommitEvery(n)option for very large exports, if a long-lived transaction proves a problem. Not in the first cut.
Not needed for the jdbc:vortex: driver work (#566). No demand yet, so this waits until someone needs it.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgood first issueGood for newcomersGood for newcomers
Activity