You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Open your dataform.json file and add or modify the defaultConfig section:
{
"defaultConfig": {
"bigquery": {
"priority": "BATCH"
}
}
}
This sets BATCH as the default priority for all BigQuery operations in your Dataform project.
Configure in config Block (Granular Control):
For specific SQLX scripts, you can set the priority directly in the config block:
Sure, but let me repeat my question: how do you run Dataform DAGs in production? Do you use Managed Dataform in GCP or you manage Dataform yourself and use the Dataform CLI?
We're working on the support of configuring batch priority in the GCP Dataform at the moment. Initially it'll be supported as configuration when starting a new workflow invocation (as follow up we may add default configuration in dataform.json / workflow_settings.yaml)
Activity
kolina commentedon Aug 18, 2025
Do you want to have this ability in the GCP Dataform or in the Dataform CLI?
nicolascelys-mb commentedon Aug 18, 2025
it will be fabulous to achieve this.
Edit dataform.json Configuration:
Open your dataform.json file and add or modify the defaultConfig section:
{
"defaultConfig": {
"bigquery": {
"priority": "BATCH"
}
}
}
This sets BATCH as the default priority for all BigQuery operations in your Dataform project.
For specific SQLX scripts, you can set the priority directly in the config block:
config {
type: "table",
bigquery: {
priority: "BATCH"
}
}
-- Your SQL code here
This allows you to control which scripts use BATCH priority.
kolina commentedon Aug 19, 2025
Sure, but let me repeat my question: how do you run Dataform DAGs in production? Do you use Managed Dataform in GCP or you manage Dataform yourself and use the Dataform CLI?
nicolascelys-mb commentedon Aug 19, 2025
I use GCP
kolina commentedon Aug 21, 2025
We're working on the support of configuring batch priority in the GCP Dataform at the moment. Initially it'll be supported as configuration when starting a new workflow invocation (as follow up we may add default configuration in
dataform.json/workflow_settings.yaml)nicolascelys-mb commentedon Oct 10, 2025
As a follow up, in GCP in the workflows, now has an option to unmark and leave the execution with batch priority. Pretty cool
nicolascelys-mb commentedon Oct 10, 2025
I believe the documentation in Datafom of GCP should highlight this.