You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Panics are silently ignored in parallel execution (spawn_execution) #3104
DF uses tokio::spawn in spawn_execution function. It uses a channel under the hood to transfer data from tasks back to the output stream. Right now, it doesn't handle panic, which causes strange behavior when the tokio show panics in logs and the output is empty.
To catch panics, it's required to mark some places in the code as AssertUnwindSafe + catch it in spawn_execution function. Any another idea?
I think returning a DataFusionError sounds like a very reasonable thing to do
When a tokio::task panics, it will return a JoinError as an error code -- perhaps somewhere datafusion does a tokio::task::spawn but then doesn't check for an error 🤔
Activity
alamb commentedon Aug 11, 2022
I think returning a
DataFusionErrorsounds like a very reasonable thing to doWhen a
tokio::taskpanics, it will return aJoinErroras an error code -- perhaps somewhere datafusion does atokio::task::spawnbut then doesn't check for an error 🤔Propagate panics
Propagate panics
Fix panic propagation in `CoalescePartitions`, consolidates panic pro…