Skip to content

Panics are silently ignored in parallel execution (spawn_execution) #3104

Closed

Description

@ovr

Describe the bug

DF uses tokio::spawn in spawn_execution function. It uses a channel under the hood to transfer data from tasks back to the output stream. Right now, it doesn't handle panic, which causes strange behavior when the tokio show panics in logs and the output is empty.

To catch panics, it's required to mark some places in the code as AssertUnwindSafe + catch it in spawn_execution function. Any another idea?

To Reproduce

  1. Introduce a new UDF function that panics
  2. Use sort to parallelize (use spawn_execution)

Expected behavior

Probably, it should return DataFushionError.

cC @alamb @andygrove

Activity

  1. added
    bugSomething isn't working
    on Aug 11, 2022
  2. alamb commented on Aug 11, 2022

    @alamb
    Contributor

    I think returning a DataFusionError sounds like a very reasonable thing to do

    When a tokio::task panics, it will return a JoinError as an error code -- perhaps somewhere datafusion does a tokio::task::spawn but then doesn't check for an error 🤔

  3. modified the milestone: 14.0.0 on Oct 30, 2022
  4. added a commit that references this issue on May 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions