Skip to content

Avoid use of Signature::user_defined for normal functions #12725

Open

Description

@findepi

Originally posted by @alamb in #12698 (comment)

Can we make it a goal that not to require Signature::user_defined to implement useful things?

Yes, I think this should be the goal

The usecase for Signature::user_defined that I recall was to implement coercion rules for very specific functions (like coalesce)

Extend signatures and function resolution so that Signature::user_defined is not needed.

Activity

  1. jayzhan211 commented on Oct 3, 2024

    @jayzhan211
    Contributor

    Before #12622 is completed, we can introduce signature like Numeric / String / Float / Integer and so on that is used in places more than 1, others use Signature::user_defined by default

  2. self-assigned this
    on Nov 12, 2025
  3. Jefffrey commented on Nov 27, 2025

    @Jefffrey
    Contributor

    Current status (as of main fb14d7c):

    Should be doable with current API

    Aiming to work through these.

    Further investigation (API changes?)

    I think these are possible, hopefully extent of API changes needed are quite limited in scope.

    • repeat - Specify u64/i64 2nd arg in where 1st arg is array
    • map_extract - Need something similar to array signature for map, where can specify a map array + its key field
    • distance - Coerce lists to ensure same inner type

    Probably stay as user defined

    Would be nice if below ones could be refactored away from user defined but likely needs wider API changes to make it possible?

    • coalesce
    • nvl2
    • greatest/least
    • concat
    • (spark) if
    • (spark) elt
  4. codetyri0n commented on Nov 27, 2025

    @codetyri0n
    Contributor

    can help out with a few here @Jefffrey

  5. 3 remaining items

  6. kumarUjjawal commented on Dec 3, 2025

    @kumarUjjawal
    Contributor

    @Jefffrey I have opened a PR for spark width_bucket.

  7. codetyri0n commented on Dec 6, 2025

    @codetyri0n
    Contributor

    are you working on hex as well? @Jefffrey - if not I can pick it up

  8. Jefffrey commented on Dec 6, 2025

    @Jefffrey
    Contributor

    are you working on hex as well? @Jefffrey - if not I can pick it up

    I'm not currently looking at it, feel free to pick it up.

  9. kumarUjjawal commented on Dec 29, 2025

    @kumarUjjawal
    Contributor

    I was looking at min_max and found that min/max could be a valid use case for Signature::user_defined for now:

    When I changed from Signature::user_defined to Signature::any(1):

    • Basic min/max operations work fine
    • Dictionary-typed columns fail (wrong results + panics)

    The coerce_types() method is only invoked when the signature is TypeSignature::UserDefined. With Signature::any(1), our coerce_types() for dictionary unwrapping is never called, causing dictionary values to be passed directly to accumulators that can't handle them.

    @Jefffrey did you get the time to look at it?

  10. Jefffrey commented on Dec 29, 2025

    @Jefffrey
    Contributor

    I was looking at min_max and found that min/max could be a valid use case for Signature::user_defined for now:

    When I changed from Signature::user_defined to Signature::any(1):

    * Basic min/max operations work fine
    
    * Dictionary-typed columns fail (wrong results + panics)
    

    The coerce_types() method is only invoked when the signature is TypeSignature::UserDefined. With Signature::any(1), our coerce_types() for dictionary unwrapping is never called, causing dictionary values to be passed directly to accumulators that can't handle them.

    @Jefffrey did you get the time to look at it?

    Perhaps for simplicity we'll keep it as it us until we have better signature support for dictionaries; see: #19458

  11. removed their assignment
    on Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions