Repository navigation
refactor: Refactor spark width bucket signature away from user defined - #19065
Conversation
Jefffrey
left a comment
There was a problem hiding this comment.
Thanks for picking this up, left some suggestions
| make_scalar_function(width_bucket_kern, vec![])(&args.args) | ||
| let [value, minv, maxv, buckets] = take_function_args(self.name(), &args.args)?; | ||
|
|
||
| let arrays = vec![ | ||
| value.to_array(args.number_rows)?, | ||
| minv.to_array(args.number_rows)?, | ||
| maxv.to_array(args.number_rows)?, | ||
| buckets.to_array(args.number_rows)?, | ||
| ]; | ||
|
|
||
| width_bucket_kern(&arrays).map(ColumnarValue::Array) |
There was a problem hiding this comment.
Was there a need for this change?
There was a problem hiding this comment.
Correct me if my understanding is wrong, I only swapped out make_scalar_function so I could call take_function_args up front, once the UDF no longer uses Signature::user_defined, we need the shared take_function_args guard to produce the standard “failed to match any signature” planner error. If we want to keep the helper, I can move the take_function_args call into the closure, but the behavior change was just to hook into the common arity handling.
There was a problem hiding this comment.
The signature should now guard that for us; take_function_args would only be used as an ergonomic way of extracting the args required. The fact that it errors if the arg count is wrong would just be an extra layer of defense but not the main defense, as only if something went wrong with the signature related code could that error path ever occur.
There was a problem hiding this comment.
Okay, Should I revert this?
| } | ||
| } | ||
|
|
||
| fn width_bucket_signature(arg: &Coercion, bucket: &Coercion) -> TypeSignature { |
There was a problem hiding this comment.
I'd prefer to inline this directly
Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com>
|
@Jefffrey The |
|
Thanks @kumarUjjawal |
Which issue does this PR close?
Signature::user_definedfor normal functions #12725Rationale for this change
What changes are included in this PR?
user_definedin spark width_bucketAre these changes tested?
Are there any user-facing changes?