You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We currently do a lot of type coercion in the physical planning phase. I would like the ability to have type coercion and validation happen in the logical plan.
This would benefit projects such as Dask SQL that are using the SQL query planner.
Describe the solution you'd like
Add a new TypeCoercion rule that adds explicit CASTs where appropriate.
Activity
liukun4515 commentedon Aug 24, 2022
@andygrove
I go through the spark code about the type coercion and find https://github.com/apache/spark/blob/55c3347c48f93a9c5c5c2fb00b30f838eb081b7f/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala#L323
Spark has put the type coercion to the analysis phase, but DF don't have this phase.
andygrove commentedon Sep 7, 2022
This was implemented in #3222