Skip to content

array function allows different data types #3123

Closed

Description

@alamb

Describe the bug
Both Spark and Postgres allow arrays of elements that have the same type -- e.g. Spark: https://spark.apache.org/docs/2.1.0/api/java/index.html?org/apache/spark/sql/types/ArrayType.html

The current array() function in datafusion creates a FixedSizeList arrow array. However, the FixedSizedList in arrow is more general as it allows the elements to be any type

Datafusion currently doesn't coerce the types to a common type

To Reproduce

DataFusion CLI v10.0.0
❯ select array(1,2,'foo');
+--------------------------------------+
| array(Int64(1),Int64(2),Utf8("foo")) |
+--------------------------------------+
| [1, 2, foo]                          |
+--------------------------------------+

Expected behavior
An error, like in postgres:

               ^
alamb=# select array[1,2,'foo'];
ERROR:  invalid input syntax for type integer: "foo"

Additional context
Found while working on #3122

The array function was originally introduced by @jorgecarleitao in https://github.com/apache/arrow/pull/8102/files

Activity

  1. comphead commented on Aug 23, 2022

    @comphead
    Contributor

    I will try to solve it

  2. comphead commented on Aug 23, 2022

    @comphead
    Contributor

    @alamb looks like it was already fixed in #2194

    DataFusion CLI v11.0.0
    ❯ select array[1, 2, 'foo'];
    NotImplemented("Arrays with different types are not supported: {Int64, Utf8}")
    
  3. alamb commented on Aug 31, 2022

    @alamb
    ContributorAuthor

    👍 thanks @comphead -- not sure what I was thinking about

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions