Skip to content

datafusion-proto crate feature json is broken #15157

Closed

Description

@alamb

Describe the bug

I am not sure what the json feature is supposed to do in datafusion-proto but it is broken

To Reproduce

cargo check --profile ci --all-targets --no-default-features -p datafusion-proto --features=json

Results in pages of errors like this

error[E0277]: the trait bound `datafusion_proto_common::Schema: Serialize` is not satisfied
    --> datafusion/proto/src/generated/pbjson.rs:159:55
     |
159  |             struct_ser.serialize_field("inputSchema", v)?;
     |                        ---------------                ^ the trait `Serialize` is not implemented for `datafusion_proto_common::Schema`
     |                        |
     |                        required by a bound introduced by this call
     |
     = note: for local types consider adding `#[derive(serde::Serialize)]` to your `datafusion_proto_common::Schema` type
     = note: for types from other crates check whether the crate offers a `serde` feature flag
     = help: the following other types implement trait `Serialize`:
               &'a T
               &'a mut T
               ()
               (T,)
               (T0, T1)
               (T0, T1, T2)
               (T0, T1, T2, T3)
               (T0, T1, T2, T3, T4)
             and 303 others
note: required by a bound in `serde::ser::SerializeStruct::serialize_field`
    --> /usr/local/cargo/registry/src/index.crates.io-1949cf8c6b5b557f/serde-1.0.218/src/ser/mod.rs:18[66](https://github.com/apache/datafusion/actions/runs/13788882508/job/38563713616?pr=15156#step:6:67):21
     |
1864 |     fn serialize_field<T>(&mut self, key: &'static str, value: &T) -> Result<(), Self::Error>
     |        --------------- required by a bound in this associated function
1865 |     where
1866 |         T: ?Sized + Serialize;
     |                     ^^^^^^^^^ required by this bound in `SerializeStruct::serialize_field`

error[E0277]: the trait bound `datafusion_proto_common::Schema: Deserialize<'_>` is not satisfied
    --> datafusion/proto/src/generated/pbjson.rs:320:51
     |
320  | ...                   input_schema__ = map_.next_value()?;
     |                                             ^^^^^^^^^^ the trait `Deserialize<'_>` is not implemented for `datafusion_proto_common::Schema`
     |
     = note: for local types consider adding `#[derive(serde::Deserialize)]` to your `datafusion_proto_common::Schema` type
     = note: for types from other crates check whether the crate offers a `serde` feature flag
     = help: the following other types implement trait `Deserialize<'de>`:
               &'a [u8]
               &'a std::path::Path
               &'a str
               ()
               (T,)
               (T0, T1)
               (T0, T1, T2)
               (T0, T1, T2, T3)
             and 4[80](https://github.com/apache/datafusion/actions/runs/13788882508/job/38563713616?pr=15156#step:6:81) others
     = note: required for `std::option::Option<datafusion_proto_common::Schema>` to implement `Deserialize<'de>`
note: required by a bound in `next_value`

See more here https://github.com/apache/datafusion/actions/runs/13788882508/job/38563713616?pr=15156

Expected behavior

I expect the feature to work, or to be removed

Additional context

Found while working on

Activity

  1. timsaucer commented on Mar 11, 2025

    @timsaucer
    Member

    Recommend adding to this issue that we will want to re-enable the Check datafusion-proto (json) as part of closing the issue

  2. deleted a comment from piyushgarg5021 on Mar 11, 2025
  3. timsaucer commented on Mar 11, 2025

    @timsaucer
    Member

    Deleting comments that appear to be from an AI model.

  4. Shreyaskr1409 commented on Mar 12, 2025

    @Shreyaskr1409
    Contributor

    https://github.com/apache/datafusion/pull/15185/files#r1991797793
    #15089 (comment)

    @irenjj maybe it is due to this issue, we are facing problems with JsonSink #15089 , as I said in one of my comments, no matter what I did, JsonSink is not getting used whereas CsvSink and others do get used. Maybe we can look into that issue after this issue is solved.

  5. irenjj commented on Mar 13, 2025

    @irenjj
    Contributor

    https://github.com/apache/datafusion/pull/15185/files#r1991797793 #15089 (comment)

    @irenjj maybe it is due to this issue, we are facing problems with JsonSink #15089 , as I said in one of my comments, no matter what I did, JsonSink is not getting used whereas CsvSink and others do get used. Maybe we can look into that issue after this issue is solved.

    Thanks @Shreyaskr1409 , explain works in #15185(even though I wasn't able to construct a non-empty file_group case), I'd like to try to add more tests after this issue resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions