Skip to content
This repository was archived by the owner on Oct 2, 2026. It is now read-only.
This repository was archived by the owner on Oct 2, 2026. It is now read-only.

Native Bindings is x10 Slower #13

Open
Open
Native Bindings is x10 Slower#13

Description

@Alois-xx
Alois-xx
opened on Jul 28, 2019

I have tried to add SimdJsonSharp to my serializer perfomrance test suite. See https://github.com/Alois-xx/SerializerTests

When you compile it and let it run on .NET Core Preview 7

D:\Source\git\SerializerTests\bin\Debug\netcoreapp3.0>SerializerTests.exe -test combined -serializer Utf8JsonSerializer,SimdJsonSharpSerializer,SimdJsonSharpSerializerN

Serializer      Objects "Time to serialize in s"        "Time to deserialize in s"      "Size in bytes" FileVersion     Framework       ProjectHome     DataFormat      FormatDetails   Supports Versioning
SimdJsonSharpSerializer 1000000 0.1310  0.116   35777803        1.5.0.0 .NET Core 3.0.0-preview7-27912-14       https://github.com/EgorBo/SimdJsonSharp based on https://github.com/lemire/simdjson     Text    Json    No
SimdJsonSharpSerializerN        1000000 0.1117  3.139   35777803        1.7.0.0 .NET Core 3.0.0-preview7-27912-14       https://github.com/EgorBo/SimdJsonSharp based on https://github.com/lemire/simdjson     Text    Json    No
Utf8JsonSerializer      1000000 0.1135  0.330   35777803        1.3.7   .NET Core 3.0.0-preview7-27912-14       https://github.com/neuecc/Utf8Json      Text    Json    Yes

I find the native version needs 3s vs 0.1s for the managed version.
grafik

When lookint at the data in PerfView I find most time is spent in paserJson. Why this is so costly I cannot tell but it looks wasteful. Can you take a look why this is so much slower? Am I using the library the wrong way?

Activity

  1. EgorBo commented on Jul 31, 2019

    @EgorBo
    EgorBo
    on Jul 31, 2019
    Owner

    @Alois-xx there is also a managed version which doesn't have pinvoke overhead (but has some C# vs C overhead).
    Anyway simdjsonsharp is good for large files (>200kb)

  2. Alois-xx commented on Aug 1, 2019

    @Alois-xx
    Alois-xx
    on Aug 1, 2019
    Author

    Yes I know about the managed version. Why should one use the native bindings if they are 10x times slower? Does it make sense to interop into such a slow solution if the interop overhead is so huge? The test with 1 million objects results in a ca. 30 MB Json file. I would have expected to get a much higher speedup compared to a normal serializer like Utf8Json.

  3. EgorBo commented on Aug 1, 2019

    @EgorBo
    EgorBo
    on Aug 1, 2019
    Owner

    @Alois-xx can you share some steps how to test that 30Mb json file, a repro project would be perfect.

  4. Alois-xx commented on Aug 1, 2019

    @Alois-xx
    Alois-xx
    on Aug 1, 2019
    Author

    Clone https://github.com/Alois-xx/SerializerTests. Compile it and see the command line help. It is a serializer performance testing framework to make it easy to compare different serializers.

    D:\Source\git\SerializerTests\bin\Release\netcoreapp3.0>SerializerTests.exe
SerializerTests is a serializer performance testing framework to evaluate and compare different serializers for .NET by Alois Kraus
SerializerTests [-Runs dd] -test [serialize, deserialize, combined, firstCall] [-reftracking] [-maxobj dd]
 -N 1,2,10000    Set the number of objects to de/serialize which is repeated -Runs times to get stable results.
 -Runs           Default is 5. The result is averaged where the first run is excluded from the average
 -test xx        xx can be serialize, deserialize, combined or firstcall to test a scenario for many different serializers
 -reftracking    If set a list with many identical references is serialized.
 -serializer xxx Execute the test only for a specific serializer with the name xxx where multiple ones can be used with ,
 -list           List all registered serializers
 -notouch        Do not touch the deserialized objects to test lazy deserialization
                 To execute deserialize you must first have called the serialize to generate serialized test data on disk to be read during deserialize
Examples
Compare protobuf against MessagePackSharp for serialize and deserialize performance
 SerializerTests -Runs 1 -test combined -serializer protobuf,MessagePackSharp
Test how serializers perform when reference tracking is enabled. Currently that are BinaryFormatter,Protobuf_net and DataContract
 Although Json.NET claim to have but it is not completely working.
 SerializerTests -Runs 1 -test combined -reftracking
Test SimdJsonSharpSerializer serializer with 3 million objects for serialize and deserialize.
 SerializerTests -test combined -N 3000000 -serializer SimdJsonSharpSerializer

    I have added SimdJsonSharpSerializer with 3 million objects which creates a 100 MB Json file with Utf8JsonSerializer and deserialized it with your parsers.

    For code see https://github.com/Alois-xx/SerializerTests/blob/master/Serializers/SimdJsonSharpSerializer.cs
    and
    https://github.com/Alois-xx/SerializerTests/blob/master/Serializers/SimdJsonSharpSerializerN.cs

  5. Alois-xx commented on Sep 5, 2019

    @Alois-xx
    Alois-xx
    on Sep 5, 2019
    Author

    @EgorBo: Any update on this?

  6. Alois-xx commented on Oct 21, 2019

    @Alois-xx
    Alois-xx
    on Oct 21, 2019 via email
    Author

  7. Alois-xx commented on Oct 21, 2019

    @Alois-xx
    Alois-xx
    on Oct 21, 2019 via email
    Author
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions