Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON configuration and Jackson Streaming Object Processor #5225

Merged
merged 56 commits into from
Jun 18, 2024

Commits on Apr 11, 2024

  1. JSON configuration and Processor

    This PR adds a declarative JSON configuration object that allows users to specify the schema of a JSON message. It is meant to have good out-of-the-box defaults, while still allowing power users to modify some of the finer parsing details (should this int field be parseable from a string? should null values be allowed? what if a field is missing? etc). The JSON configuration layer is not tied to any specific implementation; it is introspectible, and could have alternative implementations with other parsing backends. (I could imagine a DHE use-case where they do code-generation based on the JSON configuration, somewhat like the DHE avro ObjectProcessor code generator.)
    
    Out of the box, there's an ObjectProcessor implementation based on the Jackson streaming APIs; that is, the data flows from byte[]s (or InputStream, relevant for very-large-files) to the output WritableChunks without the need for the intermediating Jackson databind layer (TreeNode). This saves a large layer of allocation that our current kafka json_spec layer relies upon. The ObjectProcessor layer means that this can be used in other places that expose ObjectProcessor layers and want 1-to-1 record-to-row (currently, Kafka).
    
    Part of deephaven#5222
    devinrsmith committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    1320749 View commit details
    Browse the repository at this point in the history
  2. spotless

    devinrsmith committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    bb1d02e View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. review response

    devinrsmith committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    5c7dde9 View commit details
    Browse the repository at this point in the history
  2. spotless

    devinrsmith committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    44f4bc9 View commit details
    Browse the repository at this point in the history
  3. Boxed builders

    devinrsmith committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    c567c21 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    60fad28 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. f

    devinrsmith committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    2358a24 View commit details
    Browse the repository at this point in the history
  2. Rename to Value

    devinrsmith committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    50ec4cb View commit details
    Browse the repository at this point in the history
  3. unmodifiable set

    devinrsmith committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    5dc5aa8 View commit details
    Browse the repository at this point in the history
  4. f

    devinrsmith committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    26ff817 View commit details
    Browse the repository at this point in the history
  5. f

    devinrsmith committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    14eadba View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. f

    devinrsmith committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    5c65852 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    979bdfa View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    fd3f6ed View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. f

    devinrsmith committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    a16b30f View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. f

    devinrsmith committed May 13, 2024
    Configuration menu
    Copy the full SHA
    f564bc4 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. f

    devinrsmith committed May 21, 2024
    Configuration menu
    Copy the full SHA
    f4c674e View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. some review responses

    devinrsmith committed May 22, 2024
    Configuration menu
    Copy the full SHA
    6f45aa7 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. f

    devinrsmith committed May 23, 2024
    Configuration menu
    Copy the full SHA
    6d4ebef View commit details
    Browse the repository at this point in the history

Commits on May 24, 2024

  1. f

    devinrsmith committed May 24, 2024
    Configuration menu
    Copy the full SHA
    32841a5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7a1a2d3 View commit details
    Browse the repository at this point in the history
  3. exceptions

    devinrsmith committed May 24, 2024
    Configuration menu
    Copy the full SHA
    2d73895 View commit details
    Browse the repository at this point in the history
  4. tests

    devinrsmith committed May 24, 2024
    Configuration menu
    Copy the full SHA
    c6f098b View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Rename tests

    devinrsmith committed May 28, 2024
    Configuration menu
    Copy the full SHA
    cf81769 View commit details
    Browse the repository at this point in the history
  2. f

    devinrsmith committed May 28, 2024
    Configuration menu
    Copy the full SHA
    8d1e377 View commit details
    Browse the repository at this point in the history
  3. stuff

    devinrsmith committed May 28, 2024
    Configuration menu
    Copy the full SHA
    5828f1a View commit details
    Browse the repository at this point in the history
  4. Tests and stuff

    devinrsmith committed May 28, 2024
    Configuration menu
    Copy the full SHA
    5022c2d View commit details
    Browse the repository at this point in the history
  5. Typed object improvements

    devinrsmith committed May 28, 2024
    Configuration menu
    Copy the full SHA
    88db86d View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. Array maths

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    f4b60c2 View commit details
    Browse the repository at this point in the history
  2. deeply nested array test

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    52e5932 View commit details
    Browse the repository at this point in the history
  3. array stuff

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    dbb66de View commit details
    Browse the repository at this point in the history
  4. kv mixin

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    0c0760e View commit details
    Browse the repository at this point in the history
  5. mixin consolidation

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    c9c20e0 View commit details
    Browse the repository at this point in the history
  6. discriminated object

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    f979b56 View commit details
    Browse the repository at this point in the history
  7. outputSize

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    2338fa0 View commit details
    Browse the repository at this point in the history
  8. cleanup

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    34dda0c View commit details
    Browse the repository at this point in the history
  9. Instant array types

    devinrsmith committed May 29, 2024
    Configuration menu
    Copy the full SHA
    9fccd9d View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    c4cae88 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Configuration menu
    Copy the full SHA
    a389e88 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    91c2990 View commit details
    Browse the repository at this point in the history
  3. Remove some exceptions

    devinrsmith committed May 30, 2024
    Configuration menu
    Copy the full SHA
    1526eb4 View commit details
    Browse the repository at this point in the history
  4. math util

    devinrsmith committed May 30, 2024
    Configuration menu
    Copy the full SHA
    30ef952 View commit details
    Browse the repository at this point in the history
  5. Update python

    devinrsmith committed May 30, 2024
    Configuration menu
    Copy the full SHA
    c0ab02a View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Review responses

    devinrsmith committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    08e44de View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    aeb4628 View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Add python json tests

    devinrsmith committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    6a6f383 View commit details
    Browse the repository at this point in the history
  2. Fix test

    devinrsmith committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    c7fc14b View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    10b7794 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. Configuration menu
    Copy the full SHA
    c26304e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fb9d759 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    258a8b8 View commit details
    Browse the repository at this point in the history
  4. Review points

    devinrsmith committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    1683d9e View commit details
    Browse the repository at this point in the history
  5. Python _val

    devinrsmith committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    c53262c View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    3cf7e9f View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Review responses

    devinrsmith committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    7e81e41 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Configuration menu
    Copy the full SHA
    b319512 View commit details
    Browse the repository at this point in the history