Day 23 - json-next Gem - Read Next Generation JSON Versions (HanSON, SON, JSONX/JSON11, Etc) with Comments, Unquoted Keys, Multi-Line Strings, Trailing Commas, Optional Commas, and More

Written by {% avatar geraldb %} Gerald Bauer

A code monkey and enthusiastic collector (and publisher) of open football and beer data. Skills include Ruby, SQLite and CSV. Spec lead of CSV <3 JSON.

Let's start with a quiz:

Q: What's your favorite text ("human"/non-binary) data exchange format?

(A) JSON
(B) XML
(C) HTML
(D) YAML
(E) TOML/INI
(F) SQL
(G) CSV
(H) Other, please tell!

If you picked (A) JSON - let's continue with a question:

What's missing in JSON?

Comments, Comments, Comments
Unquoted Keys
Multi-Line Strings
- a) Folded -- Folds Newlines
- b) Unfolded
Trailing Commas in Arrays and Objects

More:

Date/DateTime/Timestamp Type
Optional Commas
Optional Unquoted String Values
"Raw" String (e.g. '' instead of "")
- No need to escape \ or " etc. To escape ' use ''' e.g. ''''Henry's Themes'''
More Data Types (set, map, symbol, etc.)
And much more

Discuss - Fixing JSON - Is it too late?

Fixing JSON. Request for comments, please!

We can easily agree on what’s wrong with JSON, and I can't help wondering if it'd be worth fixing it.

-- Tim Bray (Fixing JSON)

XML already does everything JSON does! And there's no way to differentiate between nodes and attributes! And there are no namespaces! And no schemas! What's the point of JSON?

-- Anonymous

We need to fix engineers that try to 'fix JSON', absolutely nothing is broken with JSON.

-- Anonymous

And what's your take? Enter the json-next gem.

What's the json-next gem?

The json-next gem lets you convert and read (parse) next generation json versions including: HanSON e.g. HANSON.parse, SON e.g. SON.parse, JSONX e.g. JSONX.parse.

HanSON

HanSON - JSON for Humans by Tim Jansen et al

HanSON is an extension of JSON with a few simple additions to the spec:

quotes for strings are optional if they follow JavaScript identifier rules.
you can alternatively use backticks, as in ES6's template string literal, as quotes for strings. A backtick-quoted string may span several lines and you are not required to escape regular quote characters, only backticks. Backslashes still need to be escaped, and all other backslash-escape sequences work like in regular JSON.
for single-line strings, single quotes ('') are supported in addition to double quotes ("")
you can use JavaScript comments, both single line (//) and multi-line comments (/* */), in all places where JSON allows whitespace.
Commas after the last list element or object property will be ignored.

Example:

{
  listName: "Sesame Street Monsters", // note that listName needs no quotes
  content: [
    {
      name: "Cookie Monster",
      /* Note the template quotes and unescaped regular quotes in the next string */
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    // don't worry, the trailing comma will be ignored
   ]
}

Use HANSON.convert to convert HanSON text to ye old' JSON text:

{
  "listName": "Sesame Street Monsters",
  "content": [
    { "name": "Cookie Monster",
       "background": "Cookie Monster used to be a\n ... to eat \"healthy\" food."
    },
    { "name": "Herry Monster",
      "background": "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Use HANSON.parse instead of JSON.parse to parse text to ruby hash / array / etc.:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

SON

SON - Simple Object Notation by Aleksander Gurin et al

Simple data format similar to JSON, but with some minor changes:

comments starts with # sign and ends with newline (\n)
comma after an object key-value pair is optional
comma after an array item is optional

JSON is compatible with SON in a sense that JSON data is also SON data, but not vise versa.

Example:

{
  # Personal information

  "name": "Alexander Grothendieck"
  "fields": "mathematics"
  "main_topics": [
    "Etale cohomology"
    "Motives"
    "Topos theory"
    "Schemes"
  ]
  "numbers": [1 2 3 4]
  "mixed": [1.1 -2 true false null]
}

Use SON.convert to convert SON text to ye old' JSON text:

{
  "name": "Alexander Grothendieck",
  "fields": "mathematics",
  "main_topics": [
    "Etale cohomology",
    "Motives",
    "Topos theory",
    "Schemes"
  ],
  "numbers": [1, 2, 3, 4],
  "mixed": [1.1, -2, true, false, null]
}

Use SON.parse instead of JSON.parse to parse text to ruby hash / array / etc.:

{
  "name" => "Alexander Grothendieck",
  "fields" => "mathematics",
  "main_topics" =>
    ["Etale cohomology", "Motives", "Topos theory", "Schemes"],
  "numbers" => [1, 2, 3, 4],
  "mixed" => [1.1, -2, true, false, nil]
}

JSONX

JSON with Extensions or JSON v1.1 (a.k.a. JSON11 or JSON XI or JSON II)

Includes all JSON extensions from HanSON:

quotes for strings are optional if they follow JavaScript identifier rules.
you can alternatively use backticks, as in ES6's template string literal, as quotes for strings. A backtick-quoted string may span several lines and you are not required to escape regular quote characters, only backticks. Backslashes still need to be escaped, and all other backslash-escape sequences work like in regular JSON.
for single-line strings, single quotes ('') are supported in addition to double quotes ("")
you can use JavaScript comments, both single line (//) and multi-line comments (/* */), in all places where JSON allows whitespace.
Commas after the last list element or object property will be ignored.

Plus all JSON extensions from SON:

comments starts with # sign and ends with newline (\n)
comma after an object key-value pair is optional
comma after an array item is optional

Plus some more extra JSON extensions:

unquoted strings following the JavaScript identifier rules can use the dash (-) too e.g. allows common keys such as core-js, babel-preset-es2015, eslint-config-jquery and others

Example:

{
  #  use shell-like (or ruby-like) comments

  listName: "Sesame Street Monsters"   # note: comments after key-value pairs are optional
  content: [
    {
      name: "Cookie Monster"
      // note: the template quotes and unescaped regular quotes in the next string
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    /* don't worry, the trailing comma will be ignored  */
   ]
}

Use JSONX.convert (or JSONXI.convert or JSON11.convert or JSONII.convert) to convert JSONX text to ye old' JSON text:

{
  "listName": "Sesame Street Monsters",
  "content": [
    { "name": "Cookie Monster",
       "background": "Cookie Monster used to be a\n ... to eat \"healthy\" food."
    },
    { "name": "Herry Monster",
      "background": "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Use JSONX.parse (or JSONXI.parse or JSON11.parse or JSONII.parse) instead of JSON.parse to parse text to ruby hash / array / etc.:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Live Examples

require 'json/next'

text1 =<<TXT
{
  listName: "Sesame Street Monsters", // note that listName needs no quotes
  content: [
    {
      name: "Cookie Monster",
      /* Note the template quotes and unescaped regular quotes in the next string */
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    // don't worry, the trailing comma will be ignored
   ]
}
TXT

pp HANSON.parse( text1 )  # note: is the same as JSON.parse( HANSON.convert( text ))

resulting in:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

and

text2 =<<TXT
{
  # Personal information

  "name": "Alexander Grothendieck"
  "fields": "mathematics"
  "main_topics": [
    "Etale cohomology"
    "Motives"
    "Topos theory"
    "Schemes"
  ]
  "numbers": [1 2 3 4]
  "mixed": [1.1 -2 true false null]
}
TXT

pp SON.parse( text2 )  # note: is the same as JSON.parse( SON.convert( text ))

resulting in:

{
  "name" => "Alexander Grothendieck",
  "fields" => "mathematics",
  "main_topics" =>
    ["Etale cohomology", "Motives", "Topos theory", "Schemes"],
  "numbers" => [1, 2, 3, 4],
  "mixed" => [1.1, -2, true, false, nil]
}

and

text3 =<<TXT
{
  #  use shell-like (or ruby-like) comments

  listName: "Sesame Street Monsters"   # note: comments after key-value pairs are optional
  content: [
    {
      name: "Cookie Monster"
      // note: the template quotes and unescaped regular quotes in the next string
      background: `Cookie Monster used to be a
monster that ate everything, especially cookies.
These days he is forced to eat "healthy" food.`
    }, {
      // You can single-quote strings too:
      name: 'Herry Monster',
      background: `Herry Monster is a furry blue monster with a purple nose.
He's mostly retired today.`
    },    /* don't worry, the trailing comma will be ignored  */
   ]
}
TXT

pp JSONX.parse( text3 )   # note: is the same as JSON.parse( JSONX.convert( text ))
pp JSONXI.parse( text3 )  # note: is the same as JSON.parse( JSONXI.convert( text ))
pp JSON11.parse( text3 )  # note: is the same as JSON.parse( JSON11.convert( text ))
pp JSONII.parse( text3 )  # note: is the same as JSON.parse( JSONII.convert( text ))

resulting in:

{
  "listName" => "Sesame Street Monsters",
  "content" => [
     { "name" => "Cookie Monster",
       "background" => "Cookie Monster used to be a\n ... to eat \"healthy\" food."
     },
     { "name" => "Herry Monster",
       "background" => "Herry Monster is a furry blue monster with a purple nose.\n ... today."
    }
  ]
}

Bonus: More JSON Formats

See the Awesome JSON - What's Next? collection / page.

Find out more

References

Home :: github.com/json-next/json-next
Gem :: json-next
Docs :: json-next
Talk Notes :: JSON - What's Next for the No 2. Universal (Non-Binary/Human) Data Exchange Format

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

23-json-next.md

23-json-next.md

Day 23 - json-next Gem - Read Next Generation JSON Versions (HanSON, SON, JSONX/JSON11, Etc) with Comments, Unquoted Keys, Multi-Line Strings, Trailing Commas, Optional Commas, and More

What's missing in JSON?

Discuss - Fixing JSON - Is it too late?

What's the json-next gem?

HanSON

SON

JSONX

Live Examples

Bonus: More JSON Formats

Find out more

References

Files

23-json-next.md

Latest commit

History

23-json-next.md

File metadata and controls

Day 23 - json-next Gem - Read Next Generation JSON Versions (HanSON, SON, JSONX/JSON11, Etc) with Comments, Unquoted Keys, Multi-Line Strings, Trailing Commas, Optional Commas, and More

What's missing in JSON?

Discuss - Fixing JSON - Is it too late?

What's the json-next gem?

HanSON

SON

JSONX

Live Examples

Bonus: More JSON Formats

Find out more

References