Proposal to update the Insomnia YAML format

[subnetmarco]

We are currently looking at ways to make Insomnia play nicer with Git and diff viewer tools, by simplifying its underlying YAML format. If there is any feedback or ideas, this would be the right time to start brainstorming on them.

Proposal

This is the new proposal, which is almost half the size of the current format:

type: collection.insomnia.rest/5.0
name: My Collection
description: |
  This is a long description
meta:
  id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
  created: 1724094380513
  modified: 1724420259205
environments:
  - Base Environment:
      private: false
      value:
        hello: world
  - Secret:
      private: true
      value:
        apikey: XYZ
requests:
  - New Request:
      meta:
        id: f7875eb2-2a0f-4b92-a27f-4f3baa924559
        created: 1724094380513
      method: POST
      url: 'httpbin.org/post/:pathparam'
      description: |
        This is the request description
      queryparams:
        - pathparam:
            value: ''
      params:
        - query1:
            value: val1
            description: |
              This is val1
        - query2:
            value: val2
            description: |
              This is val2
      body:
        mimeType: multipart/form-data
        params:
          - hello:
              value: world
      scripts:
        pre-request:
          value: |
            console.log("pre")
        after-response:
          value: |
            console.log("after")
      headers:
        - Content-Type:
            value: multipart/form-data
      authentication:
        basic:
          username: user
          password: pass

Current legacy

And this is the current equivalent:

_type: export
__export_format: 4
__export_date: 2024-08-23T13:58:16.643Z
__export_source: insomnia.desktop.app:v9.3.4-f9a5627.pr-7839.0
resources:
  - _id: req_37c22efa528b4a709e93ed070492ef21
    parentId: wrk_590d7bebf4434e95bdc601ba85e99edb
    modified: 1724420259205
    created: 1724094380513
    url: httpbin.org/post/:pathparam
    name: New Request
    description: This is the request description
    method: POST
    body:
      mimeType: multipart/form-data
      params:
        - id: pair_299be9e291e2453c88764aed0cbca335
          name: hello
          value: world
          description: ""
          disabled: false
    parameters:
      - id: pair_565b5471a6304f57b47ecdc3543e426d
        name: query1
        value: val1
        description: This is val1
        disabled: false
      - id: pair_a9767f0a3bc14296aae3a40a1ed801ee
        name: query2
        value: val2
        description: this is val2
        disabled: false
    headers:
      - name: Content-Type
        value: multipart/form-data
      - name: User-Agent
        value: insomnia/9.3.4-f913c8d.pr-7814.0
        id: pair_549ec5ed719044d8ac2e6d50a5aa34e9
        disabled: false
    authentication:
      type: basic
      useISO88591: false
      disabled: false
      username: user
      password: pass
    preRequestScript: console.log("pre")
    metaSortKey: -1724094380513
    isPrivate: false
    pathParameters:
      - name: pathparam
        value: ""
    afterResponseScript: console.log("after")
    settingStoreCookies: true
    settingSendCookies: true
    settingDisableRenderRequestBody: false
    settingEncodeUrl: true
    settingRebuildPath: true
    settingFollowRedirects: global
    _type: request
  - _id: wrk_590d7bebf4434e95bdc601ba85e99edb
    parentId: null
    modified: 1724094372268
    created: 1724094372268
    name: My Collection
    description: "This is a long description"
    scope: collection
    _type: workspace
  - _id: env_9eabe7c44704289ebe25a84c0366d626a7748106
    parentId: wrk_590d7bebf4434e95bdc601ba85e99edb
    modified: 1724421469673
    created: 1724094372271
    name: Base Environment
    data:
      hello: world
    dataPropertyOrder:
      "&":
        - hello
    color: null
    isPrivate: false
    metaSortKey: 1724094372271
    _type: environment
  - _id: jar_9eabe7c44704289ebe25a84c0366d626a7748106
    parentId: wrk_590d7bebf4434e95bdc601ba85e99edb
    modified: 1724420307481
    created: 1724094372273
    name: Default Jar
    cookies: []
    _type: cookie_jar
  - _id: env_7b6e7af9c74b4b4ea2e987189e4826ca
    parentId: env_9eabe7c44704289ebe25a84c0366d626a7748106
    modified: 1724421484937
    created: 1724421458910
    name: Secret
    data:
      apikey: XYZ
    dataPropertyOrder:
      "&":
        - apikey
    color: null
    isPrivate: true
    metaSortKey: 1724421458908
    _type: environment

Principles

To improve readability:

• Don't show empty or null values. Assume them to be empty.
• Don't show false boolean properties. Assume them to be false.
• The order of the objects is implied by how they are being added to the file, instead of having sort keys.
• Properly nest properties instead of the flat indentation we use now.
• The parent project relationship is not defined in the file themselves anymore.
• The meta section is optional, if missing we will generate a new id and new created fields.

Local, Git and Cloud

The new format can be used for all storage types.

Feedback?

This is a simple example of a collection format, the equivalent will also be created for design specs, mocks, and other file types.

Do you have any feedback?

[marckong]

@subnetmarco When I was touching this repo actively, what I had in mind in regards to "script" was to actually point to files instead of having the code hard coded in the yaml file.

## ...
      scripts:
        pre-request:
          path:  ${file(./prescript.ts):PATH}
          lang: ts
          version: ^5.4.4
        after-response:
          path: ${file(./afterscript.ts):PATH}
          lang: ts
          version: ^5.4.4
## ...

This way we can achieve several benefits;

• git diff viewer could actually point to a versioned file as these "scripts" are really better to be in separate files
• we can actually connect to the language server run off the machine for linting
• this will enable us a path to not ship all kinds of packages used in the "js" script files (let's say you want to import npm packages. Currently we have to ship them in the executable). Instead we point to the packages installed in the user's machine or wherever this runs. This will also allow us to provide users to write tests in the user's chosen language.

I am only bringing this up because this can benefit our Testing features as well far ahead of our competitor products. Currently the testing feature is limited and locks you in with javascript with mocha and chai.

[marckong]

100%!

[subnetmarco]

If we update the schema, it will be collection.insomnia.rest/5.1 ?

Yes. We can also support automatic discovery in a Git repository by then executing the following command in a repository (or folder):

$ find . -type f \( -name "*.yaml" -o -name "*.yml" \) -exec grep -l "type: .*insomnia\.rest" {} +

And it will recursively list all Insomnia files that are collections, design specs, mocks and so on.

[marckong]

That's great!!

[gatzjames]

Suggestion: Maybe to keep a consistent type we could have a type like inline/file:

scripts:
  pre-request:
    type: inline
    value: |
      console.log("pre")
  after-response:
    type: file
    path: scripts/after-response.js
```​

[subnetmarco]

@gatzjames the type can be inferred by value or path. I would like to remove verbosity when possible.

[cwangsmv]

Suggestion: For requests under folder, should we add one more hierarchy like:

folders:
  - My Folder:
      scripts:
        pre-request:
          value: |
            console.log("pre")
        after-response:
          value: |
            console.log("after")
      environment:
        hello: test
        foo: bar
      requests:
# ...

Besides, since we support global environment now, If the environment using in requests are from global environment rather than collection level/folder level environment, shall we also export the accordingly global environment.

[subnetmarco]

Yes, agree.

[ihexxa]

It might be great if it could also resolve some frequently asked questions such as:

• Keep reference/id in exporting and importing.

[subnetmarco]

Update to support folders:

type: collection.insomnia.rest/5.0
name: My Collection
description: |
  This is a long description
meta:
  id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
  created: 1724094380513
  modified: 1724420259205
environments:
  - Base Environment:
      private: false
      value:
        hello: world
  - Secret:
      private: true
      value:
        apikey: XYZ
data:
  - My Folder 1:
      meta:
        id: f7875eb2-2a0f-4b92-a27f-4f3baa924559
        created: 1724094380513
      description: |
        This is a folder example
      queryparams:
        - pathparam:
            value: ''
      params:
        - query1:
            value: val1
            description: |
              This is val1
        - query2:
            value: val2
            description: |
              This is val2
      body:
        mimeType: multipart/form-data
        params:
          - hello:
              value: world
      scripts:
        pre-request:
          value: |
            console.log("pre")
        after-response:
          value: |
            console.log("after")
      headers:
        - Content-Type:
            value: multipart/form-data
      authentication:
        basic:
          username: user
          password: pass
      requests:
        - Request 1:
           ..
        - Request 2:
          ..

  - New Request:
      meta:
        id: f7875eb2-2a0f-4b92-a27f-4f3baa924559
        created: 1724094380513
      method: POST
      url: 'httpbin.org/post/:pathparam'
      description: |
        This is a parent description
      queryparams:
        - pathparam:
            value: ''
      params:
        - query1:
            value: val1
            description: |
              This is val1
        - query2:
            value: val2
            description: |
              This is val2
      body:
        mimeType: multipart/form-data
        params:
          - hello:
              value: world
      scripts:
        pre-request:
          value: |
            console.log("pre")
        after-response:
          value: |
            console.log("after")
      headers:
        - Content-Type:
            value: multipart/form-data
      authentication:
        basic:
          username: user
          password: pass

[subnetmarco]

More examples for other resources.

Collections

type: collection.insomnia.rest/5.0
name: My Collection
description: |
  This is a long description
meta:
  id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
  created: 1724094380513
  modified: 1724420259205
environments:
  - Base Environment:
      private: false
      value:
        hello: world
  - Secret:
      private: true
      value:
        apikey: XYZ
data:
  - My Folder 1:
      meta:
        id: f7875eb2-2a0f-4b92-a27f-4f3baa924559
        created: 1724094380513
      description: |
        This is a folder example
      queryparams:
        - pathparam:
            value: ''
      params:
        - query1:
            value: val1
            description: |
              This is val1
        - query2:
            value: val2
            description: |
              This is val2
      body:
        mimeType: multipart/form-data
        params:
          - hello:
              value: world
      scripts:
        pre-request:
          value: |
            console.log("pre")
        after-response:
          value: |
            console.log("after")
      headers:
        - Content-Type:
            value: multipart/form-data
      authentication:
        basic:
          username: user
          password: pass
      requests:
        - Request 1:
           ..
        - Request 2:
          ..

  - New Request:
      meta:
        id: f7875eb2-2a0f-4b92-a27f-4f3baa924559
        created: 1724094380513
      method: POST
      url: 'httpbin.org/post/:pathparam'
      description: |
        This is a parent description
      queryparams:
        - pathparam:
            value: ''
      params:
        - query1:
            value: val1
            description: |
              This is val1
        - query2:
            value: val2
            description: |
              This is val2
      body:
        mimeType: multipart/form-data
        params:
          - hello:
              value: world
      scripts:
        pre-request:
          value: |
            console.log("pre")
        after-response:
          value: |
            console.log("after")
      headers:
        - Content-Type:
            value: multipart/form-data
      authentication:
        basic:
          username: user
          password: pass

Mocks:

type: mock.insomnia.rest/5.0
name: My Mock
description: |
  This is a long description
meta:
  id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
  created: 1724094380513
  modified: 1724420259205
data:
  - Mock Route:
    meta:
      id: f7875eb2-2a0f-4b92-a27f-4f3baa924559
      created: 1724094380513
    method: POST
    status: 200
    body:
        mimeType: application/json
        value: |
        {
          "args": {}, 
          "headers": {
            "Accept": "*/*", 
            "Content-Length": "25", 
            "Content-Type": "multipart/form-data; boundary=X-INSOMNIA-BOUNDARY", 
            "Host": "httpbin.org", 
            "User-Agent": "insomnia/9.1.1", 
            "X-Amzn-Trace-Id": "Root=1-667fcd36-06fe9e7e521007577f6b2687"
          }, 
          "origin": "87.13.164.94", 
          "url": "http://httpbin.org/get"
        }
        headers:
          - Content-Type:
              value: multipart/form-data

Environments:

type: environment.insomnia.rest/5.0
name: My Global Environment
description: |
  This is a long description
meta:
  id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
  created: 1724094380513
  modified: 1724420259205
data:
  - Staging:
    meta:
      id: f7875eb2-2a0f-4b92-a27f-4f3baa924559
      created: 1724094380513
    color: #cccccc
    private: true
    value:
      hello: world
      this:
        is:
          a:
            complex: "value"

Design Documents:

type: spec.insomnia.rest/5.0
name: My Design Spec
description: |
  This is a long description
meta:
  id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
  created: 1724094380513
  modified: 1724420259205
data:
  meta:
    id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
    created: 1724094380513
    modified: 1724420259205
  name: my-spec.yaml
  file: /to/path.yaml # If we have file, we cannot have spec and viceversa
  spec:
    openapi: 3.0.0
      info:
      description: "This is a sample server Petstore server.  You can find out more about
        Swagger at [http://swagger.io](http://swagger.io) or on [irc.freenode.net,
        #swagger](http://swagger.io/irc/).  For this sample, you can use the api key
        'special-key' to test the authorization filters."
      version: 1.0.2
      title: Swagger Petstore
      termsOfService: http://swagger.io/terms/
      contact:
        email: apiteam@swagger.io
      license:
        name: Apache 2.0
        url: http://www.apache.org/licenses/LICENSE-2.0.html

     ...

  collection: # Similar to standalone collection
    meta:
      id: 3fed4ff3-d737-49e3-a5f2-f17b0f3a01d2
      created: 1724094380513
      modified: 1724420259205
    environments:
      - Base Environment:
          private: false
          value:
            hello: world
      - Secret:
          private: true
          value:
            apikey: XYZ
    data:
      ..

[subnetmarco]

We should use YAML schema to re-uses common components across different files.