File Handling updates, Raw HTTP Request updates, and DDL visibility

File Handling updates

• Expiry time in signed URL is now 6 hours, as per PreSignedURL
• Added CONNECTOR_FILE_REGION and CONNECTOR_FILE_BUCKET environment variables support for defining target region and bucket (and also CONNECTOR_FILE_DEV_BUCKET for development)
• Added check for expired file/URL for download methods based on AWS signing, changing error message if detected
• Disabled decoding response in needle by default, but also allowed passing in needle options

Raw HTTP Request updates

• Operation title is now sentence cased
• Schema has been updated where some properties should have been required, such as body
• Configuration option has been added for restricting URL oneOf options
• The response format has been fixed to match the format defined by the schema (breaking change)
  • status_code is now also returned in the output (bug fix)

schema.js support and DDL visibility

• A type property is now included in the connectorsJson for every operation, with values either being public or ddil
• All DDL operations are now detected if their (programmatic) name ends with _ddl.
  • If a schema is not specified, and generic schema is set (using global schema).

The rules for operation types and being added to the connectors.json op list is as follows:

• If no schema is present
  • If operation's name ends with DDL, then it is added to connectors.json as a DDL type operation
  • Else operation is NOT added to connectors.json
• If a schema is present
  • If type is NOT provided, then assume public by default (unless the operation name ends with _ddl)
  • If type is provided
    • If public, add to connectors.json
    • If ddl, add to connectors.json
    • If private, NOT added to connectors.json

Other

• Threadneedle version updated (v1.12.0)
• The dev server can now be disabled via FALAFEL_DISABLE_DEV_SERVER environment variable when in dev mode
• The no trigger code for request.js has been updated to #no_trigger for BE
• Artisan error handling has been updated to improve connector debugging