Skip to content

Latest commit

 

History

History
306 lines (228 loc) · 18.2 KB

UPGRADING.rst

File metadata and controls

306 lines (228 loc) · 18.2 KB
Raw

Upgrading to Graylog 3.0.x

This file only contains the upgrade note for the upcoming release. Please see our documentation for the complete upgrade notes.

Elasticsearch Version Requirements

Graylog 3.0 drops support for Elasticsearch versions before 5.6.x. That means you have to upgrade Elasticsearch to at least version 5.6.5 before upgrading Graylog to version 3.0. Make sure to read the Elasticsearch upgrade guides before doing that.

Simplified HTTP interface configuration

Graylog used to have a lot of different settings regarding the various HTTP interfaces it provides, namely the Graylog REST API and the Graylog web interface.

This mostly originates from the fact that Graylog used to consist of two components before Graylog 2.0.0, a server component and a separate web interface.

The changes in this release finally merge the HTTP listeners for the Graylog REST API and web interface into a single HTTP listener, which should make the initial configuration of Graylog simpler and reduce errors caused by conflicting settings.

The path of the Graylog REST API is now hard-coded to /api, so if you're still using the legacy URI on port 12900/tcp or have been using a custom path (via the rest_listen_uri or rest_transport_uri settings), you'll have to update the URI used to access the Graylog REST API.

If you are using a reverse proxy in front of Graylog (like nginx) and configured it to set the X-Graylog-Server-URL HTTP header, you have to remove the api/ suffix because that is now the default. (as mentioned above)

Example:

# This nginx setting in Graylog <3.0 ...
header_upstream X-Graylog-Server-URL http://{host}/api

# ... needs to be changed to the following with Graylog 3.0
header_upstream X-Graylog-Server-URL http://{host}/

For a more detailed description of the new HTTP settings, please consult the annotated Graylog configuration file.

Overview of removed Graylog REST API settings:

Removed Setting New Setting Default
rest_listen_uri http_bind_address 127.0.0.1:9000
rest_transport_uri http_publish_uri http://$http_bind_address/
web_endpoint_uri http_external_uri $http_publish_uri
rest_enable_cors http_enable_cors true
rest_enable_gzip http_enable_gzip true
rest_max_header_size http_max_header_size 8192
rest_max_initial_line_length http_max_initial_line_length 4096
rest_thread_pool_size http_thread_pool_size 16
rest_enable_tls http_enable_tls false
rest_tls_cert_file http_tls_cert_file Empty
rest_tls_key_file http_tls_key_file Empty
rest_tls_key_password http_tls_key_password Empty

Overview of removed Graylog web interface settings:

Removed Setting New Setting Default
web_enable None  
web_listen_uri http_bind_address 127.0.0.1:9000
web_enable_cors http_enable_cors true
web_enable_gzip http_enable_gzip true
web_max_header_size http_max_header_size 8192
web_max_initial_line_length http_max_initial_line_length 4096
web_thread_pool_size http_thread_pool_size 16
web_enable_tls http_enable_tls false
web_tls_cert_file http_tls_cert_file Empty
web_tls_key_file http_tls_key_file Empty
web_tls_key_password http_tls_key_password Empty

Plugins merged into the Graylog server

Starting with Graylog 3.0.0, the following official plugins were merged into the Graylog server:

That means these plugins are not available as separate plugins anymore. If you manually update your Graylog installation (without using operating system packages), make sure to remove all old plugin files from the plugin_dir folder.

The old issues in these repositories are still available for reference but new issues should only be created in the Graylog server issue tracker.

The following HTTP API paths changed due to the plugin merge:

Old Path New Path
/plugins/org.graylog.plugins.map/mapdata /search/mapdata
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/pipeline /system/pipelines/pipeline
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/pipeline/parse /system/pipelines/pipeline/parse
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule /system/pipelines/rule
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule/functions /system/pipelines/rule/functions
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule/multiple /system/pipelines/rule/multiple
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/rule/parse /system/pipelines/rule/parse
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/connections /system/pipelines/connections
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/connections/to_stream /system/pipelines/connections/to_stream
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/connections/to_pipeline /system/pipelines/connections/to_pipeline
/plugins/org.graylog.plugins.pipelineprocessor/system/pipelines/simulate /system/pipelines/simulate

New "bin_dir" and "data_dir" configuration parameters

We introduced two new configuration parameters related to file system paths.

  • bin_dir config option points to the directory that contains scripts like graylogctl.
  • data_dir option configures the base directory for Graylog server state.

Please check the updated default graylog.conf configuration file for required changes to your existing file.

Removed support for Drools-based filters

For a long time, Graylog allowed to use Drools to filter messages. Unfortunately, using Drools to perform complex filter logic came with a performance penalty and wasn't as flexible as we would have liked it to be.

Starting with Graylog 3.0.0, the support for Drools-based message filters has been removed from Graylog. The rules_file configuration setting has been removed accordingly.

If you still want to be able to use Drools for filtering messages in your Graylog cluster, you can install the [Drools plugin](https://github.com/fbalicchia/graylog-plugin-drools) and keep using your rules.drl file.

We recommend migrating the Drools-based logic to Processing Pipelines.

Drools-based blacklist

Graylog provided undocumented blacklist-functionality based on Drools. This blacklist could only be modified via the Graylog REST API on the /filters/blacklist resource.

If you've been using this functionality, you'll have to migrate these blacklist rules to the Processing Pipelines.

To check if you're using the Drools-based blacklist in Graylog prior to version 3.0.0, you can run the following command:

# curl -u admin:password -H 'Accept: application/json' 'http://graylog.example.com/api/filters/blacklist?pretty=true'

String-based blacklist rule

Old blacklist rule:

{
   "id" : "54e300001234123412340001",
   "type" : "string",
   "name" : "String Blacklist",
   "description" : "Drop messages based on case-insensitive string comparison",
   "fieldName" : "custom_field",
   "pattern" : "EXAMPLE pattern",
   "creator_user_id" : "admin",
   "created_at" : "2018-04-04T12:00:00.000Z"
}

New pipeline rule:

rule "string-blacklist"
when
  has_field("custom_field") &&
  lowercase(to_string($message.custom_field)) == "example pattern"
then
  drop_message();
end

See also:

Regex-based blacklist rule

Old blacklist rule:

{
   "id" : "54e300001234123412340002",
   "type" : "regex",
   "name" : "Regex Blacklist",
   "description" : "Drop messages based on regular expression",
   "fieldName" : "custom_field",
   "pattern" : "^EXAMPLE.*",
   "creator_user_id" : "admin",
   "created_at" : "2018-04-04T12:00:00.000Z"
}

New pipeline rule:

rule "regex-blacklist"
when
  has_field("custom_field") &&
  regex("^EXAMPLE.*", to_string($message.custom_field)).matches == true
then
  drop_message();
end

See also:

IP Range-based blacklist rule

Old blacklist rule:

{
   "id" : "54e300001234123412340003",
   "type" : "iprange",
   "name" : "IP Blacklist",
   "description" : "Drop messages based on IP address",
   "fieldName" : "custom_field",
   "pattern" : "192.168.0.0/16",
   "creator_user_id" : "admin",
   "created_at" : "2018-04-04T12:00:00.000Z"
}

New pipeline rule:

rule "ip-blacklist"
when
  has_field("custom_field") &&
  cidr_match("192.168.0.0/16", to_ip($message.custom_field))
then
  drop_message();
end

See also:

Changed metrics name for stream rules

The name of the metrics for stream rules have been changed to include the stream ID which helps identifying the actual stream they are related to.

Old metric name:

org.graylog2.plugin.streams.StreamRule.${stream-rule-id}.executionTime

New metric name:

org.graylog2.plugin.streams.Stream.${stream-id}.StreamRule.${stream-rule-id}.executionTime

Email alarm callback default settings

The defaults of the configuration settings for the email alarm callback with regard to encrypted connections have been changed.

Setting Old default New default
transport_email_use_tls false true
transport_email_use_ssl true false

Furthermore, it's not possible anymore to enable both settings (SMTP with STARTTLS and SMTP over SSL) at the same time because this led to errors at runtime when Graylog tried to upgrade the connection to TLS with STARTTLS in an already existing SMTPS connection.

Most SMTP services prefer SMTP with STARTTLS to provide an encrypted connection.