In order to install the plugin, simply run: bin/plugin install org.wikimedia.elasticsearch.swift/swift-repository-plugin/<version>.
For Elasticsearch versions prior to 5.x, please refer to https://github.com/wikimedia/search-repository-swift.
Starting with version 3.0.0 of this plugin the plugin is developed on master, and compiled against all Elasticsearch versions in branches.
The resulting versioning schema looks like this: 3.0.0-es5.2.0, where the plugin version is followed by the ES version it is compatible with. A single plugin version will have several ES compatible versions.
$ curl -XPUT 'http://localhost:9200/_snapshot/my_backup' -d '{
"type": "swift",
"settings": {
"swift_url": "http://localhost:8080/auth/v1.0/",
"swift_container": "my-container",
"swift_username": "myuser",
"swift_password": "mypass!"
}
}'
See Snapshot And Restore for more information
| Setting | Description |
|---|---|
| swift_container | Swift container name. Mandatory |
| swift_url | Swift auth url. Mandatory |
| swift_authmethod | Swift auth method, one of "KEYSTONE_V3", "KEYSTONE", "TEMPAUTH" or "BASIC"(default) |
| swift_domainname | Authenticate against domain scope with this domain (default is "Default") |
| swift_tenantname | Authenticate against project scope using this tenant name, only used with keystone auth |
| swift_password | Swift password |
| swift_username | Swift username |
| swift_preferred_region | Region to use. If you do not specify a region, Swift will pick the endpoint of the first region. If you have multiple regions, the order is not guarenteed. |
| chunk_size | Maximum size for individual objects in the snapshot. Defaults to 5gb as that's the Swift default |
| compress | Turns on compression of the snapshot files. Defaults to false as it tends to break with Swift |
| max_restore_bytes_per_sec | Throttles per node restore rate. Defaults to 20mb per second. |
| max_snapshot_bytes_per_sec | Throttles per node snapshot rate. Defaults to 20mb per second. |
Plugin settings to be placed in elasticsearch YAML configuration. Keep defaults, unless problems are detected.
| Setting | Description |
|---|---|
| repository_swift.minimize_blob_exists_checks | true (default) or false. Reduces volume of SWIFT requests to check a blob's existence. |
| repository_swift.allow_caching | true or false (default). Allow JOSS caching |
| repository_swift.allow_concurrent_io | true (default) or false. Allow concurrent writes and deletes. |
| repository_swift.delete_timeout_min | timeout of snapshot deletion in minutes (default 60 min) |
| repository_swift.retry_interval_s | interval in seconds for retry-until-success-or-timeout pattern in seconds (default 10 s) |
| repository_swift.retry_count | number of attempts for retries where timing is impractical (default 3 times) |
| repository_swift.short_operation_timeout_s | timeout for short operations (like writing a small blob, deleting, or listing) in seconds (default 30 s) |
| repository_swift.long_operation_timeout_s | timeout for long operations (like writing of multi-Gig data stream) in seconds (default 600 s) |
| repository_swift.snapshot_timeout_min | timeout of taking a snapshot in minutes (default 360 min) |
| repository_swift.stream_read | true (default) or false. Reduce memory footprint on restore, at the expense of not being able to retry reads on failure. |
| repository_swift.stream_write | true or false (default). Reduce memory footprint on snapshot, at the expense of not writing concurrently. |
Since Swift has logging dependencies you have to be careful about debugging in Eclipse.
- Import this project into Eclipse using the maven connector. Do no import the main Elasticsearch code.
- Create a new java application debug configuration and set it to run ElasticsearchF.
- Go to the Classpath tab
- Click on Maven Dependiences
- Click on Advanced
- Click Add Folder
- Click ok
- Expand the tree to find /src/test/resources
- Click ok
- Click debug