Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replication namespace #464

Merged
merged 81 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
106fd00
replication added
Tokesh Aug 3, 2024
e912e13
fixing lint and trying test verbose
Tokesh Aug 3, 2024
bd877a8
changing docker compose
Tokesh Aug 3, 2024
6fd6292
Merge branch 'main' into replication-namespace
Tokesh Aug 3, 2024
addcd76
adding tests for replication namespace
Tokesh Aug 3, 2024
638d60e
fixing lint, validate specs ci
Tokesh Aug 3, 2024
a6e0a87
fixing docker-compose and validate specs ci
Tokesh Aug 3, 2024
b012e7b
fixing validate specs ci
Tokesh Aug 3, 2024
236fea7
trying to fix validate specs
Tokesh Aug 3, 2024
864ee68
trying to fix validate specs v2
Tokesh Aug 3, 2024
30fca51
Merge branch 'replication-namespace'
Tokesh Aug 14, 2024
1d051cd
Merge branch 'main' of https://github.com/Tokesh/opensearch-api-speci…
Tokesh Aug 14, 2024
0cdec4d
Merge pull request #1 from Tokesh/main
Tokesh Aug 14, 2024
cacd32e
fixing tests which using cluster of OS
Tokesh Aug 14, 2024
2548bbf
add changelog changes, fixing ci issues
Tokesh Aug 14, 2024
eaab260
fixing empty volumes in docker-compose
Tokesh Aug 14, 2024
1fbca08
trying to fix docker compose
Tokesh Aug 14, 2024
833d3fd
adding supporting cluster containers
Tokesh Aug 14, 2024
b861f51
configuring docker compose
Tokesh Aug 14, 2024
e83589c
trying to fix docker-compose problems
Tokesh Aug 14, 2024
bc2800b
adding verbose to check
Tokesh Aug 14, 2024
87bdbdb
changing follower cluster role
Tokesh Aug 14, 2024
f075bd7
adding default follower role
Tokesh Aug 14, 2024
5cd2b95
trying to catch all possible roles
Tokesh Aug 14, 2024
d114038
deleting response
Tokesh Aug 14, 2024
ac1dc59
adding all access
Tokesh Aug 14, 2024
8c6ec73
checking all_access
Tokesh Aug 14, 2024
334b09d
adding admin to rolemapping
Tokesh Aug 14, 2024
018be31
checking current user info
Tokesh Aug 14, 2024
b4f6a3c
testing new docker-compose
Tokesh Aug 15, 2024
f609a97
adding opensearch_url
Tokesh Aug 15, 2024
b2d92e8
trying to change url
Tokesh Aug 15, 2024
bbc3ae1
trying to handle in https
Tokesh Aug 15, 2024
a1c8cc0
trying to handle in https
Tokesh Aug 15, 2024
f2a898d
checking role mapping
Tokesh Aug 15, 2024
bd92a0c
adding round at the end of synopsis
Tokesh Aug 15, 2024
7803383
adding getting users and user detail
Tokesh Aug 15, 2024
e04fec9
trying to create
Tokesh Aug 15, 2024
90e72da
fixing uri
Tokesh Aug 15, 2024
e79cb97
deleting not neccessary synopsis
Tokesh Aug 15, 2024
fd4349c
deleting not neccessary synopsis
Tokesh Aug 15, 2024
2b6dbdb
waiting 201 not 200
Tokesh Aug 15, 2024
56e1c56
adding permissions
Tokesh Aug 15, 2024
ba4dcea
formatting yaml
Tokesh Aug 15, 2024
4e20ee9
testing concept from documentation
Tokesh Sep 5, 2024
a603d74
Merge branch 'main' into replication-namespace
Tokesh Sep 5, 2024
03c4bc2
fixing tabling
Tokesh Sep 5, 2024
345c054
fixing tabling v2
Tokesh Sep 5, 2024
3f98811
adding index pattern
Tokesh Sep 5, 2024
8798e82
getting all role mappings
Tokesh Sep 5, 2024
5ed10fb
checking roles
Tokesh Sep 5, 2024
50e336d
getting login roles
Tokesh Nov 9, 2024
0b3279b
trying roles
Tokesh Nov 9, 2024
a49a4e5
testing docker compose change
Tokesh Nov 9, 2024
2006adb
hotfix cluster alias in tests
Tokesh Nov 9, 2024
78f73ff
giving all access to users
Tokesh Nov 9, 2024
485e213
hotfix status code in response
Tokesh Nov 9, 2024
aaac7d7
status code fixing
Tokesh Nov 9, 2024
8ede2de
deleting response for tests
Tokesh Nov 9, 2024
aa70210
adding all_access
Tokesh Nov 9, 2024
7f1ea97
trying role mappings
Tokesh Nov 9, 2024
3b977fe
testing without security plugin
Tokesh Nov 10, 2024
f1c43e4
testing without https
Tokesh Nov 10, 2024
35ab7fe
fixing small tabular misstake in ci cd
Tokesh Nov 10, 2024
74900fb
fixing order of env parametrs and adding links to lycheeignore
Tokesh Nov 10, 2024
8ff49eb
Merge branch 'main' into replication-namespace
Tokesh Nov 10, 2024
8fc1987
ready for basic review
Tokesh Nov 10, 2024
a4f8f05
Merge remote-tracking branch 'refs/remotes/origin/replication-namespa…
Tokesh Nov 10, 2024
c1918b9
fixing path in ci/cd
Tokesh Nov 10, 2024
60a6ad3
deleting env file
Tokesh Nov 10, 2024
1c671fc
deleting extra spaces and moving one chapter to prologue
Tokesh Nov 17, 2024
a2bc5f7
Merge branch 'main' into replication-namespace
Tokesh Nov 17, 2024
9ced3ef
divide one file to few smaller
Tokesh Nov 17, 2024
03f959d
hotfix extra space
Tokesh Nov 17, 2024
08e4a2d
deleting not necessary response parts
Tokesh Nov 17, 2024
9e1e502
hotfix structure
Tokesh Nov 17, 2024
0ab0105
using common structures in replication schemes
Tokesh Nov 17, 2024
229fb19
adding verbose to see tests
Tokesh Nov 17, 2024
cca5c99
adding enum type
Tokesh Nov 17, 2024
91a61f8
fixing lint
Tokesh Nov 17, 2024
4629754
Replaced anyOf properties by allOf.
dblock Nov 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .cspell
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ authc
authinfo
authtoken
autocut
autofollow
backendroles
backpressure
beider
Expand Down Expand Up @@ -74,6 +75,7 @@ ignorecase
integ
internalusers
Intertransport
ipam
Jelinek
kibanainfo
kibanaserver
Expand All @@ -90,6 +92,7 @@ lucene
Lucene
lycheeverse
marvinpinto
memlock
metaphone
mget
millis
Expand Down Expand Up @@ -206,6 +209,7 @@ Translog
trbl
truststore
tubone
ulimits
Undeploys
unigrams
Unmanaged
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/test-spec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ jobs:
tests: routing
- version: 2.18.0
tests: snapshot
- version: 2.18.0
tests: plugins/replication
url: http://localhost:9200
- version: 2.18.0
tests: plugins/streaming
- version: 2.18.0
Expand Down Expand Up @@ -89,6 +92,7 @@ jobs:
--opensearch-insecure \
--opensearch-version=${{ matrix.entry.version }} \
--coverage coverage/test-spec-coverage-${{ steps.tests.outputs.hash }}.json \
--opensearch-url=${{ matrix.entry.url || 'https://localhost:9200'}} \
--tests=tests/${{ matrix.entry.tests || 'default' }}

- name: Get Container Logs
Expand Down
1 change: 1 addition & 0 deletions .lycheeignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
https://localhost:*
http://localhost:*
http://webhook:8080
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Added support for request headers in tests [#461](https://github.com/opensearch-project/opensearch-api-specification/pull/461)
- Added metadata additionalProperties to `ErrorCause` ([#462](https://github.com/opensearch-project/opensearch-api-specification/pull/462))
- Added `creation_date` field to `DanglingIndex` ([#462](https://github.com/opensearch-project/opensearch-api-specification/pull/462))
- Added `replication` namespace API specifications ([#464](https://github.com/opensearch-project/opensearch-api-specification/pull/464))
- Added doc on `cluster create-index blocked` workaround ([#465](https://github.com/opensearch-project/opensearch-api-specification/pull/465))
- Added `observability` namespace API specifications ([#474](https://github.com/opensearch-project/opensearch-api-specification/pull/474))
- Added support for reusing output variables as keys in payload expectations ([#471](https://github.com/opensearch-project/opensearch-api-specification/pull/471))
Expand Down
301 changes: 301 additions & 0 deletions spec/namespaces/replication.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,301 @@
openapi: 3.1.0
info:
title: OpenSearch Replication API
description: OpenSearch Replication API.
version: 1.0.0
paths:
/_plugins/_replication/{index}/_start:
put:
operationId: replication.start.0
x-operation-group: replication.start
x-version-added: '1.1'
description: Initiate replication of an index from the leader cluster to the follower cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#start-replication
parameters:
- $ref: '#/components/parameters/replication.start::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.start'
responses:
'200':
$ref: '#/components/responses/replication.start@200'
/_plugins/_replication/{index}/_stop:
post:
operationId: replication.stop.0
x-operation-group: replication.stop
x-version-added: '1.1'
description: Terminates replication and converts the follower index to a standard index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#stop-replication
parameters:
- $ref: '#/components/parameters/replication.stop::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.stop'
responses:
'200':
$ref: '#/components/responses/replication.stop@200'
/_plugins/_replication/{index}/_pause:
post:
operationId: replication.pause.0
x-operation-group: replication.pause
x-version-added: '1.1'
description: Pauses replication of the leader index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#pause-replication
parameters:
- $ref: '#/components/parameters/replication.pause::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.pause'
responses:
'200':
$ref: '#/components/responses/replication.pause@200'
/_plugins/_replication/{index}/_resume:
post:
operationId: replication.resume.0
x-operation-group: replication.resume
x-version-added: '1.1'
description: Resumes replication of the leader index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#resume-replication
parameters:
- $ref: '#/components/parameters/replication.resume::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.resume'
responses:
'200':
$ref: '#/components/responses/replication.resume@200'
/_plugins/_replication/{index}/_status:
get:
operationId: replication.status.0
x-operation-group: replication.status
x-version-added: '1.1'
description: Gets the status of index replication.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-replication-status
parameters:
- $ref: '#/components/parameters/replication.status::path.index'
responses:
'200':
$ref: '#/components/responses/replication.status@200'
/_plugins/_replication/leader_stats:
get:
operationId: replication.leader_stats.0
x-operation-group: replication.leader_stats
x-version-added: '1.1'
description: Gets information about replicated leader indexes on a specified cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-leader-cluster-stats
responses:
'200':
$ref: '#/components/responses/replication.leader_stats@200'
/_plugins/_replication/follower_stats:
get:
operationId: replication.follower_stats.0
x-operation-group: replication.follower_stats
x-version-added: '1.1'
description: Gets information about follower (syncing) indexes on a specified cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-follower-cluster-stats
responses:
'200':
$ref: '#/components/responses/replication.follower_stats@200'
/_plugins/_replication/autofollow_stats:
get:
operationId: replication.autofollow_stats.0
x-operation-group: replication.autofollow_stats
x-version-added: '1.1'
description: Gets information about auto-follow activity and any replication rules configured on the specified cluster.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#get-auto-follow-stats
responses:
'200':
$ref: '#/components/responses/replication.autofollow_stats@200'
/_plugins/_replication/{index}/_update:
put:
operationId: replication.update_settings.0
x-operation-group: replication.update_settings
x-version-added: '1.1'
description: Updates settings on the follower index.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#update-settings
parameters:
- $ref: '#/components/parameters/replication.update_settings::path.index'
requestBody:
$ref: '#/components/requestBodies/replication.update_settings'
responses:
'200':
$ref: '#/components/responses/replication.update_settings@200'
/_plugins/_replication/_autofollow:
post:
operationId: replication.create_replication_rule.0
x-operation-group: replication.create_replication_rule
x-version-added: '1.1'
description: Automatically starts replication on indexes matching a specified pattern.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#create-replication-rule
requestBody:
$ref: '#/components/requestBodies/replication.create_replication_rule'
responses:
'200':
$ref: '#/components/responses/replication.create_replication_rule@200'
delete:
operationId: replication.delete_replication_rule.0
x-operation-group: replication.delete_replication_rule
x-version-added: '1.1'
description: Deletes the specified replication rule.
externalDocs:
url: https://opensearch.org/docs/latest/tuning-your-cluster/replication-plugin/api/#delete-replication-rule
requestBody:
$ref: '#/components/requestBodies/replication.delete_replication_rule'
responses:
'200':
$ref: '#/components/responses/replication.delete_replication_rule@200'
components:
parameters:
replication.start::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.stop::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.pause::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.resume::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.status::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
replication.update_settings::path.index:
in: path
name: index
description: Name of the data stream, index, or index alias to perform bulk actions on.
required: true
schema:
$ref: '../schemas/_common.yaml#/components/schemas/IndexName'
style: simple
requestBodies:
replication.start:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/Replication'
required: true
replication.stop:
content:
application/json:
schema:
type: object
required: true
replication.pause:
content:
application/json:
schema:
type: object
required: true
replication.resume:
content:
application/json:
schema:
type: object
required: true
replication.update_settings:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/UpdateSettings'
required: true
replication.create_replication_rule:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/CreateReplicationRule'
required: true
replication.delete_replication_rule:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/DeleteReplicationRule'
required: true
responses:
replication.start@200:
content:
application/json:
schema:
$ref: '../schemas/_common.yaml#/components/schemas/AcknowledgedResponseBase'
replication.stop@200:
content:
application/json:
schema:
$ref: '../schemas/_common.yaml#/components/schemas/AcknowledgedResponseBase'
replication.pause@200:
content:
application/json:
schema:
$ref: '../schemas/_common.yaml#/components/schemas/AcknowledgedResponseBase'
replication.resume@200:
content:
application/json:
schema:
$ref: '../schemas/_common.yaml#/components/schemas/AcknowledgedResponseBase'
replication.status@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/Status'
replication.leader_stats@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/LeaderStatus'
replication.follower_stats@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/FollowerStatus'
replication.autofollow_stats@200:
content:
application/json:
schema:
$ref: '../schemas/replication._common.yaml#/components/schemas/AutoFollowStatus'
replication.update_settings@200:
content:
application/json:
schema:
$ref: '../schemas/_common.yaml#/components/schemas/AcknowledgedResponseBase'
replication.create_replication_rule@200:
content:
application/json:
schema:
$ref: '../schemas/_common.yaml#/components/schemas/AcknowledgedResponseBase'
replication.delete_replication_rule@200:
content:
application/json:
schema:
$ref: '../schemas/_common.yaml#/components/schemas/AcknowledgedResponseBase'
Loading
Loading