Replication namespace #464

Tokesh · 2024-08-03T19:18:42Z

Description

adding replication namespace specs

Issues Resolved

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tokesh <tokesh789@gmail.com>

Signed-off-by: Niyazbek Torekeldi <78027392+Tokesh@users.noreply.github.com>

github-actions · 2024-08-03T19:21:08Z

Changes Analysis

Commit SHA: 4629754
Comparing To SHA: bf35601

API Changes

Summary

├─┬Paths
│ ├──[➕] path (5480:3)
│ ├──[➕] path (5498:3)
│ ├──[➕] path (5460:3)
│ ├──[➕] path (5440:3)
│ ├──[➕] path (5518:3)
│ ├──[➕] path (5555:3)
│ ├──[➕] path (5383:3)
│ ├──[➕] path (5538:3)
│ ├──[➕] path (5420:3)
│ └──[➕] path (5572:3)
└─┬Components
  ├──[➕] requestBodies (27120:7)
  ├──[➕] requestBodies (27114:7)
  ├──[➕] requestBodies (27108:7)
  ├──[➕] requestBodies (27102:7)
  ├──[➕] requestBodies (27090:7)
  ├──[➕] requestBodies (27096:7)
  ├──[➕] requestBodies (27126:7)
  ├──[➕] responses (30234:7)
  ├──[➕] responses (30228:7)
  ├──[➕] responses (30246:7)
  ├──[➕] responses (30216:7)
  ├──[➕] responses (30210:7)
  ├──[➕] responses (30252:7)
  ├──[➕] responses (30204:7)
  ├──[➕] responses (30240:7)
  ├──[➕] responses (30264:7)
  ├──[➕] responses (30222:7)
  ├──[➕] responses (30258:7)
  ├──[➕] parameters (23710:7)
  ├──[➕] parameters (23750:7)
  ├──[➕] parameters (23718:7)
  ├──[➕] parameters (23726:7)
  ├──[➕] parameters (23734:7)
  ├──[➕] parameters (23742:7)
  ├──[➕] schemas (55615:7)
  ├──[➕] schemas (55483:7)
  ├──[➕] schemas (55447:7)
  ├──[➕] schemas (55466:7)
  ├──[➕] schemas (55494:7)
  ├──[➕] schemas (55606:7)
  ├──[➕] schemas (55650:7)
  ├──[➕] schemas (55620:7)
  ├──[➕] schemas (55501:7)
  ├──[➕] schemas (55538:7)
  ├──[➕] schemas (55566:7)
  ├──[➕] schemas (55583:7)
  ├──[➕] schemas (55662:7)
  ├──[➕] schemas (55641:7)
  └──[➕] schemas (55559:7)

Document Element	Total Changes	Breaking Changes
paths	10	0
components	39	0

Total Changes: 49
Additions: 49

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/11893879787/artifacts/2201699624

API Coverage

	Before	After	Δ
Covered (%)	588 (57.59 %)	599 (58.67 %)	11 (1.08 %)
Uncovered (%)	433 (42.41 %)	422 (41.33 %)	-11 (-1.08 %)
Unknown	42	42	0

Signed-off-by: Tokesh <tokesh789@gmail.com>

dblock · 2024-08-03T21:09:56Z

Should we split the docker-compose.yml into docker-compose-default, docker-compose-multinode.yml or something like that?

Tokesh · 2024-08-03T21:34:34Z

Should we split the docker-compose.yml into docker-compose-default, docker-compose-multinode.yml or something like that?

wow, do you work even on weekends?
Yes, I encountered this today, tested it on a local machine. Now I was thinking of asking whether it is worth separating or inventing some flags.

dblock · 2024-08-07T12:12:05Z

Take a look at #472 that allows custom docker compose and tests.

Tokesh · 2024-08-07T14:22:05Z

Take a look at #472 that allows custom docker compose and tests.

Thank you! It looks really good
I'll wait for your PR to merge so I can continue working on the replication namespace.

dblock · 2024-08-13T17:09:27Z

Take a look at #472 that allows custom docker compose and tests.

Thank you! It looks really good I'll wait for your PR to merge so I can continue working on the replication namespace.

@Tokesh that got merged FYI

…fication

update current version

Signed-off-by: Tokesh <tokesh789@gmail.com>

github-actions · 2024-08-14T18:38:31Z

Spec Test Coverage Analysis

Total	Tested
528	359 (67.99 %)

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh · 2024-11-10T18:32:06Z

@Tokesh want to finish this?

I passed the biggest part of CI/CD.
Any suggestions to refactor this code?

Signed-off-by: Tokesh <tokesh789@gmail.com>

dblock

This looks great!

I added .yml files to Vale preprocessing in #660, let's rebase this once it's merged to make sure it worked.

.github/workflows/test-spec.yml

tests/plugins/replication/replica.yaml

dblock · 2024-11-11T13:10:29Z

tests/plugins/replication/replica.yaml

+          index.number_of_replicas: 2
+    response:
+      status: 200
+  - synopsis: Create replication rule.


Extract autofollow into a separate file called autofollow.yaml.

dblock · 2024-11-11T13:12:35Z

tests/plugins/replication/replica.yaml

@@ -0,0 +1,127 @@
+$schema: ../../../json_schemas/test_story.schema.yaml


There are multiple examples here surrounding replication, one that starts replication, so see if that can be extracted in a start.yaml, and another that gets replication stats, belongs in stats.yaml.

Basically think of how these examples could be used in various parts of the documentation on their own, even if it means duplicating some API calls in prologues and epilogues.

I think i can divide to:
replica.yaml
stats.yaml
autofollow.yaml
replica-rules.yaml
And the code will be duplicated everywhere where we create an index and start replicating it, if it works, then I can divide it like this and do it in a few days.

That works. Maybe rename replica.yaml to index.yaml, that one will have the replication as the main act.

Generally these files match the namespace and/or the API path, not the "theme", so replica-rules is not a thing, it will need another name.

Signed-off-by: Tokesh <tokesh789@gmail.com>

Signed-off-by: Niyazbek Torekeldi <78027392+Tokesh@users.noreply.github.com>

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh · 2024-11-17T13:23:19Z

Ready for your review! @dblock @nhtruong

dblock

Glad we have a passing build!

Go through the fields and check whether they are semantically meaningful and need to be a $ref to something that exists (time values, index names, etc.).

.github/workflows/test-spec.yml

spec/namespaces/replication.yaml

spec/schemas/replication._common.yaml

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh · 2024-11-17T20:03:32Z

Thank you for a explained review. I read _common schemes and tried to fix all the places, where i can implement them.

dblock

Small one left, the rest LGTM!

dblock · 2024-11-17T23:26:46Z

spec/schemas/replication._common.yaml

+      type: object
+      properties:
+        settings:
+          anyOf:


Is this a combination of SettingsBody and these? So should be allOf.

Signed-off-by: dblock <dblock@amazon.com>

dblock · 2024-11-18T13:55:40Z

I made the change I suggested above and merged, thank you, this is great work @Tokesh!

Tokesh · 2024-11-19T16:53:52Z

Oh, i just checked. Thank you so much for detailed reviews and help! It's a pleasure working with you!

Tokesh and others added 4 commits August 3, 2024 22:24

replication added

106fd00

Signed-off-by: Tokesh <tokesh789@gmail.com>

fixing lint and trying test verbose

e912e13

Signed-off-by: Tokesh <tokesh789@gmail.com>

changing docker compose

bd877a8

Signed-off-by: Tokesh <tokesh789@gmail.com>

Merge branch 'main' into replication-namespace

6fd6292

Signed-off-by: Niyazbek Torekeldi <78027392+Tokesh@users.noreply.github.com>

Tokesh added 6 commits August 4, 2024 01:43

adding tests for replication namespace

addcd76

Signed-off-by: Tokesh <tokesh789@gmail.com>

fixing lint, validate specs ci

638d60e

Signed-off-by: Tokesh <tokesh789@gmail.com>

fixing docker-compose and validate specs ci

a6e0a87

Signed-off-by: Tokesh <tokesh789@gmail.com>

fixing validate specs ci

b012e7b

Signed-off-by: Tokesh <tokesh789@gmail.com>

trying to fix validate specs

236fea7

Signed-off-by: Tokesh <tokesh789@gmail.com>

trying to fix validate specs v2

864ee68

Signed-off-by: Tokesh <tokesh789@gmail.com>

dblock mentioned this pull request Aug 6, 2024

Add api spec for tiering of indices from hot to warm #368

Open

dblock mentioned this pull request Aug 7, 2024

Split test suite with separate docker-compose. #472

Merged

Tokesh and others added 4 commits August 14, 2024 23:05

Merge branch 'replication-namespace'

30fca51

Merge branch 'main' of https://github.com/Tokesh/opensearch-api-speci…

1d051cd

…fication

Merge pull request #1 from Tokesh/main

0cdec4d

update current version

fixing tests which using cluster of OS

cacd32e

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh added 7 commits August 14, 2024 23:42

add changelog changes, fixing ci issues

2548bbf

Signed-off-by: Tokesh <tokesh789@gmail.com>

fixing empty volumes in docker-compose

eaab260

Signed-off-by: Tokesh <tokesh789@gmail.com>

trying to fix docker compose

1fbca08

Signed-off-by: Tokesh <tokesh789@gmail.com>

adding supporting cluster containers

833d3fd

Signed-off-by: Tokesh <tokesh789@gmail.com>

configuring docker compose

b861f51

Signed-off-by: Tokesh <tokesh789@gmail.com>

trying to fix docker-compose problems

e83589c

Signed-off-by: Tokesh <tokesh789@gmail.com>

adding verbose to check

bc2800b

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh requested review from Xtansia, VachaShah and aabeshov as code owners November 10, 2024 18:28

deleting env file

60a6ad3

Signed-off-by: Tokesh <tokesh789@gmail.com>

dblock mentioned this pull request Nov 11, 2024

Fix: include .yml files when preparing for Vale. #660

Merged

dblock previously approved these changes Nov 11, 2024

View reviewed changes

dblock reviewed Nov 11, 2024

View reviewed changes

deleting extra spaces and moving one chapter to prologue

1c671fc

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh dismissed dblock’s stale review via 1c671fc November 17, 2024 12:32

Tokesh and others added 5 commits November 17, 2024 17:36

Merge branch 'main' into replication-namespace

a2bc5f7

Signed-off-by: Niyazbek Torekeldi <78027392+Tokesh@users.noreply.github.com>

divide one file to few smaller

9ced3ef

Signed-off-by: Tokesh <tokesh789@gmail.com>

hotfix extra space

03f959d

Signed-off-by: Tokesh <tokesh789@gmail.com>

deleting not necessary response parts

08e4a2d

Signed-off-by: Tokesh <tokesh789@gmail.com>

hotfix structure

9e1e502

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh requested a review from dblock November 17, 2024 13:23

dblock requested changes Nov 17, 2024

View reviewed changes

Tokesh added 4 commits November 18, 2024 00:45

using common structures in replication schemes

0ab0105

Signed-off-by: Tokesh <tokesh789@gmail.com>

adding verbose to see tests

229fb19

Signed-off-by: Tokesh <tokesh789@gmail.com>

adding enum type

cca5c99

Signed-off-by: Tokesh <tokesh789@gmail.com>

fixing lint

91a61f8

Signed-off-by: Tokesh <tokesh789@gmail.com>

Tokesh requested a review from dblock November 17, 2024 20:00

dblock requested changes Nov 17, 2024

View reviewed changes

Replaced anyOf properties by allOf.

4629754

Signed-off-by: dblock <dblock@amazon.com>

dblock approved these changes Nov 18, 2024

View reviewed changes

dblock merged commit 4771bfd into opensearch-project:main Nov 18, 2024
26 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replication namespace #464

Replication namespace #464

Tokesh commented Aug 3, 2024

github-actions bot commented Aug 3, 2024 •

edited

Loading

dblock commented Aug 3, 2024

Tokesh commented Aug 3, 2024 •

edited

Loading

dblock commented Aug 7, 2024

Tokesh commented Aug 7, 2024

dblock commented Aug 13, 2024

github-actions bot commented Aug 14, 2024 •

edited

Loading

Tokesh commented Nov 10, 2024

dblock left a comment

dblock Nov 11, 2024

dblock Nov 11, 2024

Tokesh Nov 11, 2024

dblock Nov 11, 2024 •

edited

Loading

Tokesh commented Nov 17, 2024

dblock left a comment

Tokesh commented Nov 17, 2024

dblock left a comment

dblock Nov 17, 2024

dblock commented Nov 18, 2024

Tokesh commented Nov 19, 2024

		@@ -0,0 +1,127 @@
		$schema: ../../../json_schemas/test_story.schema.yaml

Replication namespace #464

Replication namespace #464

Conversation

Tokesh commented Aug 3, 2024

Description

Issues Resolved

github-actions bot commented Aug 3, 2024 • edited Loading

Changes Analysis

API Changes

Summary

Report

API Coverage

dblock commented Aug 3, 2024

Tokesh commented Aug 3, 2024 • edited Loading

dblock commented Aug 7, 2024

Tokesh commented Aug 7, 2024

dblock commented Aug 13, 2024

github-actions bot commented Aug 14, 2024 • edited Loading

Spec Test Coverage Analysis

Tokesh commented Nov 10, 2024

dblock left a comment

Choose a reason for hiding this comment

dblock Nov 11, 2024

Choose a reason for hiding this comment

dblock Nov 11, 2024

Choose a reason for hiding this comment

Tokesh Nov 11, 2024

Choose a reason for hiding this comment

dblock Nov 11, 2024 • edited Loading

Choose a reason for hiding this comment

Tokesh commented Nov 17, 2024

dblock left a comment

Choose a reason for hiding this comment

Tokesh commented Nov 17, 2024

dblock left a comment

Choose a reason for hiding this comment

dblock Nov 17, 2024

Choose a reason for hiding this comment

dblock commented Nov 18, 2024

Tokesh commented Nov 19, 2024

github-actions bot commented Aug 3, 2024 •

edited

Loading

Tokesh commented Aug 3, 2024 •

edited

Loading

github-actions bot commented Aug 14, 2024 •

edited

Loading

dblock Nov 11, 2024 •

edited

Loading