-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Plugins PoC - Java #234
Comments
I was able to index data to an index using a custom REST endpoint by extending the public class RestHelloWorldAction extends RestIndexAction {
public static final String HELLO_WORLD_INDEX_NAME = "helloworld";
@Override
public String getName() {
return "rest_handler_hello_world";
}
@Override
public List<Route> routes() {
return unmodifiableList(asList(
new Route(POST, "/_plugins/hello_world")));
}
@Override
public RestChannelConsumer prepareRequest(RestRequest request, final NodeClient client) throws IOException {
assert request.params().get("id") == null : "non-null id: " + request.params().get("id");
// default to op_type create
request.params().put("index", HELLO_WORLD_INDEX_NAME );
request.params().putIfAbsent("op_type", "create");
return super.prepareRequest(request, client);
}
} #### Index a simple json using _plugins/hello_world endpoint
$ curl -XPOST http://localhost:9200/_plugins/hello_world -H "Content-Type: application/json" -d '{"field":"value"}'
{"_index":"helloworld","_id":"Wer2yY8BwC6DtDSTphCq","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}
#### Use the Id to retrieve it
curl -XGET "http://localhost:9200/helloworld/_doc/Wer2yY8BwC6DtDSTphCq"
{"_index":"helloworld","_id":"Wer2yY8BwC6DtDSTphCq","_version":1,"_seq_no":0,"_primary_term":1,"found":true,"_source":{"field":"value"}} |
I'm working on this PoC in my own repo: |
I was able to implement a very rough CRUD: $ curl -XPOST http://localhost:9200/_plugins/todo/create -H "Content-Type: application/json" -d '{"field":"value"}'
{"_index":"todo","_id":"9teryo8B79B5eeR91Cfu","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1}fede@tyner:~/IdeaProjects/opensearch-plugin-crud-sample (add-read-update-delete *+)
$ curl -XGET "http://localhost:9200/_plugins/todo/read/9teryo8B79B5eeR91Cfu"
{"_index":"todo","_id":"9teryo8B79B5eeR91Cfu","_version":1,"_seq_no":0,"_primary_term":1,"found":true,"_source":{"field":"value"}}fede@tyner:~/IdeaProjects/opensearch-plugin-crud-sample (add-read-update-delete *+)
$ curl -XPOST http://localhost:9200/_plugins/todo/update/9teryo8B79B5eeR91Cfu -H "Content-Type: application/json" -d '{"doc":{"field":"value2"}}'
{"_index":"todo","_id":"9teryo8B79B5eeR91Cfu","_version":2,"result":"updated","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1}fede@tyner:~/IdeaProjects/opensearch-plugin-crud-sample (add-read-update-delete *+)
$ curl -XGET "http://localhost:9200/_plugins/todo/read/9teryo8B79B5eeR91Cfu"
{"_index":"todo","_id":"9teryo8B79B5eeR91Cfu","_version":2,"_seq_no":1,"_primary_term":1,"found":true,"_source":{"field":"value2"}}fede@tyner:~/IdeaProjects/opensearch-plugin-crud-sample (add-read-update-delete *+)
$ curl -XDELETE "http://localhost:9200/_plugins/todo/delete/9teryo8B79B5eeR91Cfu"
{"_index":"todo","_id":"9teryo8B79B5eeR91Cfu","_version":3,"result":"deleted","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":2,"_primary_term":1}fede@tyner:~/IdeaProjects/opensearch-plugin-crud-sample (add-read-update-delete *+) For the time being I used different paths for each operation as I registered the routes through individual classes. |
I got tests somewhat working by basing my own classes on: Various warnings pop up, and this is still only testing writing to indices, so I need to dig deeper into the reference code above. |
It seems that we should use yaml based tests for the rest layer: |
I managed to get a simple REST API yaml-based test by following: Various examples are present under that folder as well: The following test definition indexes data by hitting the ---
"CRUD test":
- do:
_plugins.todo:
body: { foo: bar }
- set: { _id: document_id }
- match: { _index: todo }
- do:
get:
index: todo
id: $document_id
- match: { _source: { foo: bar } } However I'm currently stuck at creating a test for a {
"_plugins.todo": {
"stability" : "stable",
"url": {
"paths": [
{
"path": "/_plugins/todo",
"methods": [
"POST",
"PUT"
]
}
]
},
"body": {
"description": "The ToDo plugin's document manipulation endpoint"
}
}
} |
I got the In order to do so, the API needs to be properly defined in There needs to be a _plugins.todo_create.json{
"_plugins.todo_create": {
"stability" : "stable",
"url": {
"paths": [
{
"path": "/_plugins/todo",
"methods": [
"POST"
]
}
]
},
"body": {
"description": "The document"
}
}
} _plugins.todo_read.json{
"_plugins.todo_read": {
"stability" : "stable",
"url": {
"paths": [
{
"path": "/_plugins/todo/{id}",
"methods": [
"GET"
],
"parts": {
"id":{
"type":"string",
"description":"The id of the document to update"
}
}
}
]
}
}
} _plugins.todo_update.json{
"_plugins.todo_update": {
"stability" : "stable",
"url": {
"paths": [
{
"path": "/_plugins/todo/{id}",
"methods": [
"PUT"
],
"parts": {
"id":{
"type":"string",
"description":"The id of the document to update"
}
}
}
]
},
"body": {
"description": "The document"
}
}
} _plugins.todo_delete.json{
"_plugins.todo_delete": {
"stability" : "stable",
"url": {
"paths": [
{
"path": "/_plugins/todo/{id}",
"methods": [
"DELETE"
],
"parts": {
"id":{
"type":"string",
"description":"The id of the document to update"
}
}
}
]
}
}
} Each of these need to:
It's important to detail the usafe of each of these endpoints correctly, since the yaml tests are based on these definitions. The yaml tests have their own syntax and need to be placed under I've created the following test files (one for accepted API method): 20_create.yml---
"Create test":
- do:
_plugins.todo_create:
body: { foo: bar }
- set: { _id: document_id }
- match: { _index: todo }
- do:
get:
index: todo
id: $document_id
- match: { _source: { foo: bar } }
30_read.yml---
"Read test":
- do:
index:
index: todo
id: 1000
body:
foo: bar
- do:
_plugins.todo_read:
id: 1000
- match: { _index: todo }
- match: { _source: { foo: bar } } 40_update.yml---
"Update test":
- do:
index:
index: todo
id: 1000
body:
foo: bar
- do:
_plugins.todo_update:
id: 1000
body: { doc: { foo: baz } }
- do:
get:
index: todo
id: 1000
- match: { _source: { foo: baz } } 50_delete.yml---
"Delete test":
- do:
index:
index: todo
id: 1000
body:
foo: bar
- do:
_plugins.todo_delete:
id: 1000
- match: { _index: todo }
- match: { result: deleted }
The The second The rest of the tests start with the indexing of a document by means of the built-in |
It was helpful to browse through the built in yaml tests and the rest api spec files under: |
All of the above code can be checked out in the repo I've set up for these tests: I've now moved on to building unit tests for our custom classes. |
Unit tests are going through. /*
* SPDX-License-Identifier: Apache-2.0
*
* The OpenSearch Contributors require contributions made to
* this file be licensed under the Apache-2.0 license or a
* compatible open source license.
*/
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
/*
* Modifications Copyright OpenSearch Contributors. See
* GitHub history for details.
*/
package org.opensearch.rest.action;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import org.junit.Before;
import org.mockito.Mockito;
import org.opensearch.client.node.NodeClient;
import org.opensearch.common.xcontent.XContentType;
import org.opensearch.core.common.bytes.BytesArray;
import org.opensearch.rest.RestHandler;
import org.opensearch.rest.RestRequest;
import org.opensearch.test.OpenSearchTestCase;
import org.opensearch.test.rest.FakeRestChannel;
import org.opensearch.test.rest.FakeRestRequest;
public class ToDoRestIndexActionTests extends OpenSearchTestCase {
private String path;
private String requestBody;
private ToDoRestIndexAction toDoRestIndexAction;
@Before
public void setUp() throws Exception {
super.setUp();
this.path = String.format(Locale.ROOT, "%s", ToDoPlugin.TODO_BASE_URI);
this.requestBody = "{\"foo\":\"bar\"}";
this.toDoRestIndexAction = new ToDoRestIndexAction();
}
public void testGetNames() {
String name = toDoRestIndexAction.getName();
assertEquals("todo_plugin_document_index", name);
}
public void testGetRoutes() {
List<RestHandler.Route> routes = toDoRestIndexAction.routes();
assertEquals(this.path, routes.get(0).getPath());
}
public void testPrepareRequest() throws IOException {
Map<String, String> params = new HashMap<>();
FakeRestRequest request = new FakeRestRequest.Builder(xContentRegistry()).withMethod(RestRequest.Method.POST)
.withPath(this.path)
.withParams(params)
.withContent(new BytesArray(this.requestBody), XContentType.JSON)
.build();
final FakeRestChannel channel = new FakeRestChannel(request, true, 0);
this.toDoRestIndexAction.prepareRequest(request, Mockito.mock(NodeClient.class));
assertEquals(channel.responses().get(), 0);
assertEquals(channel.errors().get(), 0);
}
} These tests run automatically as part of the Below is a screenshot of the tests' results: |
ConclusionsBefore closing this issue, we'll be reviewing each of the tasks.
However, we did code how to create indices on start, but we have examples about how to do that. |
Description
To better understand how plugins work, their lifecylce, how they interact with core functionalities and how to develop them in general, we are going to develop a simple plugin for
wazuh-indexer
that exposes an API to manage ToDos, as a proof of concept. We'll use Java as the programming language.As a definition of done, we expect to have an API from which it's possible to:
ToDo model
The ToDos will be stored using an index, for example:
wazuh-todos
. Each of the documents in the index could have these properties:id
: uuidtitle
: stringdescription
: textdue_date
: dateThe index name and its fields must follow the naming convention.
Tasks
The text was updated successfully, but these errors were encountered: