Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update indices with agents information #544

Merged
merged 19 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 34 additions & 6 deletions ecs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,24 +45,24 @@ files to generate the mappings. These are the inputs for the ECS generator.
* INDEXER_SRC: Path to the wazuh-indexer repository
* MODULE: Module to generate mappings for
* --upload <URL>: Upload generated index template to the OpenSearch cluster. Defaults to https://localhost:9200
Example: generate.sh v8.11.0 ~/wazuh-indexer vulnerability-detector --upload https://indexer:9200
Example: generate.sh v8.11.0 ~/wazuh-indexer states-vulnerabilities --upload https://indexer:9200
```

3. Use the `generate.sh` script to generate the mappings for a module. The script takes 3 arguments,
plus 2 optional arguments to upload the mappings to the `wazuh-indexer`. Both, composable and legacy mappings
are generated. For example, to generate the mappings for the `vulnerability-detector` module using the
are generated. For example, to generate the mappings for the `states-vulnerabilities` module using the
ECS version `v8.11.0` and assuming that path of this repository is `~/wazuh/wazuh-indexer`:

```bash
./generate.sh v8.11.0 ~/wazuh/wazuh-indexer vulnerability-detector
./generate.sh v8.11.0 ~/wazuh/wazuh-indexer states-vulnerabilities
```

The tool will output the folder where they have been generated.

```console
Loading schemas from git ref v8.11.0
Running generator. ECS version 8.11.0
Mappings saved to ~/wazuh/wazuh-indexer/ecs/vulnerability-detector/mappings/v8.11.0
Mappings saved to ~/wazuh/wazuh-indexer/ecs/states-vulnerabilities/mappings/v8.11.0
```

4. When you are done. Exit the virtual environment.
Expand Down Expand Up @@ -93,7 +93,7 @@ The script takes care of these changes automatically, generating the `opensearch
You can either upload the index template using cURL or the UI (dev tools).

```bash
curl -u admin:admin -k -X PUT "https://indexer:9200/_index_template/wazuh-vulnerability-detector" -H "Content-Type: application/json" -d @opensearch-template.json
curl -u admin:admin -k -X PUT "https://indexer:9200/_index_template/wazuh-states-vulnerabilities" -H "Content-Type: application/json" -d @opensearch-template.json
```

Notes:
Expand All @@ -117,7 +117,7 @@ are required.
### Event generator

For testing purposes, the script `generate_events.py` can be used to generate events for a given module.
Currently, it is only able to generate events for the `vulnerability-detector` module. To support other
Currently, it is only able to generate events for the `states-vulnerabilities` module. To support other
modules, please extend of refactor the script.

The script prompts for the required parameters, so it can be launched without arguments:
Expand All @@ -137,3 +137,31 @@ The script uses log file. Check it out for debugging or additional information.
- [ECS repository](https://github.com/elastic/ecs)
- [ECS usage](https://github.com/elastic/ecs/blob/main/USAGE.md)
- [ECS field reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html)

### All-in-one script

```bash
#!/bin/bash

indices=(
agent
alerts
command
states-fim
states-inventory-hardware
states-inventory-hotfixes
states-inventory-networks
states-inventory-packages
states-inventory-ports
states-inventory-processes
states-inventory-system
states-vulnerabilities
)

ECS="v8.11.0"
WI_REPO_PATH=~/wazuh/wazuh-indexer

for index in "${indices[@]}"; do
bash generate.sh $ECS $WI_REPO_PATH "$index"
done
```
9 changes: 7 additions & 2 deletions ecs/agent/fields/custom/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,13 @@
level: custom
description: >
The last time the agent logged in.
- name: is_connected
type: boolean
- name: status
type: keyword
level: custom
description: >
Agents' interpreted connection status depending on `agent.last_login`.
allowed_values:
- name: active
description: Active agent status
- name: disconnected
description: Disconnected agent status
6 changes: 6 additions & 0 deletions ecs/agent/fields/custom/host.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: host
reusable:
top_level: false
expected:
- agent
6 changes: 6 additions & 0 deletions ecs/agent/fields/custom/os.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: os
reusable:
top_level: false
expected:
- agent.host
6 changes: 6 additions & 0 deletions ecs/agent/fields/custom/risk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: risk
reusable:
top_level: false
expected:
- agent.host
10 changes: 3 additions & 7 deletions ecs/agent/fields/subset.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,6 @@ fields:
groups: {}
key: {}
last_login: {}
is_connected: {}
host:
fields:
ip: {}
os:
fields:
full: {}
status: {}
host:
fields: "*"
22 changes: 21 additions & 1 deletion ecs/alerts/fields/custom/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,24 @@
type: keyword
level: custom
description: >
The groups the agent belongs to.
List of groups the agent belong to.
- name: key
type: keyword
level: custom
description: >
The registration key of the agent.
- name: last_login
type: date
level: custom
description: >
The agent's last login.
- name: status
type: keyword
level: custom
description: >
Agents' interpreted connection status depending on `agent.last_login`.
allowed_values:
- name: active
description: Active agent status
- name: disconnected
description: Disconnected agent status
6 changes: 6 additions & 0 deletions ecs/alerts/fields/custom/host.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: host
reusable:
top_level: true
expected:
- { at: agent, as: host }
6 changes: 6 additions & 0 deletions ecs/alerts/fields/custom/os.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: os
reusable:
top_level: false
expected:
- agent.host
6 changes: 6 additions & 0 deletions ecs/alerts/fields/custom/risk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: risk
reusable:
top_level: false
expected:
- agent.host
9 changes: 8 additions & 1 deletion ecs/alerts/fields/subset.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,14 @@ fields:
base:
fields: "*"
agent:
fields: "*"
fields:
groups: {}
id: {}
name: {}
type: {}
version: {}
host:
fields: "*"
as:
fields: "*"
client:
Expand Down
6 changes: 6 additions & 0 deletions ecs/states-fim/fields/custom/host.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: host
reusable:
top_level: true
expected:
- { at: agent, as: host }
6 changes: 6 additions & 0 deletions ecs/states-fim/fields/custom/os.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: os
reusable:
top_level: false
expected:
- agent.host
6 changes: 6 additions & 0 deletions ecs/states-fim/fields/custom/risk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: risk
reusable:
top_level: false
expected:
- agent.host
9 changes: 8 additions & 1 deletion ecs/states-fim/fields/subset.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,13 @@ fields:
tags: []
agent:
fields:
id: {}
groups: {}
id: {}
name: {}
type: {}
version: {}
host:
fields: "*"
file:
fields:
attributes: {}
Expand All @@ -28,6 +33,8 @@ fields:
type: {}
uid: {}
owner: {}
host:
fields: "*"
registry:
fields:
key: {}
Expand Down
21 changes: 21 additions & 0 deletions ecs/states-inventory-hardware/fields/custom/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,30 @@
title: Wazuh Agents
short: Wazuh Inc. custom fields.
type: group
group: 2
fields:
- name: groups
type: keyword
level: custom
description: >
List of groups the agent belong to.
- name: key
type: keyword
level: custom
description: >
The registration key of the agent.
- name: last_login
type: date
level: custom
description: >
The agent's last login.
- name: status
type: keyword
level: custom
description: >
Agents' interpreted connection status depending on `agent.last_login`.
allowed_values:
- name: active
description: Active agent status
- name: disconnected
description: Disconnected agent status
8 changes: 4 additions & 4 deletions ecs/states-inventory-hardware/fields/custom/host.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
- name: host
title: host
type: group
description: >
Host related data.
reusable:
top_level: true
expected:
- { at: agent, as: host }
fields:
- name: memory
description: >
Expand Down
6 changes: 6 additions & 0 deletions ecs/states-inventory-hardware/fields/custom/os.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: os
reusable:
top_level: false
expected:
- agent.host
6 changes: 6 additions & 0 deletions ecs/states-inventory-hardware/fields/custom/risk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: risk
reusable:
top_level: false
expected:
- agent.host
21 changes: 7 additions & 14 deletions ecs/states-inventory-hardware/fields/subset.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,15 @@ fields:
"@timestamp": {}
agent:
fields:
id: {}
groups: {}
id: {}
name: {}
type: {}
version: {}
host:
fields: "*"
observer:
fields:
serial_number: {}
host:
fields:
memory:
fields:
total: {}
free: {}
used:
fields:
percentage: {}
cpu:
fields:
name: {}
cores: {}
speed: {}
fields: "*"
21 changes: 21 additions & 0 deletions ecs/states-inventory-hotfixes/fields/custom/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,30 @@
title: Wazuh Agents
short: Wazuh Inc. custom fields.
type: group
group: 2
fields:
- name: groups
type: keyword
level: custom
description: >
List of groups the agent belong to.
- name: key
type: keyword
level: custom
description: >
The registration key of the agent.
- name: last_login
type: date
level: custom
description: >
The agent's last login.
- name: status
type: keyword
level: custom
description: >
Agents' interpreted connection status depending on `agent.last_login`.
allowed_values:
- name: active
description: Active agent status
- name: disconnected
description: Disconnected agent status
6 changes: 6 additions & 0 deletions ecs/states-inventory-hotfixes/fields/custom/host.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: host
reusable:
top_level: true
expected:
- { at: agent, as: host }
6 changes: 6 additions & 0 deletions ecs/states-inventory-hotfixes/fields/custom/os.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: os
reusable:
top_level: false
expected:
- agent.host
6 changes: 6 additions & 0 deletions ecs/states-inventory-hotfixes/fields/custom/risk.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
- name: risk
reusable:
top_level: false
expected:
- agent.host
Loading