Skip to content

Commit

Permalink
Merge pull request #1451 from Andy17MB/Snorty3_Multithreading_AC_Edits
Browse files Browse the repository at this point in the history
Copy edited. Note:
  • Loading branch information
jasonrandrews authored Dec 18, 2024
2 parents d1812e7 + 8594101 commit 5987e00
Show file tree
Hide file tree
Showing 4 changed files with 36 additions and 35 deletions.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Scaling Snort3 - use multithreading for improved performance
title: Scaling Snort 3 - use multithreading for improved performance

draft: true
cascade:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@
review:
- questions:
question: >
Which of the following is a key benefit of Snort3's multithreading support?
Which of the following is a key benefit of Snort 3's multithreading support?
answers:
- It allows Snort to detect encrypted traffic.
- It improves packet processing performance
- It enables Snort to be run on legacy hardware
- It improves packet processing performance.
- It enables Snort to be run on legacy hardware.
- It support multiple rule sets at the same time.
correct_answer: 2
explanation: >
It improves packet processing performance by parallelizing tasks.
- questions:
question: >
Which parameter is used to enable multithreading in Snort3?
Which parameter is used to enable multithreading in Snort 3?
answers:
- --max-packet-threads
- --enable-threads
Expand All @@ -34,7 +34,7 @@ review:
- pcap
correct_answer: 3
explanation: >
The dump module in Snort3 is used to read capture files (such as .pcap or .pcapng files) for offline packet analysis.
The dump module in Snort 3 is used to read capture files (such as .pcap or .pcapng files) for offline packet analysis.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Install Snort3 and the required dependencies
title: Installing Snort 3 and the required dependencies
weight: 2

### FIXED, DO NOT MODIFY
Expand All @@ -10,15 +10,15 @@ Snort is an Open Source Intrusion Prevention System (IPS). Snort uses a series o

Multithreading in Snort 3 refers to the ability to associate multiple threads with a single Snort instance enabling the concurrent processing of multiple packet files. This optimization frees up additional memory for further packet processing.

In order to enable multithreading in Snort3, specify the quantity of threads designated for processing network traffic using either the '--max-packet-threads' or '-z' option.
In order to enable multithreading in Snort 3, specify the number of threads designated for processing network traffic using either the `--max-packet-threads` or `-z` option.

{{%notice Note%}}
The instructions provided have been tested on AWS EC2 Graviton4 instance, based on Neoverse V2. The examples are easiest to use if you have at least 16 cores in the system.
{{%/notice%}}

## Compile and build Snort3

To install Snort3, use a text editor to save the script below on your Arm server in a file named `install-snort.sh`.
To install Snort 3, use a text editor to save the script below on your Arm server in a file named `install-snort.sh`.

<!-- add github link for the below file [build_snort3.sh]() -->
``` bash
Expand Down Expand Up @@ -195,7 +195,7 @@ echo ' export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"'

The script takes 2 arguments:
- the directory used to build Snort3 and its dependencies
- the number of processors to use for the build
- the number of processors to use for the build.

To build in a new directory named `build` with the number of processors in your system, run the script:

Expand All @@ -205,7 +205,7 @@ bash ./install-snort.sh build `nproc`

You don't need to run the script as `root` but it assumes you are on Ubuntu 20.04 or 22.04 and have sudo permission.

When the build completes you have the snort3 directory with all compiled software, and the `snort` executable is located in `/usr/local/bin`.
When the build completes you have the `snort3` directory with all compiled software, and the `snort` executable is located in `/usr/local/bin`.

To verify the installation is complete, run the command below and see the version printed:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,13 @@ layout: learningpathall

Before testing multithreading performance, perform the following steps to configure your system:

1. Configure Grub settings
2. Set up the Snort3 rule set
3. Download the PCAP files
4. Adjust Lua configurations
1. Configure the Grub settings
2. Set up the Snort 3 rule set
3. Download the packet capture files
4. Adjust the Lua configurations
5. Review the Snort parameters

## Configure Grub settings
## Configure the Grub settings

To enable Transparent HugePages (THP) and configure CPU isolation and affinity, append the following line to the /etc/default/grub file:

Expand Down Expand Up @@ -71,7 +72,7 @@ The output shows the isolated processors:
0-9
```

## Set up the Snort3 rule set
## Set up the Snort 3 rule set

Download the rule set from https://www.snort.org/ and extract it into your working directory. You should start in the `build` directory you used to build snort.

Expand All @@ -95,9 +96,9 @@ Copy the `lua` folder from the `snort3` source directory into the rules director
cp -r snort3/lua/ Test/snortrules/
```

## Download the packet capture (PCAP) files
## Download the packet capture files

You can use any PCAP files that are relevant to your test scenario.
You can use any packet capture (PCAP) files that are relevant to your test scenario.

One place to get PCAP files is:
https://www.netresec.com/?page=MACCDC
Expand All @@ -112,21 +113,21 @@ mkdir Test/Pcap
cp maccdc2010_00000_20100310205651.pcap Test/Pcap/
```

## Adjust Lua configurations
## Adjust the Lua configurations

There are two modifications to the Lau configurations:
- Pin each Snort thread to a unique core, ensuring that the cores match those isolated in the GRUB configuration
- Enable the desired ruleset and enabling profiling
- pin each Snort thread to a unique core, ensuring that the cores match those isolated in the GRUB configuration
- enable the desired ruleset and enabling profiling.

### Pin snort threads to unique cpu core

Navigate to the `Test/snortrules/lua` directory.
Navigate to the `Test/snortrules/lua` directory:

```bash
cd Test/snortrules/lua
````

Use an editor to create a file named `common.lua` with the contents below.
Use an editor to create a file named `common.lua` with the contents below:

```bash
-------------------------------------------------------------------------------
Expand All @@ -151,7 +152,7 @@ search_engine = { }
snort_whitelist_append("threads")
```
Include the above file in `snort.lua` by editing the file and adding the line below to the end of the file.
Include the above file in `snort.lua` by editing the file and adding the line below to the end of the file:
``` bash
include('common.lua')
Expand All @@ -176,25 +177,25 @@ Continue to edit `snort.lua` and comment out the `profiler` and `latency` lines
### Modify the IPS policy
Snort3 allows you to fine-tune setups with the `--tweaks` parameter. This feature allows you to use one of Snort's policy files to enhance the detection engine for improved performance or increased security.
Snort 3 allows you to fine-tune setups with the `--tweaks` parameter. This feature allows you to use one of Snort's policy files to enhance the detection engine for improved performance or increased security.
Snort3 includes four preset policy files: max_detect, security, balanced, and connectivity.
Snort 3 includes four preset policy files: `max_detect`, `security`, `balanced`, and `connectivity`.
The max_detect policy favors maximum security, whereas the connectivity policy focuses on performance and uptime, which may come at the expense of security.
The `max_detect` policy favors maximum security, whereas the `connectivity` policy focuses on performance and uptime, which may come at the expense of security.
### Specify the data acquisition module
Snort supports DAQ modules which serves as an abstraction layer for interfacing with data source such as network interface.
Snort supports data acquisition (DAQ) modules which serve as an abstraction layer for interfacing with a data source such as a network interface.
To see list of DAQ modules supported by snort use `--daq-list` command.
To see list of DAQ modules supported by Snort use `--daq-list` command.
Return to the `build` directory:
```bash
cd $HOME/build
```
Run using the command:
Run Snort with the command:
``` bash
snort --daq-dir ./snort3/dependencies/libdaq/install/lib/daq --daq-list
Expand Down Expand Up @@ -250,15 +251,15 @@ trace(v1): inline unpriv wrapper
For testing, you can use `--daq dump` to analyze PCAP files.
## Spawn Snort3 process with multithreading
## How do I spawn a Snort 3 process with multithreading?
To run Snort3 with multithreading start from the `Test` directory.
To run Snort 3 with multithreading start from the `Test` directory.
```bash
cd $HOME/build/Test
```
The following example shows how to use multiple Snort threads to analyze PCAP files.
The following example shows how to use multiple Snort threads to analyze PCAP files:
``` bash
MPSE=hyperscan POLICY=./snortrules/lua/snort.lua TCMALLOC_MEMFS_MALLOC_PATH=/dev/hugepages/test snort -c ./snortrules/lua/snort.lua --lua detection.allow_missing_so_rules=true --pcap-filter maccdc2010_00000_20100310205651.pcap --pcap-loop 10 --snaplen 0 --max-packet-threads 10 --daq dump --daq-dir /usr/local/lib/daq --daq-var output=none -H --pcap-dir Pcap -Q --warn-conf-strict --tweaks security
Expand Down Expand Up @@ -290,7 +291,7 @@ The output is similar to:
22:52:28 9 97.50 0.00 2.50 0.00 0.00 0.00 0.00 0.00 0.00 0.00
```
## Test Snort3 multi-threading to process single pcap file
## How do I test Snort 3 multithreading to process a single pcap file?
The example usage demonstrates how multithreading increases the number of packets processed per second.
Expand Down

0 comments on commit 5987e00

Please sign in to comment.