-
Notifications
You must be signed in to change notification settings - Fork 541
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Linux XDP support #3849
Merged
Merged
Linux XDP support #3849
Changes from 176 commits
Commits
Show all changes
179 commits
Select commit
Hold shift + click to select a range
420590b
built on Linux
ami-GS d63fd7f
ne files
ami-GS 8cb383d
renaming
ami-GS a32d79b
split socket funcs
ami-GS ba2b925
refactor from win
ami-GS 7667acc
build from linux
ami-GS 5cfd379
more on linux
ami-GS 05d191a
rename and generate
ami-GS 11773d7
use libbpf
ami-GS 7e71548
oops
ami-GS 2d79d2e
rebase and add new member
ami-GS 2ca3399
unify CxPlatTryAddSocket
ami-GS 74ab9cb
pass datapath tests
ami-GS cf46e48
integrate Ubuntu 22.04 default libbpf v0.5.0
ami-GS ef8bd89
temporally
ami-GS 8647673
multithread. all test passed
ami-GS 1f2b3fd
secnetperf works
3483760
update build scripts
ami-GS b4656a4
nit update
ami-GS b3805f0
add duonic installer script for linux
ami-GS 3ae0c0a
add xdp-tools directory
ami-GS 1b74349
Merge commit '707c1fab' into dev/daiki/xdp_split_functional
ami-GS 008ab54
Merge commit '88f0269' into dev/daiki/xdp_split_functional
ami-GS 20961da
Merge commit 'dc64ccb47adf847d01f171bd3ccab1818b5d92d8' into dev/daik…
ami-GS 17d9fa0
Merge commit 'f596ae5313a3b925556890482a2839a81c9d0392' into dev/daik…
ami-GS 9a6bcae
Merge commit 'b6633eace10ed493cb234f86e18cc6d9d7526417' into dev/daik…
ami-GS a2c7477
Merge commit '8af9c3b214e83ea3146f88c072a0220a22a48ee6' into dev/daik…
ami-GS 846aa31
Merge commit 'c885d5f1fdab56a8630aa1ec8a0b0bc41da291ca' into dev/daik…
ami-GS 59aa678
Merge commit 'e6d504407e3a0e739d3da4c3393da43af61ae69f' into dev/daik…
ami-GS 170279d
cleanup
ami-GS aa8f0d9
fix Windows build
ami-GS 457dd4a
fix windows xdp build
ami-GS df4c7bd
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS 1ffded4
add Github action
ami-GS d57341d
nit fix
ami-GS 421ea69
fix dependencies for Install-DuoNic
ami-GS c57ee78
add udpate
ami-GS ee0e042
add linux xdp dependencies to docker file
ami-GS a744683
fix installation script
ami-GS 7dc44cd
initialize submodules/xdp-tools recursively
ami-GS b5a846b
Merge commit 'f9d26ac' into dev/daiki/xdp_split_functional
ami-GS 027e886
Merge commit 'ae3db5cd991ea872efcf9e32dbe63f4f203245af' into dev/daik…
ami-GS 2f1687f
Merge commit '284738a861272c63f31d900c490e9d5e856c8ac2' into dev/daik…
ami-GS 3d1e986
Merge commit '4ef3731d82757451f9b62df0e7b41f32c87a1392' into dev/daik…
ami-GS 6cf52f4
Merge commit '649f971b1bcce59ce0aba2d30a199d19dd91a4d2' into dev/daik…
ami-GS 2f21a23
Merge commit '8e1eb1a74b6baed0af7dad899b31baf970051d26' into dev/daik…
ami-GS f569699
Merge commit '6e1c3d6af6ecade0979ec3430221c8920e562e91' into dev/daik…
ami-GS a122ae7
Merge commit 'fe9c2971a3abb432f92fff86664f1f131fe728b1' into dev/daik…
ami-GS bbe0af7
Merge commit 'e73ce1082364c697f89bfaf81c67b3158b9987c0' into dev/daik…
ami-GS e803c64
Merge commit '9483184dd39559f20daf57f95fdbcb14c367ff01' into dev/daik…
ami-GS 6dd87ff
Merge commit 'cacfccae195ab36f91fd7e2c27ab56a863935ba3' into dev/daik…
ami-GS 1dd8e52
temporal fix
ami-GS 23b7b06
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS b3743dc
make XDP optional
ami-GS f34a3e8
use ShutdownSqe
ami-GS 560730b
fix yaml files
ami-GS 8c8676f
remove gcc-multilib and include explicitly
ami-GS d69e762
fix tcp socket to allocate raw
ami-GS 943d91f
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS 649b948
install xdp deps for linux
ami-GS d7a50b4
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS a2b332e
libxdp bug workaround
ami-GS baf452e
oops
ami-GS 1ff6ba3
fix idempotency and add one more dep
ami-GS 29f9898
TODO: remove this
ami-GS eab2f7f
enable x86_64 and Ubuntu22.04
ami-GS 1802e4e
fix type mismatch of uint16_t and UINT16
ami-GS b4c7ce6
add aarch64
ami-GS e375786
fix
ami-GS 35b63f1
use flag
ami-GS c48bdbb
change arch check and file copy
ami-GS 92a0b8c
fix static build and android
ami-GS c074126
update clog
ami-GS 05d86a8
clog fix
ami-GS 7999808
fix Clang link issue
ami-GS 063f325
fix CodeCheck and prepare-machine for build
ami-GS bdd0161
disable ANDROID for xdp
ami-GS cdaf797
fix clog
ami-GS d1639dd
Support XDP from 'test.ps1 -UseXdp'
ami-GS dd6a74d
link libxdp from artifact
ami-GS 5acb167
fix dependency
ami-GS 671e112
fix dep
ami-GS 1d69ad9
update xdp-tools to v1.4.2
ami-GS ebb21b9
fix runtime deps
ami-GS 7106913
fix deps
ami-GS 8fa7f5c
Fix Umem addressing and make Rx path async to efficiently fill FQ
ami-GS 2983028
fix CodeQL analyze deps
ami-GS e3be961
fix Analyze
ami-GS 7f68f6e
fix analyze
ami-GS d3c7eb6
linux xdp to use sudo
ami-GS 2100f5e
change 'if' order
ami-GS 360c28a
path fix?
ami-GS d434b3d
sudo pwsh
ami-GS 89a361c
add retry for xsk_socket__create when returns -EBUSY
ami-GS aa8cf57
cleanup xdp program before running
ami-GS 70d4622
Fix cargo build --all
ami-GS 5d9b1e8
fix cargo test dynamic linking
ami-GS 9490d94
remove Sanitize for initial commit
ami-GS d17770c
generate artifact
ami-GS e30e083
fix yml syntax issue
ami-GS 19f7abc
fix CodeQl Analysys
ami-GS 3cae069
forget to add Partition declaration
ami-GS e694653
Use Rundown to wait for deleting socket before binding next
ami-GS 2c41090
fix Rundown for all xdp socket
ami-GS 50844d2
Fix EBUSY issue when binding xdp socket
ami-GS c9fd1e3
explictly disable port sharing test
ami-GS 90f81b6
refactoring and cleanup
ami-GS d64dbd1
fix pipeline. else for normal linux socket
ami-GS 02ffcca
Fix cargo linking issue?
ami-GS ed95167
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS 10301b3
disable xdp for cargo build
ami-GS 06acce7
fix codeql
ami-GS 2d17796
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS d8d122b
rollback build.rs
ami-GS 4c878e0
fix package runtime link and codeql
ami-GS faedae1
wrap packet dump function to DEBUG macro
ami-GS b1633e6
check dependency
ami-GS 0bf636f
another check
ami-GS 293f8bd
leave only LD_LIBRARY_PATH
ami-GS 6b49bb6
iterate attach mode
ami-GS 8aa3402
fix codeql
ami-GS d56ce17
add IP address check in XDP prog
ami-GS 8c8f463
break is enough & avoid compiler bug
ami-GS c305f11
fix CodeCheck
ami-GS 0222e70
Fix use of RxHeadroom
ami-GS 606c3b8
fix umem leak when failed to initialize
ami-GS 0c782c8
use pool alloc for Interface
ami-GS 51cdebc
fix ipv4 check
ami-GS d3dc703
remove workaround
ami-GS b4decc6
cleanup. from snake case to pascal case
ami-GS d1588b6
fix clog
ami-GS 6cd580c
try ubuntu20.04 and enable stress test
ami-GS 706e4da
few fix of stress test yml
ami-GS 56a7fb1
integrate package libxdp linking
ami-GS 600ad66
fix RX parse error case
ami-GS ca7bde5
disable ubuntu20.04
ami-GS 4569402
remove 2 env variable to run
ami-GS 23a9291
add document
ami-GS f96feab
fix ForBuild && clog
ami-GS 4b829ce
stop installing deps on Ubuntu 20.04
ami-GS 7ce0f8d
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS 31d817f
fix build
ami-GS 1d43c09
fix libc6-dev-i386 side effect
ami-GS ae95b04
Add one more dependency (libzstd) for static build
ami-GS e27243f
new line fix
ami-GS 05376dc
always install libc6-dev-i386
ami-GS 6c792c8
fix install order
ami-GS 561044e
fix CMakeLists.txt to cover qns.Dockerfile case
ami-GS 2f4f171
remove env var dep, update doc
ami-GS c88fe20
fix stress.yml
ami-GS 7f6fbd0
add env var for xdp knob and fix stress
ami-GS 2b25b39
fix stress
ami-GS d31a0f5
update clog
ami-GS 57bfa9f
update doc
ami-GS 714a655
Fix doc appearance and words
ami-GS 6768679
add UseXdp
ami-GS f0dab8b
Update docs/BUILD.md
ami-GS bbe0a5a
Update docs/BUILD.md
ami-GS ad74b33
use SKB mode for now
ami-GS af18bef
move Linux XDP logging to TSG.md
ami-GS 9a7acdf
stop warning print from libxdp/libbpf
ami-GS 14a8fd7
move CxPlat definitions to each plat
ami-GS e961a3b
initial commit
ami-GS 8ecc440
common xdp utilities
ami-GS 5f1da03
move sudo inside scripts
ami-GS 6d2339c
fix comments
ami-GS 037ea20
Merge branch 'dev/daiki/xdp_merge' into dev/daiki/xdp_split_functional
ami-GS c87e720
fix clog
ami-GS ed69f0a
Merge branch 'main' into dev/daiki/xdp_split_functional
ami-GS e560f3f
fix ifdef syntax
ami-GS 2cfa516
fix annotations
ami-GS 9cd42d0
all arch to build dataptah_xplat.c
ami-GS 17825c0
fix mac build
ami-GS 74eed9f
fix lisence of ebpf prog
ami-GS eebc505
fix spin.ps1 to propagate parameter
ami-GS 4eda469
Ether and IPv6 was not set appropriately
ami-GS 8924bed
Update docs/TSG.md
nibanks 1703841
remove warning supression
ami-GS 98a1222
remove BuildLibXdpFromSource
ami-GS File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -64,6 +64,9 @@ elseif (APPLE) | |
set(CX_PLATFORM "darwin") | ||
elseif (UNIX) | ||
set(CX_PLATFORM "linux") | ||
file(STRINGS "/etc/lsb-release" LSB_RELEASE_CONTENT) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This line broke our Alpine docker build, using /etc/os-release would be better as it is present in both distributions. |
||
string(REGEX MATCH "DISTRIB_RELEASE=([0-9]+\\.[0-9]+)" _ ${LSB_RELEASE_CONTENT}) | ||
set(UBUNTU_VERSION ${CMAKE_MATCH_1}) | ||
endif() | ||
message(STATUS "QUIC Platform: ${CX_PLATFORM}") | ||
|
||
|
@@ -434,13 +437,13 @@ if (NOT MSVC AND NOT APPLE AND NOT ANDROID) | |
endif() | ||
endif() | ||
|
||
if(WIN32) | ||
if (CMAKE_GENERATOR_PLATFORM STREQUAL "") | ||
string(TOLOWER ${CMAKE_SYSTEM_PROCESSOR} SYSTEM_PROCESSOR) | ||
else() | ||
string(TOLOWER ${CMAKE_GENERATOR_PLATFORM} SYSTEM_PROCESSOR) | ||
endif() | ||
if (CMAKE_GENERATOR_PLATFORM STREQUAL "") | ||
string(TOLOWER ${CMAKE_SYSTEM_PROCESSOR} SYSTEM_PROCESSOR) | ||
else() | ||
string(TOLOWER ${CMAKE_GENERATOR_PLATFORM} SYSTEM_PROCESSOR) | ||
endif() | ||
|
||
if(WIN32) | ||
# Generate the MsQuicEtw header file. | ||
file(MAKE_DIRECTORY ${QUIC_BUILD_DIR}/inc) | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
#!/bin/bash | ||
|
||
# Set the number of NIC pairs | ||
NumNicPairs=1 | ||
|
||
if [ "$1" == "install" ]; then | ||
# Configure each pair separately with its own hard-coded subnet, ie 192.168.x.0/24 and fc00::x/112 | ||
for ((i=1; i<=NumNicPairs; i++)); do | ||
echo "Plumbing IP config for pair $i" | ||
|
||
# Generate the "ID" of the NICs, eg 1 and 2 for the first pair | ||
nic1="duo$((i * 2 - 1))" | ||
nic2="duo$((i * 2))" | ||
|
||
# Create veth pair | ||
sudo ip link add ${nic1} type veth peer name ${nic2} | ||
|
||
# Set the veth interfaces up | ||
sudo ip link set ${nic1} up | ||
sudo ip link set ${nic2} up | ||
|
||
# Assign IPv4 addresses | ||
sudo ip addr add 192.168.${i}.11/24 dev ${nic1} | ||
sudo ip addr add 192.168.${i}.12/24 dev ${nic2} | ||
|
||
# Assign IPv6 addresses | ||
sudo ip -6 addr add fc00::${i}:11/112 dev ${nic1} | ||
sudo ip -6 addr add fc00::${i}:12/112 dev ${nic2} | ||
|
||
# Add static neighbor entries (ARP) | ||
sudo ip neigh add 192.168.${i}.12 lladdr 22:22:22:22:00:0$((i * 2)) dev ${nic1} nud permanent | ||
sudo ip neigh add 192.168.${i}.11 lladdr 22:22:22:22:00:0$((i * 2 - 1)) dev ${nic2} nud permanent | ||
|
||
# Add static neighbor entries (IPv6) | ||
sudo ip -6 neigh add fc00::${i}:12 lladdr 22:22:22:22:00:0$((i * 2)) dev ${nic1} nud permanent | ||
sudo ip -6 neigh add fc00::${i}:11 lladdr 22:22:22:22:00:0$((i * 2 - 1)) dev ${nic2} nud permanent | ||
|
||
# Configure routing rules for IPv4 | ||
sudo ip route add 192.168.${i}.12/32 dev ${nic1} metric 0 | ||
sudo ip route add 192.168.${i}.11/32 dev ${nic2} metric 0 | ||
|
||
|
||
|
||
# Configure routing rules for IPv6 | ||
sudo ip -6 route add fc00::${i}:12/128 dev ${nic1} metric 0 | ||
sudo ip -6 route add fc00::${i}:11/128 dev ${nic2} metric 0 | ||
|
||
# Configure firewall rules for IPv4 | ||
sudo iptables -A INPUT -p all -s 192.168.${i}.0/24 -i ${nic1} -j ACCEPT | ||
sudo iptables -A INPUT -p all -s 192.168.${i}.0/24 -i ${nic2} -j ACCEPT | ||
|
||
# Configure firewall rules for IPv6 | ||
sudo ip6tables -A INPUT -p all -s fc00::${i}:0/112 -i ${nic1} -j ACCEPT | ||
sudo ip6tables -A INPUT -p all -s fc00::${i}:0/112 -i ${nic2} -j ACCEPT | ||
|
||
sleep 2 | ||
sudo ip route change 192.168.${i}.12 dev duo1 src 192.168.${i}.11 | ||
sudo ip route change 192.168.${i}.11 dev duo2 src 192.168.${i}.12 | ||
|
||
sudo ip -6 route change fc00::${i}:12 dev duo1 src fc00::${i}:11 | ||
sudo ip -6 route change fc00::${i}:11 dev duo2 src fc00::${i}:12 | ||
done | ||
elif [ "$1" == "uninstall" ]; then | ||
# Cleanup each pair separately | ||
for ((i=1; i<=NumNicPairs; i++)); do | ||
echo "Cleaning up pair $i" | ||
|
||
# Generate the "ID" of the NICs, eg 1 and 2 for the first pair | ||
nic1="duo$((i * 2 - 1))" | ||
nic2="duo$((i * 2))" | ||
|
||
# Delete firewall rules for IPv4 | ||
sudo iptables -D INPUT -p all -s 192.168.${i}.0/24 -i ${nic1} -j ACCEPT | ||
sudo iptables -D INPUT -p all -s 192.168.${i}.0/24 -i ${nic2} -j ACCEPT | ||
|
||
# Delete firewall rules for IPv6 | ||
sudo ip6tables -D INPUT -p all -s fc00::${i}:0/112 -i ${nic1} -j ACCEPT | ||
sudo ip6tables -D INPUT -p all -s fc00::${i}:0/112 -i ${nic2} -j ACCEPT | ||
|
||
# Remove routing rules for IPv4 | ||
sudo ip route del 192.168.${i}.12/32 dev ${nic1} | ||
sudo ip route del 192.168.${i}.11/32 dev ${nic2} | ||
|
||
# Remove routing rules for IPv6 | ||
sudo ip -6 route del fc00::${i}:12/128 dev ${nic1} | ||
sudo ip -6 route del fc00::${i}:11/128 dev ${nic2} | ||
|
||
# Remove static neighbor entries (ARP) | ||
sudo ip neigh del 192.168.${i}.12 lladdr 22:22:22:22:00:0$((i * 2)) dev ${nic1} | ||
sudo ip neigh del 192.168.${i}.11 lladdr 22:22:22:22:00:0$((i * 2 - 1)) dev ${nic2} | ||
|
||
# Remove static neighbor entries (IPv6) | ||
sudo ip -6 neigh del fc00::${i}:12 lladdr 22:22:22:22:00:0$((i * 2)) dev ${nic1} | ||
sudo ip -6 neigh del fc00::${i}:11 lladdr 22:22:22:22:00:0$((i * 2 - 1)) dev ${nic2} | ||
|
||
# Remove the veth pair | ||
sudo ip link delete ${nic1} | ||
done | ||
else | ||
echo "Usage: $0 {install|uninsatll}" | ||
exit 1 | ||
fi |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
XDP doesn't work with sanitizer? Why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I previously saw data alignment related error by test.yml.
Not sure stress test. I enabled stress test later than test.yml
I disabled sanitizer from functionality point of view
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should follow up (in a later PR) on the sanitizer.