Adversarial Patch for 3D Local Feature Extractor

Install packages

pip intsall -r requirements.txt

Prepare data

• Download the data with the mask annotations from data.tar.gz

• Or download the original HPatches data from data hpatches-sequences-release.tar.gz from HPatches, and generate your own mask by maskgen.py.

Patch generation

Chessboard pattern

python3 chessboard.py [-h] [--rect_width RECT_WIDTH] [--rect_height RECT_HEIGHT]
                      [--width WIDTH] [--height HEIGHT] [--save SAVE]

Adversarial patch

python3 patchgen.py [-h] [--cuda] [--epoch EPOCH] [--width WIDTH]
                    [--height HEIGHT] [--decay DECAY] [--alpha ALPHA]
                    [--multiplier MULTIPLIER] [--save SAVE] [--model MODEL]
                    [--untargeted] [--init INIT] [--init-pattern INIT_PATTERN]
                    [--prob PROB]

Mask generation

python3 maskgen.py [-h] [--dirs [DIRS ...]] [--dir DIR] [--mask-file MASK_FILE]
                   [--patch-width PATCH_WIDTH] [--patch-height PATCH_HEIGHT]
                   [--H H] [--individual] [--overlapping]

Evaluate the attack

python3 patch_eval.py [-h] [--dirs [DIRS ...]] [--dir DIR] [--mask-file MASK_FILE]
                      [--null-mask] [--match-point-num MATCH_POINT_NUM]
                      [--patch-file PATCH_FILE] [--model MODEL] [--model-weight MODEL_WEIGHT]
                      [--device DEVICE] [--log LOG]

eg.

• Evaluates the chess-init with size 128 while attacking the targeted viewpoints using masking size 128.

python3 patch_eval.py \
    --dirs data/hpatches-mask/* \
    --mask-file mask_indiv_128.json \
    --patch-file patches/patch_chess_w128_h128.png \
    --device cuda \
    --log log/

• Evaluates the chessboard pattern with size 100 while attacking the untargeted viewpoints using masking size 128 and the model is SIFT.

python3 patch_eval.py \
    --dirs data/hpatches-mask/* \
    --mask-file mask_128.json \
    --patch-file patches/chessboard_128.png \
    --model sift \
    --device cuda \
    --log log/

Visual results of physical attacks

The green point is the points in other viewpoint that match with the points in the source mask of the source viewpoint.

Folder structure

.
├── chessboard.py
├── data
│   └── hpatches-mask
│       ├── v_abstract/
│       └── ...
├── homography_transforms.py
├── LICENSE
├── maskgen.py
├── models
│   ├── __init__.py
│   ├── sift.py
│   ├── superpoint.py
│   └── superpoint_v1.pth
├── patches
│   ├── chessboard_100.png
│   └── ...
├── patch_eval.py
├── patchgen.py
├── README.md
├── report.pdf
└── requirements.txt