Skip to content

Commit

Permalink
Docker ghidriff (#57)
Browse files Browse the repository at this point in the history
* Add Dockerfile and workflow
* Update readme docker section
  • Loading branch information
clearbluejar authored Dec 11, 2023
1 parent 53703e6 commit d51337f
Show file tree
Hide file tree
Showing 3 changed files with 186 additions and 8 deletions.
111 changes: 111 additions & 0 deletions .github/workflows/docker-build-publish.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Docker-PyPi-Build-Publish

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

on:
schedule:
- cron: '31 19 * * *'
push:
branches: [ "main","docker" ]
workflow_dispatch:

env:
# Use docker.io for Docker Hub if empty
REGISTRY: ghcr.io
# github.repository as <account>/<repo>
IMAGE_NAME: ${{ github.repository }}


jobs:
get-ver:
runs-on: ubuntu-latest
# Map a step output to a job output
outputs:
version: ${{ steps.step1.outputs.version }}
steps:
- id: step1
run: |
# https://stackoverflow.com/questions/70693375/how-to-output-the-latest-version-of-a-pip-package
VERSION=`python -m pip install --upgrade ghidriff== 2>&1 | grep "from versions:" | sed "s/^.*from versions: \(.*\))/\1/" | awk -F ", " '{print $NF}'`
echo $VERSION
echo "version=$VERSION" >> "$GITHUB_OUTPUT"
build:
runs-on: ubuntu-latest
needs: get-ver
permissions:
contents: read
packages: write
# This is used to complete the identity challenge
# with sigstore/fulcio when running outside of PRs.
id-token: write

steps:
- name: Checkout repository
uses: actions/checkout@v3

# Install the cosign tool except on PR
# https://github.com/sigstore/cosign-installer
- name: Install cosign
if: github.event_name != 'pull_request'
uses: sigstore/cosign-installer@6e04d228eb30da1757ee4e1dd75a0ec73a653e06 #v3.1.1
with:
cosign-release: 'v2.1.1'

# Set up BuildKit Docker container builder to be able to build
# multi-platform images and export cache
# https://github.com/docker/setup-buildx-action
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f95db51fddba0c2d1ec667646a06c2ce06100226 # v3.0.0

# Login against a Docker registry except on PR
# https://github.com/docker/login-action
- name: Log into registry ${{ env.REGISTRY }}
if: github.event_name != 'pull_request'
uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3.0.0
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

# Extract metadata (tags, labels) for Docker
# https://github.com/docker/metadata-action
- name: Extract Docker metadata
id: meta
uses: docker/metadata-action@96383f45573cb7f253c731d3b3ab81c87ef81934 # v5.0.0
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
latest
${{needs.get-ver.outputs.version}}
# Build and push Docker image with Buildx (don't push on PR)
# https://github.com/docker/build-push-action
- name: Build and push Docker image
id: build-and-push
uses: docker/build-push-action@v5 # v5.0.0
with:
context: .
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
platforms: linux/amd64,linux/arm64

# Sign the resulting Docker image digest except on PRs.
# This will only write to the public Rekor transparency log when the Docker
# repository is public to avoid leaking data. If you would like to publish
# transparency data even for private images, pass --force to cosign below.
# https://github.com/sigstore/cosign
- name: Sign the published Docker image
if: ${{ github.event_name != 'pull_request' }}
env:
# https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable
TAGS: ${{ steps.meta.outputs.tags }}
DIGEST: ${{ steps.build-and-push.outputs.digest }}
# This step uses the identity token to provision an ephemeral certificate
# against the sigstore community Fulcio instance.
run: echo "${TAGS}" | xargs -I {} cosign sign --yes {}@${DIGEST}
19 changes: 19 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# From https://github.com/clearbluejar/ghidra-python
FROM ghcr.io/clearbluejar/ghidra-python:10.4ghidra3.11python-bookworm

ENV GHIDRA_INSTALL_DIR=/ghidra

USER vscode
WORKDIR /home/vscode/

# install latest from pip
RUN pip install ghidriff

RUN python -m pyhidra.install_plugins

# point absolute ghidriffs dir to user
# this supports absoulte mapping "docker run --rm -it -v ${PWD}/ghidriffs:/ghidriffs ghidriff /bin/cat1 /bin/cat2"
RUN ln -s /ghidriffs /home/vscode/

ENTRYPOINT ["/home/vscode/.local/bin/ghidriff"]

64 changes: 56 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

It leverages the power of Ghidra's ProgramAPI and [FlatProgramAPI](https://ghidra.re/ghidra_docs/api/ghidra/program/flatapi/FlatProgramAPI.html) to find the *added*, *deleted*, and *modified* functions of two arbitrary binaries. It is written in Python3 using `pyhidra` to orchestrate Ghidra and `jpype` as the Python to Java interface to Ghidra.

Its primary use case is patch diffing. Its ability to perform a patch diff with a single command makes it ideal for automated analysis. The diffing results are stored in JSON and rendered in markdown (optionally side-by-side HTML). The markdown output promotes "social" diffing, as results are easy to publish in a gist or include in your next writeup or blog post.
Its primary use case is patch diffing. Its ability to perform a patch diff with a single command makes it ideal for automated analysis. The diffing results are stored in JSON and rendered in markdown (optionally side-by-side HTML). The markdown output promotes "social" diffing, as results are easy to publish in a gist or include in your next writeup or blog post.

## High Level

Expand Down Expand Up @@ -61,7 +61,7 @@ end
- Docker support
- Automated Testing
- Ghidra (No license required)

See below for [CVE diffs and sample usage](#sample-usage)

## Design Goals
Expand Down Expand Up @@ -150,7 +150,16 @@ options:
-o OUTPUT_PATH, --output-path OUTPUT_PATH
Output path for resulting diffs (default: ghidriffs)
--summary SUMMARY Add a summary diff if more than two bins are provided (default: False)
```
#### Extendend Usage
There are quite a few options here, and some complexity. Generally you can succeed with the defaults, but you can override the defaults as needed. One example might be to increase the JVM RAM used to run Ghidra to enable faster analysis of large binaries (`--max-ram-percent 80`). See help for details of other options.
<details><summary>Show Extended Usage</summary>
```bash
Ghidra Project Options:
-p PROJECT_LOCATION, --project-location PROJECT_LOCATION
Ghidra Project Path (default: .ghidra_projects)
Expand Down Expand Up @@ -187,7 +196,7 @@ Markdown Options:
--md-title MD_TITLE Overwrite default title for markdown diff (default: None)
```
There are quite a few options here, and some complexity. Generally you can succeed with the defaults, but you can override the defaults as needed. One example might be to increase the JVM RAM used to run Ghidra to enable faster analysis of large binaries (`--max-ram-percent 80`). See help for details of other options.
</details>
## Quick Start Environment Setup
Expand All @@ -210,21 +219,60 @@ pip install ghidriff
### Ghidriff in a Box - Devcontainer / Docker
Don't want to install Ghidra and Java on your host?
Don't want to install Ghidra and Java on your host? Try "Ghidriff in a box". It supports multiple-platforms (x64 and arm64).
<p align='center'>
<img src="https://github.com/clearbluejar/ghidriff/assets/3752074/688756fc-038c-471a-8e49-e56a1c06e77c" height="300">
</p>
`docker pull ghcr.io/clearbluejar/ghidriff:latest`
This is a docker container with the latest [PyPi version of Ghidriff](https://pypi.org/project/ghidriff/) installed. You can check the latest container [here](https://github.com/clearbluejar/ghidriff/pkgs/container/ghidriff).
#### For Docker command-line diffing
You will need to map the binaries you want to compare into the container. See below for an example.
```bash
mkdir -p ghidriffs
wget https://msdl.microsoft.com/download/symbols/clfs.sys/9848245C6f000/clfs.sys -O ghidriffs/clfs.sys.x64.10.0.22621.2506
wget https://msdl.microsoft.com/download/symbols/clfs.sys/D929C6E56f000/clfs.sys -O ghidriffs/clfs.sys.x64.10.0.22621.2715
docker run -it --rm -v $(pwd)/ghidriffs:/ghidriffs ghcr.io/clearbluejar/ghidriff:latest ghidriffs/clfs.sys.x64.10.0.22621.2506 ghidriffs/clfs.sys.x64.10.0.22621.2715
```
docker pull ghcr.io/clearbluejar/ghidra-python:latest
pip install ghidriff
The result will produce the following.
```bash
tree ghidriffs
ghidriffs
├── clfs.sys.x64.10.0.22621.2506
├── clfs.sys.x64.10.0.22621.2506-clfs.sys.x64.10.0.22621.2715.ghidriff.md
├── clfs.sys.x64.10.0.22621.2715
├── ghidra_projects
│ └── ghidriff-clfs.sys.x64.10.0.22621.2506-clfs.sys.x64.10.0.22621.2715
│ ├── ghidriff-clfs.sys.x64.10.0.22621.2506-clfs.sys.x64.10.0.22621.2715.gpr
│ ├── ghidriff-clfs.sys.x64.10.0.22621.2506-clfs.sys.x64.10.0.22621.2715.lock
│ └── ghidriff-clfs.sys.x64.10.0.22621.2506-clfs.sys.x64.10.0.22621.2715.rep
├── ghidriff.log
├── json
│ └── clfs.sys.x64.10.0.22621.2506-clfs.sys.x64.10.0.22621.2715.ghidriff.json
└── symbols
├── 000admin
├── clfs.pdb
│ ├── 6EAE8987F981603FEFA0E55DE0CE2C521
│ │ └── clfs.pdb
│ └── E3D1FEA241ECEC3DC6DB2B278A22A6A31
│ └── clfs.pdb
└── pingme.txt
```
#### For development
#### For Ghidriff development
Use the [.devcontainer](.devcontainer) in this repo. If you don't know how, follow the detailed instructions here: [ghidra-python-vscode-devcontainer-skeleton quick setup](https://github.com/clearbluejar/ghidra-python-vscode-devcontainer-skeleton#quick-start-setup---dev-container--best-option).
## Sample Usage
### Diffing a full Windows Kernel
Expand Down

0 comments on commit d51337f

Please sign in to comment.