Skip to content

Home

Jump to bottom
Qyriad edited this page Nov 1, 2022 · 5 revisions

About

Purpose

Bmputil is a tool for managing firmware for the Black Magic Probe hardware. It arose to provide an all-in-one solution for use cases that various scripts and dfu-util have historically provided, in additional to custom functionality like firmware settings. At the time of this writing, Bmputil is far from complete, but it already provides an out of the box "It Just Works" experience for flashing firmware to Black Magic Probe devices. See Usage for details on that.

Installation

While precompiled binaries and installers are not yet available for download, Bmputil is designed to be simple to build from source. As Bmputil is written in Rust, it will require a Rust toolchain setup to compile from source. The instructions on their website should get you started. After that, you can install this project with the following commands in a terminal:

git clone https://github.com/blackmagic-debug/bmputil
cd bmputil
cargo install --path .

Windows

On Windows, installation is the same — setting up a Rust toolchain, and running the above commands — but with one more prerequisite. You will need to install the Windows Driver Kit 8.0 redistributable components (link from this page).

Cross Compiling

This project is designed with seamless cross compilation in mind. Cross compiling for Windows is actually the simplest, as cargo-xwin will set up most of a Windows toolchain for you, though you will need a clang installation (MinGW-based toolchains do not work, as they fail to compile the external dependency libusb1-sys). You will only need to download and extract the WDK 8.0 redistributable components to some directory, and set the WDK_DIR environment variable to that directory. For example:

rustup target add x86_64-pc-windows-msvc
wget --content-disposition "https://go.microsoft.com/fwlink/p/?LinkID=253170"
msiextract wdfcoinstaller.msi
mkdir -p ~/.local/opt/wdk
mv "Program Files/Windows Kits/8.0" ~/.local/opt/wdk/8.0
export WDK_DIR=~/.local/opt/wdk/8.0
cargo xwin build --release --target=x86_64-pc-windows-msvc

For Linux and macOS targets, you will unfortunately need to setup a cross toolchain yourself. However with that done compiling Bmputil should be as simple as e.g. cargo build --release --target=aarch64-apple-darwin.

For advanced details on how the cross compilation works for Windows targets behind the scenes, take a look at Internals.

Usage

To start, you can see if Bmputil is working correctly by running bmputil info on the command line to display information about all detected Black Magic Probe devices. If you're on Windows and this is the first time you've run Bmputil then it will pop up a UAC prompt in order to install drivers for the Black Magic Probe device. If you want to know how the driver installation works behind the scenes, take a look at at Internals.

The most useful operation Bmputil can perform at the moment is flashing firmware to a Black Magic Probe device. You can download firmware binaries at https://github.com/blackmagic-debug/blackmagic/releases. Bmputil supports binaries in raw binary (.bin) or ELF (.elf) formats. After downloading one of those, flashing that firmware can be done with the bmputil flash <file> command. For example, if you downloaded blackmagic-native-v1_8_2.elf to your "Downloads" folder, you could use the following commands:

cd Downloads
bmputil flash blackmagic-native-v1_8_2.elf

Internals

Project Structure

The core of the logic is in src/bmp.rs, in the struct BmpDevice. The way these structs are typically constructed is a little involved, however. There is a simple fn from_usb_device(device: rusb::Device<rusb::Context>) -> Result<Self, Error> constructor, but Bmputil is designed to be able to handle cases where multiple Black Magic Probes are plugged in to the same machine, and to allow the user to filter these devices with command-line arguments, so construction in the codebase often involves the wrapper struct BmpMatcher. This struct stores a device index, serial number, and port (all optional), and uses those to filter the BMP devices that are attached to the machine. The method find_matching_probes() then constructs all matching BMP devices it found with from_usb_device(). The value that find_matching_probes() returns is also a helper struct, BmpMatchResults, which contains all found devices, what devices it filtered out, and what errors it encountered along the way,

The struct BmpMatchResults also contains helper functions for getting the found devices out of its results and printing helpful warnings to the console based on the results, so the user isn't confused e.g by a DeviceNotFound error because they passed the wrong --serial.

For example, to find all attached BMPs with the serial number "FOOBARDEADBEEF":

let matcher = BmpMatcher {
    index: None,
    serial: Some(String::from("FOOBARDEADBEEF")),
    port: None,
};

// Will print a warning to the console if the serial number filtered out the only BMP(s) connected to the system,
// so the user hopefully isn't so confused by a DeviceNotFound error if they got the serial number wrong in `--serial`.
let found_probes = matcher.find_matching_probes().pop_all()?;

Once a BmpDevice is constructed, flashing is done with the BmpDevice::download() method. This requires passing the firmware type, which can be detected automatically with FirmwareType::detect_from_firmware(), which parses the ARM vector table at the beginning of the firmware to determine where the binary is linked.

After download() has finished, however, the physical device represented by the BmpDevice struct will reboot. With the device rebooted, the OS handles BmpDevice has will no longer be valid. self is not consumed by download(), but any further methods that require IO to the device will fail. You can use the free function wait_for_probe_reboot() with the port string the device was on (which can be retrieved with BmpDevice::port()) to re-create a BmpDevice for the newly rebooted Black Magic Probe device.

Logic for handling the DFU protocol itself largely is handled by the dfu-core and dfu-libusb external library crates.

Though only code for the native Black Magic Probe platform is implemented at the time of this writing, the codebase has scaffolding for supporting multiple different Black Magic Probe platforms, largely encapsulated in the enum BmpPlatform. Adding a new platform should roughly involve the following:

  1. Add a variant for the new platform to the BmpPlatform enum
  2. Add two new constants for the VID/PID pairs of the platform to match pub const NATIVE_RUNTIME_VID_PID and pub const NATIVE_DFU_VID_PID
  3. Add match arms for the new platform VID/PIDs to the match expressions in the BmpPlatform functions from_vid_pid(), runtime_ids(), and dfu_ids()
  4. Add a match arm to the match expression in BmpPlatform::load_address() to indicate where firmware is loaded for that platform

Error Typing

Unlike some Rust projects, Bmputil does not use a crate like anyhow for error construction and reporting. Instead, in src/error.rs, there is an Error struct and an associated ErrorKind enum like Rust's standard library io module. ErrorKind attempts to divide errors into top-level categories primarily meant for consumption by the user — the user should be able to understand what the error at least means, even if they don't understand why they're getting it, and they should at least be able to know what part of the attempted operation failed. The error types don't hide information, though — sources of errors (i.e. OS IO calls) are conserved in Error structs, and when reported to the user are printed as part of the error chain (and part of the backtrace, if compiled with a supported Rust version).

Because of the separation between Error and ErrorKind, error construction looks a little different than in some Rust projects, but care has been taken to ensure it's both convenient and readable.

To create an Error with no source error (i.e. a semantic error that doesn't come from something like a failed syscall), use the .error() method on the relevant ErrorKind variant. For example, to indicate that a BMP device was not found, you may write:

return Err(ErrorKind::DeviceNotFound.error());

To create an Error with a source error (such as a failed syscall or IO operation), use the .error_from() method on the relevant ErrorKind variant. For example:

let open_dev = || Err(std::io::Error::from(std::io::ErrorKind::TimedOut));
open_dev().map_err(|io_error| ErrorKind::DeviceNotFound.error_from(io_error))?;

Some ErrorKind variants have a data field to give a little bit of extra information about what thing the error is relevant to. For example, FirmwareFileIo definition is FirmwareFileIo(/** filename **/ Option<String>), allowing the programmer to indicate the name of the file that couldn't be read/written to/opened/etc. Ex:

let filename = String::from("firmware.elf");
let firm_file = std::fs::File::open(&filename)
    .map_err(|file_err| ErrorKind::FirmwareFileIo(Some(filename)))?;

Finally, Errors can have context, which is a string that indicates to the user the operation the error occurred in. To complete the above example:

let filename = String::from("firmware.elf");
let firm_file = std::fs::File::open(&filename)
    .map_err(|file_err| ErrorKind::FirmwareFileIo(Some(filename)).error_from(file_err))
    .map_err(|e| e.with_ctx("determining firmware kind"))?;

On stable, this prints:

Error: (while determining firmware kind): failed to read firmware file firmware.elf
Caused by: entity not found

note: recompile with nightly toolchain and run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

When constructing your errors, consider adding context so the user knows what step failed (and, if reasonable, why that step was being performed), and consider if the error has a source error that caused it. Anything with the bounds of E: std::error::Error + Send + Sync + 'static can be passed to .error_from().

Windows Compilation

Compilation for Windows targets (whether you're on Windows or not) is internally a fair bit more complicated. This is largely because Bmputil on Windows depends on libwdi, and the Rust bindings crate also builds and statically links in libwdi as part of the build process. libusb-sys's build.rs contains most of the complexity, which comes mostly from the following:

  • libwdi's build process includes executable artifacts, which cc-rs does not support
    • cargo-xwin, when cross compiling, also does not provide the linker arguments we need to link executables, so we have to infer them from the arguments we do get
  • libwdi's build process also includes a host executable artifact, which must be executed to generate headers
  • Some of libwdi's sources require patching to build under the conditions we need
  • On macOS, absolute paths tend to be misinterpreted by clang-cl, as paths like /Users/foobar/some/path get interpreted as MSVC's /U switch
    • This one we solve with a slightly terrible hack of simply prepending a second / to absolute paths on macOS, becoming things like //Users/foobar/some/path

libwdi-sys's build script is well commented, but here is an overview of the steps it takes:

  1. Completely copy all needed source files from the libwdi-sys submodule, patching the files that need to be patched (and converting the patch files from CRLF to LF if need be, to be independent of the users core.autocrlf Git config value)
  2. If cross compiling, copy libwdi's config.h to a separate directory so it can be included for host builds without overriding host system headers
  3. If cross compiling, grab the path to the WDK redistributable components from a WDK_DIR environment variable
  4. Compile the host binary embedder by asking cc-rs to do most of the work, and then converting the cc::Build to a std::process::Command and adding the necessary arguments
  5. Compile the target binary installer_x64.exe, using the same method as above, also parsing out the Windows SDK path from what cargo-xwin has given us if necessary, so the target binary can be linked
  6. Run the host binary created in step 4, which generates a header used in step 7
  7. Finally, compile wdi.lib
Clone this wiki locally