Skip to content

Testing on a big endian platform

Vincent Fu edited this page Mar 4, 2024 · 1 revision

I have been studying fio's verify code recently and examining the verify state file. I noticed that struct thread_io_list defines depth and nofiles as 32-bit unsigned integers.

/*
 * For dumping current write state
 */
struct file_comp {
        uint64_t fileno;
        uint64_t offset;
};

struct thread_io_list {
        uint64_t no_comps;
        uint32_t depth;
        uint32_t nofiles;
        uint64_t numberio;
        uint64_t index;
        struct thread_rand_state rand;
        uint8_t name[64];
        struct file_comp comps[0];
};

However, when this structure is being prepared for writing to disk, these two members are treated as 64-bit unsigned integers:

        s->depth = cpu_to_le64((uint64_t) td->o.iodepth);
        s->nofiles = cpu_to_le64((uint64_t) td->o.nr_files);

When these two members are loaded from disk they are appropriately converted as 32-bit unsigned integers: 

        s->depth = le32_to_cpu(s->depth);
        s->nofiles = le32_to_cpu(s->nofiles);

Testing this functionality revealed that this type mismatch produces no issues on my x86_64 platform. This is not a surprise since x86_64 is a little endian platform and the cpu_to_le64 and le32_to_cpu functions are essentially noops.

What about a big-endian platform? On such a system, when the byte order is reversed, zero will be assigned to s->depth and s->nofiles as the most significant 32 bits of the result of cpu_to_le64 will be discarded.

How can we test this? Internet searching led me to Giovanni Mascellani's Debian Quick Image Baker project. It creates Debian images for a variety of architectures that can be run with QEMU. I found the powerpc-g3beige platform listed on the project page, downloaded the artifacts from the link, and unzipped the resulting file.

vincent@localhost:~/fio-dev/ppc-g3beige$ wget https://gitlab.com/api/v4/projects/giomasce%2Fdqib/jobs/artifacts/master/download?job=convert_powerpc-g3beige
--2024-02-16 09:28:04--  https://gitlab.com/api/v4/projects/giomasce%2Fdqib/jobs/artifacts/master/download?job=convert_powerpc-g3beige
Resolving gitlab.com (gitlab.com)... 172.65.251.78, 2606:4700:90:0:f22e:fbec:5bed:a9b9
Connecting to gitlab.com (gitlab.com)|172.65.251.78|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn.artifacts.gitlab-static.net/9d/f6/9df69d693033f3071b9dee279b40dead9517d1ec9a95eb4bf3404e5d8767b683/@final/1d/79/b34ad77a76f2b3668d02256891b81658313ad584e78e6beda4b4fdfc1642?Expires=1708105084&KeyName=gprd-artifacts-cdn&Signature=Ger4a4phacne2GzyAlDZ82xTBSc= [following]
--2024-02-16 09:28:04--  https://cdn.artifacts.gitlab-static.net/9d/f6/9df69d693033f3071b9dee279b40dead9517d1ec9a95eb4bf3404e5d8767b683/@final/1d/79/b34ad77a76f2b3668d02256891b81658313ad584e78e6beda4b4fdfc1642?Expires=1708105084&KeyName=gprd-artifacts-cdn&Signature=Ger4a4phacne2GzyAlDZ82xTBSc=
Resolving cdn.artifacts.gitlab-static.net (cdn.artifacts.gitlab-static.net)... 34.110.204.38
Connecting to cdn.artifacts.gitlab-static.net (cdn.artifacts.gitlab-static.net)|34.110.204.38|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 288900861 (276M) [application/octet-stream]
Saving to: ‘download?job=convert_powerpc-g3beige’

download?job=convert_powerpc-g3beige                                    100%[============================================================================================================================================================================>] 275.52M  46.7MB/s    in 6.5s

2024-02-16 09:28:11 (42.1 MB/s) - ‘download?job=convert_powerpc-g3beige’ saved [288900861/288900861]

vincent@localhost:~/fio-dev/ppc-g3beige$ ls -l
total 282132
-rw-rw-r-- 1 vincent vincent 288900861 Feb 10 22:32 'download?job=convert_powerpc-g3beige'
vincent@localhost:~/fio-dev/ppc-g3beige$ mv download\?job\=convert_powerpc-g3beige ppc-g3beige.zip
vincent@localhost:~/fio-dev/ppc-g3beige$ unzip ppc-g3beige.zip
Archive:  ppc-g3beige.zip
   creating: dqib_powerpc-g3beige/
  inflating: dqib_powerpc-g3beige/image.qcow2
  inflating: dqib_powerpc-g3beige/initrd
  inflating: dqib_powerpc-g3beige/kernel
  inflating: dqib_powerpc-g3beige/readme.txt
  inflating: dqib_powerpc-g3beige/ssh_user_ecdsa_key
  inflating: dqib_powerpc-g3beige/ssh_user_ed25519_key
  inflating: dqib_powerpc-g3beige/ssh_user_rsa_key

Each set of artifacts also includes a readme.txt document with a very helpful QEMU invocation line.

vincent@localhost:~/fio-dev/ppc-g3beige$ cd dqib_powerpc-g3beige/
vincent@localhost:~/fio-dev/ppc-g3beige/dqib_powerpc-g3beige$ ls -l
total 686192
-rw-r--r-- 1 vincent vincent 655818752 Feb 10 22:30 image.qcow2
-rw-r--r-- 1 vincent vincent  24032022 Feb 10 22:26 initrd
-rw-r--r-- 1 vincent vincent  22786468 Feb  4 12:28 kernel
-rw-r--r-- 1 vincent vincent       832 Feb 10 22:26 readme.txt
-rw-r--r-- 1 vincent vincent       513 Feb 10 22:24 ssh_user_ecdsa_key
-rw-r--r-- 1 vincent vincent       411 Feb 10 22:24 ssh_user_ed25519_key
-rw-r--r-- 1 vincent vincent      2602 Feb 10 22:24 ssh_user_rsa_key
vincent@localhost:~/fio-dev/ppc-g3beige/dqib_powerpc-g3beige$ cat readme.txt
This is a Debian image generated by the Debian Quick Image Baker.
It was created on Sun Feb 11 06:26:23 UTC 2024
See https://gitlab.com/giomasce/dqib for more information.
(Try to) boot with:
qemu-system-ppc -machine 'g3beige' -cpu 'g4' -m 1G -drive file=image.qcow2 -device e1000,netdev=net -netdev user,id=net,hostfwd=tcp::2222-:22 -kernel kernel -initrd initrd -nographic -append "root=LABEL=rootfs console=ttyPZ0"
You can use Ctrl-a x to exit from QEMU.
You can login with root:root or debian:debian, or using the identities ssh_user_*_key in this directory.
You can also login with SSH on the local port 2222 (this will give you a much better terminal emulation than the QEMU default).
Some machines support a video interface, which you can enable removing the -nographic option and possibly also the console= kernel argument.

I used the provided invocation (increasing the memory to the maximum of 2047M) and several moments later saw the login prompt: 

vincent@localhost:~/fio-dev/ppc-g3beige/dqib_powerpc-g3beige$ /home/vincent/qemu/qemu/build/qemu-system-ppc -machine 'g3beige' -cpu 'g4' -m 2047M -drive file=image.qcow2 -device e1000,netdev=net -netdev user,id=net,hostfwd=tcp::2222-:22 -kernel kernel -initrd initrd -nographic -append "root=LABEL=rootfs console=ttyPZ0"

>> =============================================================
>> OpenBIOS 1.1 [Mar 7 2023 22:21]
>> Configuration device id QEMU version 1 machine id 2
>> CPUs: 1
>> Memory: 2047M
>> UUID: 00000000-0000-0000-0000-000000000000
>> CPU type PowerPC,G4
milliseconds isn't unique.
Welcome to OpenBIOS v1.1 built on Mar 7 2023 22:21
>> [ppc] Kernel already loaded (0x01000000 + 0x015e876c) (initrd 0x026e9000 + 0x016eb316)
>> [ppc] Kernel command line: root=LABEL=rootfs console=ttyPZ0
>> switching to new context:
OF stdout device is: /pci@80000000/mac-io@10/escc@13000/ch-a@13020
Preparing to boot Linux version 6.6.15-powerpc-smp (debian-kernel@lists.debian.org) (gcc-13 (Debian 13.2.0-13) 13.2.0, GNU ld (GNU Binutils for Debian) 2.42) #1 SMP Debian 6.6.15-2 (2024-02-04)
Detected machine type: 00000400
command line:  console=ttyS0,9600 console=tty0
memory layout at init:
  memory_limit : 00000000 (16 MB aligned)
  alloc_bottom : 03dd5000
  alloc_top    : 30000000
  alloc_top_hi : 7ff00000
  rmo_top      : 30000000
  ram_top      : 7ff00000
found display   : /pci@80000000/QEMU,VGA@1, opening... done
copying OF device tree...
...
[  OK  ] Finished systemd-user-sessions.service - Permit User Sessions.
[  OK  ] Started getty@tty1.service - Getty on tty1.
[  OK  ] Started getty@tty2.service - Getty on tty2.
[  OK  ] Started getty@tty3.service - Getty on tty3.
[  OK  ] Started getty@tty4.service - Getty on tty4.
[  OK  ] Started getty@tty5.service - Getty on tty5.
[  OK  ] Started getty@tty6.service - Getty on tty6.
[  OK  ] Started serial-getty@ttyPZ0.service - Serial Getty on ttyPZ0.
[  OK  ] Reached target getty.target - Login Prompts.
[  OK  ] Started ssh.service - OpenBSD Secure Shell server.
[  OK  ] Finished e2scrub_reap.service - Re…line ext4 Metadata Check Snapshots.
[  OK  ] Reached target multi-user.target - Multi-User System.
[  OK  ] Reached target graphical.target - Graphical Interface.
         Starting systemd-update-utmp-runle…- Record Runlevel Change in UTMP...

Debian GNU/Linux trixie/sid debian ttyPZ0

debian login:

I used ssh to login with the provided credentials, updated the system, and installed basic packages for building fio: 

vincent@localhost:~$ ssh -p 2222 root@localhost
The authenticity of host '[localhost]:2222 ([127.0.0.1]:2222)' can't be established.
ECDSA key fingerprint is SHA256:fxlfI2OlhQlvzhG06S5dhQmhdgEg3i2/Ex6CEBxcJ3E.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:2222' (ECDSA) to the list of known hosts.
root@localhost's password:
Linux debian 6.6.15-powerpc-smp #1 SMP Debian 6.6.15-2 (2024-02-04) ppc

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
root@debian:~# apt update && apt upgrade -y && apt install vim git build-essential pkg-config zlib1g-dev -y
Get:1 http://deb.debian.org/debian-ports unstable InRelease [80.4 kB]
Get:2 http://deb.debian.org/debian-ports unreleased InRelease [54.1 kB]
...
Setting up libheif1:powerpc (1.17.6-1) ...
Setting up libgd3:powerpc (2.3.3-9+b2) ...
Setting up libc-devtools (2.37-15) ...
Setting up libheif-plugin-libde265:powerpc (1.17.6-1) ...
Setting up libheif-plugin-x265:powerpc (1.17.6-1) ...
Setting up libheif-plugin-aomenc:powerpc (1.17.6-1) ...
Processing triggers for libc-bin (2.37-15) ...
Processing triggers for ca-certificates (20240203) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
root@debian:~# 

I also confirmed that this was a big endian platform:

root@debian:~# lscpu | grep Endian
Byte Order:          Big Endian

I built fio and ran a simple job that saved a verify state file: 

root@debian:~/fio# ./fio --name=test --ioengine=posixaio --nrfiles=2 --iodepth=2 --filesize=8k --verify_state_save=1 --verify=md5 --rw=write
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=2
fio-3.36-82-g2d0de
Starting 1 process
test: Laying out IO files (2 files / total 0MiB)

test: (groupid=0, jobs=1): err= 0: pid=2785: Fri Feb 16 19:11:43 2024
  read: IOPS=363, BW=1455KiB/s (1489kB/s)(16.0KiB/11msec)
    slat (usec): min=17, max=346, avg=173.85, stdev=137.08
    clat (usec): min=586, max=3044, avg=1686.29, stdev=1260.82
     lat (usec): min=603, max=3241, avg=1860.13, stdev=1266.17
    clat percentiles (usec):
     |  1.00th=[  586],  5.00th=[  586], 10.00th=[  586], 20.00th=[  586],
     | 30.00th=[  644], 40.00th=[  644], 50.00th=[  644], 60.00th=[ 2474],
     | 70.00th=[ 2474], 80.00th=[ 3032], 90.00th=[ 3032], 95.00th=[ 3032],
     | 99.00th=[ 3032], 99.50th=[ 3032], 99.90th=[ 3032], 99.95th=[ 3032],
     | 99.99th=[ 3032]
  write: IOPS=90, BW=364KiB/s (372kB/s)(16.0KiB/44msec); 0 zone resets
    slat (usec): min=595, max=7213, avg=3094.36, stdev=3063.25
    clat (usec): min=2050, max=24787, avg=11159.73, stdev=10580.38
     lat (usec): min=2646, max=25724, avg=14254.09, stdev=11073.71
    clat percentiles (usec):
     |  1.00th=[ 2057],  5.00th=[ 2057], 10.00th=[ 2057], 20.00th=[ 2057],
     | 30.00th=[ 3556], 40.00th=[ 3556], 50.00th=[ 3556], 60.00th=[14222],
     | 70.00th=[14222], 80.00th=[24773], 90.00th=[24773], 95.00th=[24773],
     | 99.00th=[24773], 99.50th=[24773], 99.90th=[24773], 99.95th=[24773],
     | 99.99th=[24773]
  lat (usec)   : 750=25.00%
  lat (msec)   : 4=50.00%, 20=12.50%, 50=12.50%
  cpu          : usr=53.45%, sys=10.34%, ctx=15, majf=0, minf=22
  IO depths    : 1=37.5%, 2=62.5%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=4,4,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
   READ: bw=1455KiB/s (1489kB/s), 1455KiB/s-1455KiB/s (1489kB/s-1489kB/s), io=16.0KiB (16.4kB), run=11-11msec
  WRITE: bw=364KiB/s (372kB/s), 364KiB/s-364KiB/s (372kB/s-372kB/s), io=16.0KiB (16.4kB), run=44-44msec

Disk stats (read/write):
  sda: ios=0/0, sectors=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

However, running a fio job that loads the verify state file results in an error:

root@debian:~/fio# ./fio --name=test --ioengine=posixaio --nrfiles=2 --iodepth=2 --filesize=8k --verify_state_load=1 --verify=md5 --rw=write --verify_only=1
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=2
fio-3.36-82-g2d0de
Starting 1 process
malloc(): invalid size (unsorted)
Aborted

Fio actually has a utility for displaying the contents of verify state files. Running this utility shows that the IO depth is read back as zero and the file numbers and offsets listed in the Completions section are nonsensical:

root@debian:~/fio# t/fio-verify-state local-test-0-verify.state
Version:        0x3
Size:           152
CRC:            0xb7c35db4
Thread:         0
Name:           test
Completions:    4
Depth:          0
Number IOs:     4
Index:          0
Completions:
        (file= 0) 0
        (file=10) 0
        (file=659889948385951025) 3470340378564627042
        (file= 0) 1226104998551617536

The patch below (already committed upstream) resolves the issue:

commit 38d01cc56366aa2fd3af42dbab522888b6359dec (HEAD -> master, origin/master, origin/HEAD)
Author: Vincent Fu <vincent.fu@samsung.com>
Date:   Fri Feb 16 01:33:21 2024 +0000

    verify: fix integer sizes in verify state file

    nofiles and depth are 32-bit integers. So we shouldn't use 64-bit
    conversion functions and casts. The current code actually works fine on
    little-endian platforms since the conversion is a noop but this is
    broken on big-endian platforms.

    Fixes: 94a6e1bb ("Fix verify state for multiple files")
    Signed-off-by: Vincent Fu <vincent.fu@samsung.com>

diff --git a/verify.c b/verify.c
index b438eed6..b2fede24 100644
--- a/verify.c
+++ b/verify.c
@@ -1619,8 +1619,8 @@ struct all_io_list *get_all_io_list(int save_mask, size_t *sz)
                comps = fill_file_completions(td, s, &index);

                s->no_comps = cpu_to_le64((uint64_t) comps);
-               s->depth = cpu_to_le64((uint64_t) td->o.iodepth);
-               s->nofiles = cpu_to_le64((uint64_t) td->o.nr_files);
+               s->depth = cpu_to_le32((uint32_t) td->o.iodepth);
+               s->nofiles = cpu_to_le32((uint32_t) td->o.nr_files);
                s->numberio = cpu_to_le64((uint64_t) td->io_issues[DDIR_WRITE]);
                s->index = cpu_to_le64((uint64_t) __td_index);
                if (td->random_state.use64) {

We can repeat the jobs that save and load the verify state files and we no longer see errors:

root@debian:~/fio# ./fio --name=test --ioengine=posixaio --nrfiles=2 --iodepth=2 --filesize=8k --verify_state_save=1 --verify=md5 --rw=write
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=2
fio-3.36-83-g38d01
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=2857: Fri Feb 16 19:15:29 2024
  read: IOPS=285, BW=1143KiB/s (1170kB/s)(16.0KiB/14msec)
    slat (usec): min=67, max=1924, avg=1013.22, stdev=980.50
    clat (usec): min=389, max=3131, avg=1332.62, stdev=1233.06
     lat (usec): min=787, max=3400, avg=2345.83, stdev=1156.43
    clat percentiles (usec):
     |  1.00th=[  392],  5.00th=[  392], 10.00th=[  392], 20.00th=[  392],
     | 30.00th=[  717], 40.00th=[  717], 50.00th=[  717], 60.00th=[ 1090],
     | 70.00th=[ 1090], 80.00th=[ 3130], 90.00th=[ 3130], 95.00th=[ 3130],
     | 99.00th=[ 3130], 99.50th=[ 3130], 99.90th=[ 3130], 99.95th=[ 3130],
     | 99.99th=[ 3130]
  write: IOPS=90, BW=364KiB/s (372kB/s)(16.0KiB/44msec); 0 zone resets
    slat (usec): min=271, max=7171, avg=2944.05, stdev=3280.50
    clat (usec): min=3828, max=13622, avg=6614.60, stdev=4706.06
     lat (usec): min=4099, max=20793, avg=9558.65, stdev=7646.94
    clat percentiles (usec):
     |  1.00th=[ 3818],  5.00th=[ 3818], 10.00th=[ 3818], 20.00th=[ 3818],
     | 30.00th=[ 3916], 40.00th=[ 3916], 50.00th=[ 3916], 60.00th=[ 5080],
     | 70.00th=[ 5080], 80.00th=[13566], 90.00th=[13566], 95.00th=[13566],
     | 99.00th=[13566], 99.50th=[13566], 99.90th=[13566], 99.95th=[13566],
     | 99.99th=[13566]
  lat (usec)   : 500=12.50%, 750=12.50%
  lat (msec)   : 2=12.50%, 4=37.50%, 10=12.50%, 20=12.50%
  cpu          : usr=59.02%, sys=18.03%, ctx=16, majf=0, minf=21
  IO depths    : 1=50.0%, 2=50.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=4,4,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=2

Run status group 0 (all jobs):
   READ: bw=1143KiB/s (1170kB/s), 1143KiB/s-1143KiB/s (1170kB/s-1170kB/s), io=16.0KiB (16.4kB), run=14-14msec
  WRITE: bw=364KiB/s (372kB/s), 364KiB/s-364KiB/s (372kB/s-372kB/s), io=16.0KiB (16.4kB), run=44-44msec

Disk stats (read/write):
  sda: ios=0/0, sectors=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
root@debian:~/fio# ./fio --name=test --ioengine=posixaio --nrfiles=2 --iodepth=2 --filesize=8k --verify_state_load=1 --verify=md5 --rw=write --verify_only=1
test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=posixaio, iodepth=2
fio-3.36-83-g38d01
Starting 1 process

test: (groupid=0, jobs=1): err= 0: pid=2862: Fri Feb 16 19:15:34 2024
  read: IOPS=173, BW=696KiB/s (712kB/s)(16.0KiB/23msec)
    slat (usec): min=467, max=6971, avg=2582.31, stdev=3009.09
    clat (usec): min=26, max=4634, avg=1711.04, stdev=2012.53
     lat (usec): min=494, max=11606, avg=4293.35, stdev=4999.10
    clat percentiles (usec):
     |  1.00th=[   27],  5.00th=[   27], 10.00th=[   27], 20.00th=[   27],
     | 30.00th=[ 1074], 40.00th=[ 1074], 50.00th=[ 1074], 60.00th=[ 1106],
     | 70.00th=[ 1106], 80.00th=[ 4621], 90.00th=[ 4621], 95.00th=[ 4621],
     | 99.00th=[ 4621], 99.50th=[ 4621], 99.90th=[ 4621], 99.95th=[ 4621],
     | 99.99th=[ 4621]
  lat (usec)   : 50=12.50%
  lat (msec)   : 2=25.00%, 10=12.50%, >=2000=50.00%
  cpu          : usr=80.95%, sys=0.00%, ctx=17, majf=0, minf=19
  IO depths    : 1=75.0%, 2=25.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=4,4,0,0 short=0,0,0,0 dropped=0,0,0,0

Run status group 0 (all jobs):
   READ: bw=696KiB/s (712kB/s), 696KiB/s-696KiB/s (712kB/s-712kB/s), io=16.0KiB (16.4kB), run=23-23msec

Disk stats (read/write):
  sda: ios=0/0, sectors=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%

The contents of the verify state file are no longer corrupted:

root@debian:~/fio# t/fio-verify-state local-test-0-verify.state
Version:        0x3
Size:           216
CRC:            0x58b57c6f
Thread:         0
Name:           test
Completions:    4
Depth:          2
Number IOs:     4
Index:          0
Completions:
        (file= 1) 0
        (file= 1) 4096
        (file= 0) 0
        (file= 0) 4096

Giovanni Mascellani's DQIB project was a great help in verifying this fix. From time to time users report fio issues that may be due to endian-ness. The DQIB images will be helpful in resolving those as well.

Notes

  • I first tried the DQIB s390x image but it failed to boot with the supplied QEMU invocation. I later found a pull request that resolves the boot failure: https://gitlab.com/giomasce/dqib/-/merge_requests/2
  • The next time I need to work on a big endian platform I will use s390x as PowerPC Gen3 is dog slow and from 1997.
  • I also tried other PowerPC platforms but found they were little endian.
  • One use case for the verify state file is powercut testing. If the write phase of a verify workload is interrupted, fio will record the final set of completions to each file in the verify state file. When fio is invoked again to verify the written data, the verify state file is used to decide when to stop the verification phase.
  • struct thread_io_list is defined in verify-state.h; the quoted code preparing the verify state file is in verify.c:get_all_io_list(); and the code reading back the verify state file is in verify.c:verify_assign_state().