Skip to content

Latest commit

 

History

History
939 lines (685 loc) · 43.5 KB

README.md

File metadata and controls

939 lines (685 loc) · 43.5 KB

Emulate PinePhone with Unicorn Emulator

Read the articles...

We're porting a new operating system (Apache NuttX RTOS) to Pine64 PinePhone. And I wondered...

To make PinePhone testing easier...

Can we emulate Arm64 PinePhone with Unicorn Emulator?

Let's find out! We'll call the Unicorn Emulator in Rust (instead of C).

(Because I'm too old to write meticulous C... But I'm OK to get nagged by Rust Compiler if I miss something!)

We begin by emulating simple Arm64 Machine Code...

Emulate Arm64 Machine Code

Emulate Arm64 Machine Code

Suppose we wish to emulate some Arm64 Machine Code...

0xab, 0x05, 0x00, 0xb8, // str w11, [x13], #0
0xaf, 0x05, 0x40, 0x38, // ldrb w15, [x13], #0

Here's our Rust Program that calls Unicorn Emulator to emulate the Arm64 Machine Code...

use unicorn_engine::{Unicorn, RegisterARM64};
use unicorn_engine::unicorn_const::{Arch, Mode, Permission};
fn main() {
// Arm64 Code
let arm64_code: Vec<u8> = vec![
0xab, 0x05, 0x00, 0xb8, // str w11, [x13], #0
0xaf, 0x05, 0x40, 0x38, // ldrb w15, [x13], #0
];
// Initialize emulator in ARM64 mode
let mut unicorn = Unicorn::new(
Arch::ARM64,
Mode::LITTLE_ENDIAN
).expect("failed to initialize Unicorn instance");
let emu = &mut unicorn;
// memory address where emulation starts
const ADDRESS: u64 = 0x10000;
// map 2MB memory for this emulation
emu.mem_map(
ADDRESS,
2 * 1024 * 1024,
Permission::ALL
).expect("failed to map code page");
// write machine code to be emulated to memory
emu.mem_write(
ADDRESS,
&arm64_code
).expect("failed to write instructions");
// Register Values
const X11: u64 = 0x12345678; // X11 register
const X13: u64 = 0x10000 + 0x8; // X13 register
const X15: u64 = 0x33; // X15 register
// initialize machine registers
emu.reg_write(RegisterARM64::X11, X11)
.expect("failed to set X11");
emu.reg_write(RegisterARM64::X13, X13)
.expect("failed to set X13");
emu.reg_write(RegisterARM64::X15, X15)
.expect("failed to set X15");
let _ = emu.emu_start(
ADDRESS,
ADDRESS + arm64_code.len() as u64,
0, // Previously: 10 * SECOND_SCALE,
0 // Previously: 1000
);
assert_eq!(emu.reg_read(RegisterARM64::X15), Ok(0x78));
}

We add unicorn-engine to Cargo.toml...

[dependencies]
unicorn-engine = "2.0.0"

And we run our Rust Program...

→ cargo run --verbose
  Fresh cc v1.0.79
  Fresh cmake v0.1.49
  Fresh pkg-config v0.3.26
  Fresh bitflags v1.3.2
  Fresh libc v0.2.139
  Fresh unicorn-engine v2.0.1
  Fresh pinephone-emulator v0.1.0
Finished dev [unoptimized + debuginfo] target(s) in 0.08s
  Running `target/debug/pinephone-emulator`

Our Rust Program works OK for emulating Arm64 Memory and Arm64 Registers.

Let's talk about Arm64 Memory-Mapped Input / Output...

Memory Access Hook for Arm64 Emulation

Memory Access Hook for Arm64 Emulation

How will we emulate Arm64 Memory-Mapped Input / Output?

Unicorn Emulator lets us attach hooks to Emulate Memory Access.

Here's a Hook Function for Memory Access...

// Hook Function for Memory Access.
// Called once for every Arm64 Memory Access.
fn hook_memory(
_: &mut Unicorn<()>, // Emulator
mem_type: MemType, // Read or Write Access
address: u64, // Accessed Address
size: usize, // Number of bytes accessed
value: i64 // Read / Write Value
) -> bool {
// TODO: Simulate Memory-Mapped Input/Output (UART Controller)
println!("hook_memory: mem_type={:?}, address={:#x}, size={:?}, value={:#x}", mem_type, address, size, value);
true
}

Our Hook Function prints all Read / Write Access to Emulated Arm64 Memory.

(Return value is unused)

This is how we attach the Hook Function to the Unicorn Emulator...

// Add Hook for Arm64 Memory Access
let _ = emu.add_mem_hook(
HookType::MEM_ALL,
0,
u64::MAX,
hook_memory
).expect("failed to add memory hook");
// Emulate machine code in infinite time (last param = 0),
// or when all code has completed
let _ = emu.emu_start(
ADDRESS,
ADDRESS + arm64_code.len() as u64,
0, // Previously: 10 * SECOND_SCALE,
0 // Previously: 1000
);

When we run our Rust Program, we see the Read and Write Memory Accesses made by our Emulated Arm64 Code...

hook_memory: 
  mem_type=WRITE, 
  address=0x10008, 
  size=4, 
  value=0x12345678

hook_memory: 
  mem_type=READ, 
  address=0x10008, 
  size=1, 
  value=0x0

This Memory Access Hook Function will be helpful when we emulate Memory-Mapped Input/Output on PinePhone.

(Like for the Allwinner A64 UART Controller)

Unicorn Emulator allows Code Execution Hooks too...

Code Execution Hook for Arm64 Emulation

Code Execution Hook for Arm64 Emulation

Can we intercept every Arm64 Instruction that will be emulated?

Yep we can call Unicorn Emulator to add a Code Execution Hook.

Here's a sample Hook Function that will be called for every Arm64 Instruction...

// Hook Function for Code Emulation.
// Called once for each Arm64 Instruction.
fn hook_code(
_: &mut Unicorn<()>, // Emulator
address: u64, // Instruction Address
size: u32 // Instruction Size
) {
// TODO: Handle special Arm64 Instructions
println!("hook_code: address={:#x}, size={:?}", address, size);
}

And this is how we call Unicorn Emulator to add the above Hook Function...

// Add Hook for emulating each Arm64 Instruction
let _ = emu.add_code_hook(
ADDRESS, // Begin Address
ADDRESS + arm64_code.len() as u64, // End Address
hook_code // Hook Function for Code Emulation
).expect("failed to add code hook");

When we run our Rust Program, we see the Address of every Arm64 Instruction emulated (and its size)...

hook_code:
  address=0x10000,
  size=4

hook_code:
  address=0x10004,
  size=4

We might use this to emulate special Arm64 Instructions.

If we don't need to intercept every single instruction, try the Block Execution Hook...

Block Execution Hook for Arm64 Emulation

Is there something that works like a Code Execution Hook...

But doesn't stop at every single Arm64 Instruction?

Yep Unicorn Emulator supports Block Execution Hooks.

This Hook Function will be called once when executing a Block of Arm64 Instructions...

// Hook Function for Block Emulation.
// Called once for each Basic Block of Arm64 Instructions.
fn hook_block(
_: &mut Unicorn<()>, // Emulator
address: u64, // Block Address
size: u32 // Block Size
) {
// TODO: Trace the flow of emulated code
println!("hook_block: address={:#x}, size={:?}", address, size);
}

This is how we add the Block Execution Hook...

// Add Hook for emulating each Basic Block of Arm64 Instructions
let _ = emu.add_block_hook(hook_block)
.expect("failed to add block hook");

When we run the Rust Program, we see that that the Block Size is 8...

hook_block:
  address=0x10000,
  size=8

Which means that Unicorn Emulator calls our Hook Function only once for the entire Block of 2 Arm64 Instructions.

This Block Execution Hook will be super helpful for monitoring the Execution Flow of our emulated code.

Let's talk about the Block...

What is a Block of Arm64 Instructions?

What exactly is a Block of Arm64 Instructions?

When we run this code from Apache NuttX RTOS (that handles UART Output)...

SECTION_FUNC(text, up_lowputc)
  ldr   x15, =UART0_BASE_ADDRESS
  400801f0:	580000cf 	ldr	x15, 40080208 <up_lowputc+0x18>
nuttx/arch/arm64/src/chip/a64_lowputc.S:89
  early_uart_ready x15, w2
  400801f4:	794029e2 	ldrh	w2, [x15, #20]
  400801f8:	721b005f 	tst	w2, #0x20
  400801fc:	54ffffc0 	b.eq	400801f4 <up_lowputc+0x4>  // b.none
nuttx/arch/arm64/src/chip/a64_lowputc.S:90
  early_uart_transmit x15, w0
  40080200:	390001e0 	strb	w0, [x15]
nuttx/arch/arm64/src/chip/a64_lowputc.S:91
  ret
  40080204:	d65f03c0 	ret

(Arm64 Disassembly)

(Source Code)

We observe that Unicorm Emulator treats 400801f0 to 400801fc as a Block of Arm64 Instructins...

hook_block:  address=0x400801f0, size=16
hook_code:   address=0x400801f0, size=4
hook_code:   address=0x400801f4, size=4
hook_code:   address=0x400801f4, size=4
hook_code:   address=0x400801f8, size=4
hook_code:   address=0x400801fc, size=4

hook_block:  address=0x400801f4, size=12
hook_code:   address=0x400801f4, size=4
hook_code:   address=0x400801f8, size=4
hook_code:   address=0x400801fc, size=4

hook_block:  address=0x400801f4, size=12
hook_code:   address=0x400801f4, size=4
hook_code:   address=0x400801f8, size=4
hook_code:   address=0x400801fc, size=4

(Source)

The Block ends at 400801fc because there's an Arm64 Branch Instruction b.eq.

From this we deduce that Unicorn Emulator treats a sequence of Arm64 Instructions as a Block, until it sees a Branch Instruction. (Including function calls)

Unmapped Memory in Unicorn Emulator

What happens when Unicorn Emulator tries to access memory that isn't mapped?

Unicorn Emulator will call our Memory Access Hook with mem_type set to READ_UNMAPPED...

hook_memory:
  address=0x01c28014,
  size=2,
  mem_type=READ_UNMAPPED,
  value=0x0

(Source)

The log above says that address 0x01c2 8014 is unmapped.

This is how we map the memory...

// Map 16 MB at 0x0100 0000 for Memory-Mapped I/O by Allwinner A64 Peripherals
// https://github.com/apache/nuttx/blob/master/arch/arm64/src/a64/hardware/a64_memorymap.h#L33-L51
emu.mem_map(
0x0100_0000, // Address
16 * 1024 * 1024, // Size
Permission::READ | Permission::WRITE // Read and Write Access
).expect("failed to map memory mapped I/O");

(See the NuttX Memory Map)

Can we map Memory Regions during emulation?

Yep we may use a Memory Access Hook to map memory regions on the fly. (See this)

Run Apache NuttX RTOS in Unicorn Emulator

Run Apache NuttX RTOS in Unicorn Emulator

Let's run Apache NuttX RTOS in Unicorn Emulator!

We have compiled Apache NuttX RTOS for PinePhone into an Arm64 Binary Image nuttx.bin.

This is how we load the NuttX Binary Image into Unicorn...

// Arm64 Memory Address where emulation starts
const ADDRESS: u64 = 0x4008_0000;
// Arm64 Machine Code for the above address
let arm64_code = include_bytes!("../nuttx/nuttx.bin");
// Initialize emulator in Arm64 mode
let mut unicorn = Unicorn::new(
Arch::ARM64,
Mode::LITTLE_ENDIAN
).expect("failed to initialize Unicorn instance");
let emu = &mut unicorn;
// Map 128 MB Executable Memory at 0x4000 0000 for Arm64 Machine Code
// https://github.com/apache/nuttx/blob/master/arch/arm64/include/a64/chip.h#L44-L52
emu.mem_map(
0x4000_0000, // Address
128 * 1024 * 1024, // Size
Permission::ALL // Read, Write and Execute Access
).expect("failed to map code page");
// Map 512 MB Read/Write Memory at 0x0000 0000 for
// Memory-Mapped I/O by Allwinner A64 Peripherals
// https://github.com/apache/nuttx/blob/master/arch/arm64/include/a64/chip.h#L44-L52
emu.mem_map(
0x0000_0000, // Address
512 * 1024 * 1024, // Size
Permission::READ | Permission::WRITE // Read and Write Access
).expect("failed to map memory mapped I/O");
// Write Arm64 Machine Code to emulated Executable Memory
emu.mem_write(
ADDRESS,
arm64_code
).expect("failed to write instructions");

In our Rust Program above, we mapped 2 Memory Regions for NuttX...

  • Map 128 MB Executable Memory at 0x4000 0000 for Arm64 Machine Code

  • Map 512 MB Read/Write Memory at 0x0000 0000 for Memory-Mapped I/O by Allwinner A64 Peripherals

This is based on the NuttX Memory Map for PinePhone.

When we run this, Unicorn Emulator loops forever. Let's find out why...

Unicorn Emulator Waits Forever for UART Controller Ready

Emulating the Allwinner A64 UART Controller

Here's the output when we run NuttX RTOS in Unicorn Emulator...

hook_memory: address=0x01c28014, size=2, mem_type=READ, value=0x0
hook_code:   address=0x400801f8, size=4
hook_code:   address=0x400801fc, size=4
hook_block:  address=0x400801f4, size=12
hook_code:   address=0x400801f4, size=4

hook_memory: address=0x01c28014, size=2, mem_type=READ, value=0x0
hook_code:   address=0x400801f8, size=4
hook_code:   address=0x400801fc, size=4
hook_block:  address=0x400801f4, size=12
hook_code:   address=0x400801f4, size=4

hook_memory: address=0x01c28014, size=2, mem_type=READ, value=0x0
hook_code:   address=0x400801f8, size=4
hook_code:   address=0x400801fc, size=4
hook_block:  address=0x400801f4, size=12
hook_code:   address=0x400801f4, size=4
...

(Source)

The above log shows that Unicorn Emulator loops forever at address 0x4008 01f4, while reading the data from address 0x01c2 8014.

Let's check the NuttX Arm64 Code at address 0x4008 01f4...

SECTION_FUNC(text, up_lowputc)
  ldr   x15, =UART0_BASE_ADDRESS
  400801f0:	580000cf 	ldr	x15, 40080208 <up_lowputc+0x18>
nuttx/arch/arm64/src/chip/a64_lowputc.S:89
  early_uart_ready x15, w2
  400801f4:	794029e2 	ldrh	w2, [x15, #20]
  400801f8:	721b005f 	tst	w2, #0x20
  400801fc:	54ffffc0 	b.eq	400801f4 <up_lowputc+0x4>  // b.none
nuttx/arch/arm64/src/chip/a64_lowputc.S:90
  early_uart_transmit x15, w0
  40080200:	390001e0 	strb	w0, [x15]
nuttx/arch/arm64/src/chip/a64_lowputc.S:91
  ret
  40080204:	d65f03c0 	ret

(Arm64 Disassembly)

Which comes from this NuttX Source Code...

/* Wait for A64 UART to be ready to transmit
 * xb: Register that contains the UART Base Address
 * wt: Scratch register number
 */
.macro early_uart_ready xb, wt
1:
  ldrh  \wt, [\xb, #0x14]      /* UART_LSR (Line Status Register) */
  tst   \wt, #0x20             /* Check THRE (TX Holding Register Empty) */
  b.eq  1b                     /* Wait for the UART to be ready (THRE=1) */
.endm

(Source Code)

This code waits for the UART Controller to be ready (before printing UART Output), by checking the value at 0x01c2 8014. The code is explained here...

What is 0x01c2 8014?

According to the Allwinner A64 Doc...

0x01c2 8014 is the UART Line Status Register (UART_LSR) at Offset 0x14.

Bit 5 needs to be set to 1 to indicate that the UART Transmit FIFO is ready.

We emulate the UART Ready Bit like so...

// Allwinner A64 UART Line Status Register (UART_LSR) at Offset 0x14.
// To indicate that the UART Transmit FIFO is ready:
// Set Bit 5 to 1.
// https://lupyuen.github.io/articles/serial#wait-to-transmit
emu.mem_write(
0x01c2_8014, // UART Register Address
&[0b10_0000] // UART Register Value
).expect("failed to set UART_LSR");

And Unicorn Emulator stops looping! It continues execution to memset() (to init the BSS Section to 0)...

hook_block:  address=0x40089328, size=8
hook_memory: address=0x400b6a52, size=1, mem_type=WRITE, value=0x0
hook_block:  address=0x40089328, size=8
hook_memory: address=0x400b6a53, size=1, mem_type=WRITE, value=0x0
hook_block:  address=0x40089328, size=8
hook_memory: address=0x400b6a54, size=1, mem_type=WRITE, value=0x0
...

(Source)

But we don't see any UART Output. Let's print the UART Output...

Emulate UART Output in Unicorn Emulator

Emulating UART Output in Unicorn Emulator

How do we print the UART Output?

According to the Allwinner A64 Doc...

NuttX RTOS will write the UART Output to the UART Transmit Holding Register (THR) at 0x01c2 8000.

In our Memory Access Hook, let's intercept all writes to 0x01c2 8000 and dump the characters written to UART Output...

// Hook Function for Memory Access.
// Called once for every Arm64 Memory Access.
fn hook_memory(
_: &mut Unicorn<()>, // Emulator
mem_type: MemType, // Read or Write Access
address: u64, // Accessed Address
size: usize, // Number of bytes accessed
value: i64 // Read / Write Value
) -> bool {
// Ignore RAM access, we only intercept Memory-Mapped Input / Output
if address >= 0x4000_0000 { return true; }
println!("hook_memory: address={:#010x}, size={:?}, mem_type={:?}, value={:#x}", address, size, mem_type, value);
// If writing to UART Transmit Holding Register (THR):
// Print the output
// https://lupyuen.github.io/articles/serial#transmit-uart
if address == 0x01c2_8000 {
println!("uart output: {:?}", value as u8 as char);
}
// Always return true, value is unused by caller
true
}

When we run this, we see a long chain of UART Output...

→ cargo run | grep uart
uart output: '-'
uart output: ' '
uart output: 'R'
uart output: 'e'
uart output: 'a'
uart output: 'd'
uart output: 'y'
...

(Source)

Which reads as...

- Ready to Boot CPU
- Boot from EL2
- Boot from EL1
- Boot to C runtime for OS Initialize

(Similar to this)

Yep NuttX RTOS is booting on Unicorn Emulator! But Unicorn Emulator halts while booting NuttX...

Unicorn Emulator Halts in NuttX MMU

Unicorn Emulator halts with an Arm64 Exception at address 4008 0EF8...

hook_block:  address=0x40080cec, size=16
hook_code:   address=0x40080cec, size=4
hook_memory: address=0x400c3f90, size=8, mem_type=READ, value=0x0
hook_memory: address=0x400c3f98, size=8, mem_type=READ, value=0x0
hook_code:   address=0x40080cf0, size=4
hook_memory: address=0x400c3fa0, size=8, mem_type=READ, value=0x0
hook_code:   address=0x40080cf4, size=4
hook_memory: address=0x400c3f80, size=8, mem_type=READ, value=0x0
hook_memory: address=0x400c3f88, size=8, mem_type=READ, value=0x0
hook_code:   address=0x40080cf8, size=4
hook_block:  address=0x40080eb0, size=12
hook_code:   address=0x40080eb0, size=4
hook_code:   address=0x40080eb4, size=4
hook_code:   address=0x40080eb8, size=4
hook_block:  address=0x40080ebc, size=16
hook_code:   address=0x40080ebc, size=4
hook_code:   address=0x40080ec0, size=4
hook_code:   address=0x40080ec4, size=4
hook_code:   address=0x40080ec8, size=4
hook_block:  address=0x40080ecc, size=16
hook_code:   address=0x40080ecc, size=4
hook_code:   address=0x40080ed0, size=4
hook_code:   address=0x40080ed4, size=4
hook_code:   address=0x40080ed8, size=4
hook_block:  address=0x40080edc, size=12
hook_code:   address=0x40080edc, size=4
hook_code:   address=0x40080ee0, size=4
hook_code:   address=0x40080ee4, size=4
hook_block:  address=0x40080ee8, size=4
hook_code:   address=0x40080ee8, size=4
hook_block:  address=0x40080eec, size=16
hook_code:   address=0x40080eec, size=4
hook_code:   address=0x40080ef0, size=4
hook_code:   address=0x40080ef4, size=4
hook_code:   address=0x40080ef8, size=4
err=Err(EXCEPTION)

(See the Complete Log)

Unicorn Emulator halts at the NuttX MMU (EL1) code at 0x4008 0ef8...

nuttx/arch/arm64/src/common/arm64_mmu.c:544
  write_sysreg((value | SCTLR_M_BIT | SCTLR_C_BIT), sctlr_el1);
    40080ef0:	d28000a1 	mov	x1, #0x5                   	// #5
    40080ef4:	aa010000 	orr	x0, x0, x1
    40080ef8:	d5181000 	msr	sctlr_el1, x0

Why did MSR fail with an Exception?

Here's the context...

enable_mmu_el1():
nuttx/arch/arm64/src/common/arm64_mmu.c:533
  write_sysreg(MEMORY_ATTRIBUTES, mair_el1);
    40080ebc:	d2808000 	mov	x0, #0x400                 	// #1024
    40080ec0:	f2a88180 	movk	x0, #0x440c, lsl #16
    40080ec4:	f2c01fe0 	movk	x0, #0xff, lsl #32
    40080ec8:	d518a200 	msr	mair_el1, x0
nuttx/arch/arm64/src/common/arm64_mmu.c:534
  write_sysreg(get_tcr(1), tcr_el1);
    40080ecc:	d286a380 	mov	x0, #0x351c                	// #13596
    40080ed0:	f2a01000 	movk	x0, #0x80, lsl #16
    40080ed4:	f2c00020 	movk	x0, #0x1, lsl #32
    40080ed8:	d5182040 	msr	tcr_el1, x0
nuttx/arch/arm64/src/common/arm64_mmu.c:535
  write_sysreg(((uint64_t)base_xlat_table), ttbr0_el1);
    40080edc:	d00001a0 	adrp	x0, 400b6000 <g_uart1port>
    40080ee0:	91200000 	add	x0, x0, #0x800
    40080ee4:	d5182000 	msr	ttbr0_el1, x0
arm64_isb():
nuttx/arch/arm64/src/common/barriers.h:58
  __asm__ volatile ("isb" : : : "memory");
    40080ee8:	d5033fdf 	isb
enable_mmu_el1():
nuttx/arch/arm64/src/common/arm64_mmu.c:543
  value = read_sysreg(sctlr_el1);
    40080eec:	d5381000 	mrs	x0, sctlr_el1
nuttx/arch/arm64/src/common/arm64_mmu.c:544
  write_sysreg((value | SCTLR_M_BIT | SCTLR_C_BIT), sctlr_el1);
    40080ef0:	d28000a1 	mov	x1, #0x5                   	// #5
    40080ef4:	aa010000 	orr	x0, x0, x1
    40080ef8:	d5181000 	msr	sctlr_el1, x0
arm64_isb():
nuttx/arch/arm64/src/common/barriers.h:58
    40080efc:	d5033fdf 	isb

Which comes from this NuttX Source Code: arm64_mmu.c

// Enable the MMU and data cache:
// Read from System Control Register EL1
value = read_sysreg(sctlr_el1);

// Write to System Control Register EL1
write_sysreg(  // Write to System Register...
  value | SCTLR_M_BIT | SCTLR_C_BIT,  // Enable Address Translation and Caching
  sctlr_el1    // System Control Register EL1
);

Let's dump the Arm64 Exception...

Dump the Arm64 Exception

To find out the cause of the Arm64 Exception, let's dump the Exception Syndrome Register (ESR).

But this won't work...

println!("err={:?}", err);
println!("CP_REG={:?}", emu.reg_read(RegisterARM64::CP_REG));
println!("ESR_EL0={:?}", emu.reg_read(RegisterARM64::ESR_EL0));
println!("ESR_EL1={:?}", emu.reg_read(RegisterARM64::ESR_EL1));
println!("ESR_EL2={:?}", emu.reg_read(RegisterARM64::ESR_EL2));
println!("ESR_EL3={:?}", emu.reg_read(RegisterARM64::ESR_EL3));

Because ESR_EL is no longer supported and CP_REG can't be read in Rust...

err=Err(EXCEPTION)
CP_REG=Err(ARG)
ESR_EL0=Ok(0)
ESR_EL1=Ok(0)
ESR_EL2=Ok(0)
ESR_EL3=Ok(0)

(See the Complete Log)

CP_REG can't be read in Rust because Unicorn needs a pointer to uc_arm64_cp_reg...

static uc_err reg_read(CPUARMState *env, unsigned int regid, void *value) {
  ...
  case UC_ARM64_REG_CP_REG:
      ret = read_cp_reg(env, (uc_arm64_cp_reg *)value);
      break;

(Source)

Which isn't supported by the Rust Bindings.

(Works in Python though)

So instead we set a breakpoint at arm64_reg_read() (pic below) in...

.cargo/registry/src/github.com-1ecc6299db9ec823/unicorn-engine-2.0.1/qemu/target/arm/unicorn_aarch64.c

(arm64_reg_read() calls reg_read() in unicorn_aarch64.c)

Which shows the Exception as...

env.exception = {
  syndrome: 0x8600 003f, 
  fsr: 5, 
  vaddress: 0x400c 3fff,
  target_el: 1
}

Let's study the Arm64 Exception...

Debug the Arm64 Exception

Arm64 MMU Exception

Earlier we saw this Arm64 Exception in Unicorn Emulator...

env.exception = {
  syndrome: 0x8600 003f, 
  fsr: 5, 
  vaddress: 0x400c 3fff,
  target_el: 1
}

TODO: What is address 0x400c 3fff?

What is Syndrome 0x8600 003f?

Bits 26-31 of Syndrome = 0b100001, which means...

0b100001: Instruction Abort taken without a change in Exception level.

Used for MMU faults generated by instruction accesses and synchronous External aborts, including synchronous parity or ECC errors. Not used for debug-related exceptions.

(Source)

What is FSR 5?

FSR 5 means...

0b00101: Translation Fault (in) Section

(Source)

Why the MMU Fault?

Unicorn Emulator triggers the exception when NuttX writes to SCTLR_EL1...

  /* Enable the MMU and data cache */
  value = read_sysreg(sctlr_el1);
  write_sysreg((value | SCTLR_M_BIT | SCTLR_C_BIT), sctlr_el1);

(Source)

The above code sets these flags in SCTLR_EL1 (System Control Register EL1)...

  • SCTLR_M_BIT (Bit 0): Enable Address Translation for EL0 and EL1 Stage 1

  • SCTLR_C_BIT (Bit 2): Enable Caching for EL0 and EL1 Stage 1

(More about SCTLR_EL1)

TODO: Why did the Address Translation (or Caching) fail?

TODO: Should we skip the MMU Update to SCTLR_EL1? Since we don't use MMU?

Debug the Unicorn Emulator

To troubleshoot the Arm64 MMU Exception...

Can we use a debugger to step through Unicorn Emulator?

Yes but it gets messy.

To trace the exception in the debugger. Look for...

$HOME/.cargo/registry/src/github.com-1ecc6299db9ec823/unicorn-engine-2.0.1/qemu/target/arm/translate-a64.c

Set a breakpoint in aarch64_tr_translate_insn()

  • Which calls disas_b_exc_sys()

  • Which calls disas_system()

  • Which calls handle_sys() to handle system instructions

To inspect the Emulator Settings, set a breakpoint at cpu_aarch64_init() in...

$HOME/.cargo/registry/src/github.com-1ecc6299db9ec823/unicorn-engine-2.0.1/qemu/target/arm/cpu64.c

TODO: Check that the CPU Setting is correct for PinePhone. (CPU Model should be Cortex-A53)

Map Address to Function with ELF File

Read the article...

Our Block Execution Hook now prints the Function Name and the Filename...

hook_block:  
  address=0x40080eb0, 
  size=12, 
  setup_page_tables, 
  arch/arm64/src/common/arm64_mmu.c:516:25

hook_block:  
  address=0x40080eec, 
  size=16, 
  enable_mmu_el1, 
  arch/arm64/src/common/arm64_mmu.c:543:11

err=Err(EXCEPTION)

(Source)

Our Hook Function looks up the Address in the DWARF Debug Symbols of the NuttX ELF File.

This is explained here...

Call Graph for Apache NuttX RTOS

To troubleshoot the Apache NuttX MMU Fault on Unicorn Emulator, we auto-generated this Call Graph...

(To see the NuttX Source Code: Right-click the Node and select "Open Link")

  flowchart TD
  START --> arm64_head
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L104" "arch/arm64/src/common/arm64_head.S " _blank
  a64_lowputc --> arm64_head
  click a64_lowputc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_lowputc.S#L58" "arch/arm64/src/a64/a64_lowputc.S " _blank
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L177" "arch/arm64/src/common/arm64_head.S " _blank
  a64_lowputc --> arm64_head
  click a64_lowputc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_lowputc.S#L87" "arch/arm64/src/a64/a64_lowputc.S " _blank
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  a64_lowputc --> arm64_head
  click a64_lowputc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_lowputc.S#L87" "arch/arm64/src/a64/a64_lowputc.S " _blank
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  a64_lowputc --> arm64_head
  click a64_lowputc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_lowputc.S#L87" "arch/arm64/src/a64/a64_lowputc.S " _blank
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  a64_lowputc --> arm64_head
  click a64_lowputc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_lowputc.S#L87" "arch/arm64/src/a64/a64_lowputc.S " _blank
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  a64_lowputc --> arm64_head
  click a64_lowputc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_lowputc.S#L87" "arch/arm64/src/a64/a64_lowputc.S " _blank
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  a64_lowputc --> arm64_head
  click a64_lowputc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_lowputc.S#L87" "arch/arm64/src/a64/a64_lowputc.S " _blank
  arm64_head --> a64_lowputc
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  arm64_head --> arm64_boot_el1_init
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  arm64_boot_el1_init --> arm64_isb
  click arm64_boot_el1_init href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_boot.c#L137" "arch/arm64/src/common/arm64_boot.c " _blank
  arm64_isb --> arm64_boot_el1_init
  click arm64_isb href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/barriers.h#L57" "arch/arm64/src/common/barriers.h " _blank
  arm64_boot_el1_init --> arm64_isb
  click arm64_boot_el1_init href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_boot.c#L145" "arch/arm64/src/common/arm64_boot.c " _blank
  arm64_isb --> arm64_boot_el1_init
  click arm64_isb href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/barriers.h#L57" "arch/arm64/src/common/barriers.h " _blank
  arm64_head --> arm64_boot_primary_c_routine
  click arm64_head href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_head.S#L298" "arch/arm64/src/common/arm64_head.S " _blank
  arm64_boot_primary_c_routine --> memset
  click arm64_boot_primary_c_routine href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_boot.c#L180" "arch/arm64/src/common/arm64_boot.c " _blank
  memset --> arm64_boot_primary_c_routine
  click memset href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/libs/libc/string/lib_memset.c#L168" "libs/libc/string/lib_memset.c " _blank
  arm64_boot_primary_c_routine --> arm64_chip_boot
  click arm64_boot_primary_c_routine href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_boot.c#L181" "arch/arm64/src/common/arm64_boot.c " _blank
  arm64_chip_boot --> arm64_mmu_init
  click arm64_chip_boot href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/a64/a64_boot.c#L81" "arch/arm64/src/a64/a64_boot.c " _blank
  arm64_mmu_init --> setup_page_tables
  click arm64_mmu_init href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L583" "arch/arm64/src/common/arm64_mmu.c " _blank
  setup_page_tables --> init_xlat_tables
  click setup_page_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L490" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> calculate_pte_index
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L417" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> init_xlat_tables
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L246" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> pte_desc_type
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  pte_desc_type --> new_prealloc_table
  click pte_desc_type href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L234" "arch/arm64/src/common/arm64_mmu.c " _blank
  new_prealloc_table --> init_xlat_tables
  click new_prealloc_table href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L378" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> calculate_pte_index
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L437" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> pte_desc_type
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L246" "arch/arm64/src/common/arm64_mmu.c " _blank
  pte_desc_type --> calculate_pte_index
  click pte_desc_type href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L234" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> init_xlat_tables
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L247" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  set_pte_block_desc --> init_xlat_tables
  click set_pte_block_desc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L293" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> calculate_pte_index
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L456" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> init_xlat_tables
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L246" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> pte_desc_type
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  pte_desc_type --> init_xlat_tables
  click pte_desc_type href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L234" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> calculate_pte_index
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L472" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> pte_desc_type
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L246" "arch/arm64/src/common/arm64_mmu.c " _blank
  pte_desc_type --> calculate_pte_index
  click pte_desc_type href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L234" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> init_xlat_tables
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L247" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> calculate_pte_index
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L456" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> pte_desc_type
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> calculate_pte_index
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L472" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> pte_desc_type
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L246" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> pte_desc_type
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  calculate_pte_index --> pte_desc_type
  click calculate_pte_index href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L246" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> set_pte_block_desc
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L447" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> setup_page_tables
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L456" "arch/arm64/src/common/arm64_mmu.c " _blank
  pte_desc_type --> new_prealloc_table
  click pte_desc_type href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L234" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> setup_page_tables
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L456" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> new_prealloc_table
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L472" "arch/arm64/src/common/arm64_mmu.c " _blank
  new_prealloc_table --> split_pte_block_desc
  click new_prealloc_table href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L378" "arch/arm64/src/common/arm64_mmu.c " _blank
  split_pte_block_desc --> set_pte_table_desc
  click split_pte_block_desc href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L401" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> setup_page_tables
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L456" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> setup_page_tables
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L456" "arch/arm64/src/common/arm64_mmu.c " _blank
  init_xlat_tables --> setup_page_tables
  click init_xlat_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L456" "arch/arm64/src/common/arm64_mmu.c " _blank
  setup_page_tables --> enable_mmu_el1
  click setup_page_tables href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L515" "arch/arm64/src/common/arm64_mmu.c " _blank
  enable_mmu_el1 --> arm64_isb
  click enable_mmu_el1 href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L532" "arch/arm64/src/common/arm64_mmu.c " _blank
  arm64_isb --> enable_mmu_el1
  click arm64_isb href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/barriers.h#L57" "arch/arm64/src/common/barriers.h " _blank
  enable_mmu_el1 --> ***_HALT_***
  click enable_mmu_el1 href "https://github.com/apache/nuttx/blob/0f20888a0ececc5dc7419d57a01ac508ac3ace5b/arch/arm64/src/common/arm64_mmu.c#L542" "arch/arm64/src/common/arm64_mmu.c " _blank
Loading

We generated the Call Graph with this command...

cargo run | grep call_graph | cut -c 12-

(cut command removes columns 1 to 11)

Which produces this Mermaid Flowchart...

→ cargo run | grep call_graph | cut -c 12- 
  flowchart TD
  arm64_boot_el1_init --> arm64_isb
  click arm64_boot_el1_init href "https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/arm64_boot.c#L137" "arch/arm64/src/common/arm64_boot.c "
  arm64_isb --> arm64_boot_el1_init
  click arm64_isb href "https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/barriers.h#L57" "arch/arm64/src/common/barriers.h "
  arm64_boot_el1_init --> arm64_isb
  click arm64_boot_el1_init href "https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/arm64_boot.c#L145" "arch/arm64/src/common/arm64_boot.c "
  arm64_isb --> arm64_boot_el1_init
  ...
  setup_page_tables --> enable_mmu_el1
  click setup_page_tables href "https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/arm64_mmu.c#L515" "arch/arm64/src/common/arm64_mmu.c "
  enable_mmu_el1 --> arm64_isb
  click enable_mmu_el1 href "https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/arm64_mmu.c#L532" "arch/arm64/src/common/arm64_mmu.c "
  arm64_isb --> enable_mmu_el1
  click arm64_isb href "https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/barriers.h#L57" "arch/arm64/src/common/barriers.h "
  enable_mmu_el1 --> ***_HALT_***
  click enable_mmu_el1 href "https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/arm64_mmu.c#L542" "arch/arm64/src/common/arm64_mmu.c "

(Source)

The Call Graph is generated by our Block Execution Hook like so...

/// Hook Function for Block Emulation.
/// Called once for each Basic Block of Arm64 Instructions.
fn hook_block(
_: &mut Unicorn<()>, // Emulator
address: u64, // Block Address
size: u32 // Block Size
) {
// Ignore the memset() loop. TODO: Read the ELF Symbol Table to get address of memset().
if address >= 0x4008_9328 && address <= 0x4008_933c { return; }
print!("hook_block: address={:#010x}, size={:02}", address, size);
// Print the Function Name
let function = map_address_to_function(address);
if let Some(ref name) = function {
print!(", {}", name);
}
// Print the Source Filename
let loc = map_address_to_location(address);
if let Some((ref file, line, col)) = loc {
let file = file.clone().unwrap_or("".to_string());
let line = line.unwrap_or(0);
let col = col.unwrap_or(0);
print!(", {}:{}:{}", file, line, col);
}
println!();
// Print the Call Graph
call_graph(address, size, function, loc);
}

call_graph prints the Call Graph by looking up the Block Address in the ELF Context...

/// Print the Mermaid Call Graph for this Function Call:
/// cargo run | grep call_graph | cut -c 12-
fn call_graph(
_address: u64, // Code Address
_size: u32, // Size of Code Block
function: Option<String>, // Function Name
loc: Option<( // Source Location
Option<String>, // Filename
Option<u32>, // Line
Option<u32> // Column
)>
) {
// Get the Function Name
let Some(fname) = function
else { return; };
// Unsafe because `LAST_FNAME` is a Static Mutable
unsafe {
// Skip if we are still in the same Function
static mut LAST_FNAME: String = String::new();
static mut LAST_LOC: Option<(Option<String>, Option<u32>, Option<u32>)> = None;
if fname.eq(&LAST_FNAME) { return; }
// If this function has not been shown too often...
if can_show_function(&fname) {
// Print the Call Flow
if LAST_FNAME.is_empty() {
println!("call_graph: flowchart TD"); // Top-Down Flowchart
} else {
// URL looks like https://github.com/apache/nuttx/blob/master/arch/arm64/src/common/arm64_mmu.c#L541
let (file, line, _) = LAST_LOC.clone().unwrap_or((Some("".to_string()), None, None));
let file = file.unwrap_or("".to_string());
let line = line.unwrap_or(1) - 1;
let url = format!("https://github.com/apache/nuttx/blob/master/{file}#L{line}");
println!("call_graph: {LAST_FNAME} --> {fname}");
println!("call_graph: click {LAST_FNAME} href \"{url}\" \"{file} \"");
}
}
LAST_FNAME = fname;
LAST_LOC = loc;
}
}

We map the Block Address to Function Name and Source File in map_address_to_function and map_address_to_location...

/// Map the Arm64 Code Address to the Function Name by looking up the ELF Context
fn map_address_to_function(
address: u64 // Code Address
) -> Option<String> { // Function Name
// Lookup the Arm64 Code Address in the ELF Context
let context = ELF_CONTEXT.context.borrow();
let mut frames = context.find_frames(address)
.expect("failed to find frames");
// Return the Function Name
if let Some(frame) = frames.next().unwrap() {
if let Some(func) = frame.function {
if let Ok(name) = func.raw_name() {
let s = String::from(name);
return Some(s);
}
}
}
None
}
/// Map the Arm64 Code Address to the Source Filename, Line and Column
fn map_address_to_location(
address: u64 // Code Address
) -> Option<( // Returns...
Option<String>, // Filename
Option<u32>, // Line
Option<u32> // Column
)> {
// Lookup the Arm64 Code Address in the ELF Context
let context = ELF_CONTEXT.context.borrow();
let loc = context.find_location(address)
.expect("failed to find location");
// Return the Filename, Line and Column
if let Some(loc) = loc {
if let Some(file) = loc.file {
let s = String::from(file)
.replace("/private/tmp/nuttx/nuttx/", "")
.replace("arch/arm64/src/chip", "arch/arm64/src/a64"); // TODO: Handle other chips
Some((Some(s), loc.line, loc.column))
} else {
Some((None, loc.line, loc.column))
}
} else {
None
}
}

ELF_CONTEXT is explained here...

Other Emulators

What about emulating popular operating systems: Linux / macOS / Windows / Android?

Check out the Qiling Binary Emulation Framework...

How about other hardware platforms: STM32 Blue Pill and ESP32?

Check out QEMU...

TODO

TODO: Emulate Input Interupts from UART Controller

TODO: Emulate Apache NuttX NSH Shell with UART Controller

TODO: Select Arm Cortex-A53 as CPU

TODO: Emulate Multiple CPUs

TODO: Emulate Arm64 Memory Protection

TODO: Emulate Arm64 Generic Interrupt Controller version 2

TODO: Read the Symbol Table in NuttX ELF File to match the addresses received by Block Execution Hook

TODO: Emulate PinePhone's Allwinner A64 Display Engine. How to render the emulated graphics: Use Web Browser + WebAssembly + Unicorn.js? Will framebuffer emulation be slow?