Unclear panic due to what seems to be lack of memory / buffer overrun

[yanshay]

I'm writing an application using embassy and picoserve web server, and at a certain point it started panicking when serving a page, when I only added a route that wasn't even accessed by the client.
Just adding small piece of code, that doesn't get to run, caused the application to panic.

The panic is below.

Eventually I found that if I reduce embassy task-arena size, things get back to work. So seems like lack of memory caused panick.

The question is - why would lack of memory result in this kind of exception (InstProhibited)? I would expect it to either panic on allocation if it's alloc, or not compile if it lacks memory for static allocations.

InstrProhibited sounds like data overran the code, can this happen?
Doesn't that mean that it opens the system to buffer over-runs attacks which when using rust the promise is can't take place?

Exception occurred 'InstrProhibited'
Context
PC=0x00000000       PS=0x00060530
A0=0x8008cedf       A1=0x3ffdb9c0       A2=0x00000023       A3=0x00000007       A4=0x3ffd75d8
0x3ffd75d8 - g_osi_funcs_p
    at ??:??
A5=0x00000000       A6=0x00000001       A7=0x00000003       A8=0x80085645       A9=0x3ffc940c
0x3ffc940c - embassy_executor::_export::ARENA
    at ??:??
A10=0x3ffbb520      A11=0x3ffdb150      A12=0x3ffc9190      A13=0x3ffc919c      A14=0x3ffc9100
0x3ffbb520 - esp32xx_embassy_wifi::____embassy_main_task::{{closure}}::HEAP
    at ??:??
0x3ffc9190 - embassy_executor::_export::ARENA
    at ??:??
0x3ffc919c - embassy_executor::_export::ARENA
    at ??:??
0x3ffc9100 - embassy_executor::_export::ARENA
    at ??:??
A15=0x00000005
SAR=00000020
EXCCAUSE=0x00000014 EXCVADDR=0x00000000
LBEG=0x4000c2e0     LEND=0x4000c2f6     LCOUNT=0x00000000
THREADPTR=0x00000000
SCOMPARE1=0x00000000
BR=0x00000000
ACCLO=0x00000000    ACCHI=0x00000000
M0=0x00000000       M1=0x00000000       M2=0x00000000       M3=0x00000000
F64R_LO=0x00000000  F64R_HI=0x00000000  F64S=0x00000000
FCR=0x00000000      FSR=0x00000000
F0=0x00000000       F1=0x00000000       F2=0x00000000       F3=0x00000000       F4=0x00000000
F5=0x00000000       F6=0x00000000       F7=0x00000000       F8=0x00000000       F9=0x00000000
F10=0x00000000      F11=0x00000000      F12=0x00000000      F13=0x00000000      F14=0x00000000
F15=0x00000000
0x40139fc8
esp_wifi_internal_free_rx_buffer
    at ??:??
0x400ebaa3
smoltcp::iface::interface::Interface::socket_ingress
    at ??:??
0x400e4570
embassy_executor::raw::SyncExecutor::poll::{{closure}}
    at ??:??
0x400fe0ce
main
    at ??:??
0x40110be9
Reset
    at ??:??
0x401047fa
ESP32Reset
    at ??:??

[bugadani]

A bit of a technicality, but this isn't a panic, this is an exception raised by the hardware.

Note that your PC is 0. Please provide some information about what you are doing, ideally a reproducer we can try. As your top stackframe looks to point at esp_wifi_internal_free_rx_buffer we'd like to know how this issue can occur.

[yanshay]

I'll try to build a reproducer.

[yanshay]

Attached is a small messy program reproducing the issue.
Just:
On esp32 (probably matters due to exact memory size):

• cargo run --release
• Browse to http://device-ip-from-log/config

BTW, I noticed that if I add a bit more code (that is never really accessed), it starts issuing "LoadProhibited" instead of "InstrProhibited".
esp32-memory-crash.zip

[MabezDev]

Try enabling debug assist, here's the example: https://github.com/esp-rs/esp-hal/blob/main/examples/src/bin/debug_assist.rs

That should detect if it's a stack overflow.

[yanshay]

btw I cannot build your reproducer because of a plethora of compile errors

That's really trange, I just downloaded the zip from here, extracted to a new folder and cargo build --release after setting SSID and PASSWORD environment vars.

A single warning and it was built.

Can you upload the compile errors?
Maybe you're using some newer version of something?

[bjoernQ]

With both 1.81 and 1.82:

warning: unused import: `response::IntoResponse`
  --> src/main.rs:36:5
   |
36 |     response::IntoResponse,
   |     ^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_imports)]` on by default

error: item does not constrain `____embassy_main_task::{closure#0}::T::{opaque#0}`, but has it in its signature
   --> src/main.rs:242:19
    |
242 |         let app = make_static!(make_app());
    |                   ^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: consider moving the opaque type's declaration and defining uses into a separate module
note: this opaque type is in the signature
   --> src/main.rs:242:19
    |
242 |         let app = make_static!(make_app());
    |                   ^^^^^^^^^^^^^^^^^^^^^^^^
    = note: this error originates in the macro `$crate::make_static` which comes from the expansion of the macro `make_static` (in Nightly builds, run with -Z macro-backtrace for more info)

error: item does not constrain `AppRouter::{opaque#0}`, but has it in its signature
   --> src/main.rs:330:10
    |
330 | async fn web_task(
    |          ^^^^^^^^
    |
    = note: consider moving the opaque type's declaration and defining uses into a separate module
note: this opaque type is in the signature
   --> src/main.rs:327:18
    |
327 | type AppRouter = impl picoserve::routing::PathRouter;
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: item does not constrain `AppRouter::{opaque#0}`, but has it in its signature
   --> src/main.rs:336:8
    |
336 |   ) -> ! {
    |  ________^
337 | |     let port = 80;
338 | |     let mut tcp_rx_buffer = [0; 1024];
339 | |     let mut tcp_tx_buffer = [0; 1024];
...   |
352 | |     .await
353 | | }
    | |_^
    |
    = note: consider moving the opaque type's declaration and defining uses into a separate module
note: this opaque type is in the signature
   --> src/main.rs:327:18
    |
327 | type AppRouter = impl picoserve::routing::PathRouter;
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: item does not constrain `AppRouter::{opaque#0}`, but has it in its signature
   --> src/main.rs:329:1
    |
329 | #[embassy_executor::task(pool_size = WEB_TASK_POOL_SIZE)]
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: consider moving the opaque type's declaration and defining uses into a separate module
note: this opaque type is in the signature
   --> src/main.rs:327:18
    |
327 | type AppRouter = impl picoserve::routing::PathRouter;
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    = note: this error originates in the attribute macro `embassy_executor::task` (in Nightly builds, run with -Z macro-backtrace for more info)

error: item does not constrain `____embassy_main_task::{closure#0}::T::{opaque#0}`, but has it in its signature
   --> src/main.rs:244:22
    |
244 |           let config = make_static!(picoserve::Config::new(picoserve::Timeouts {
    |  ______________________^
245 | |             start_read_request: Some(Duration::from_secs(5)),
246 | |             read_request: Some(Duration::from_secs(1)),
247 | |             write: Some(Duration::from_secs(1)),
248 | |         })
249 | |         .keep_connection_alive());
    | |_________________________________^
    |
    = note: consider moving the opaque type's declaration and defining uses into a separate module
note: this opaque type is in the signature
   --> src/main.rs:244:22
    |
244 |           let config = make_static!(picoserve::Config::new(picoserve::Timeouts {
    |  ______________________^
245 | |             start_read_request: Some(Duration::from_secs(5)),
246 | |             read_request: Some(Duration::from_secs(1)),
247 | |             write: Some(Duration::from_secs(1)),
248 | |         })
249 | |         .keep_connection_alive());
    | |_________________________________^
    = note: this error originates in the macro `$crate::make_static` which comes from the expansion of the macro `make_static` (in Nightly builds, run with -Z macro-backtrace for more info)

warning: `esp32xx-embassy-wifi` (bin "esp32xx-embassy-wifi") generated 1 warning
error: could not compile `esp32xx-embassy-wifi` (bin "esp32xx-embassy-wifi") due to 5 previous errors; 1 warning emitted
err process exited unsuccessfully: exit code: 101

[bugadani]

I think let app = make_static!(make_app()); works only with 1.78 or maybe 79 at most

[yanshay]

I'm on rustc 1.80.0-nightly (1b259e0b8 2024-07-22) (1.80.0.0)

So when I'll upgrade I'll need to change all make_static! to the mk_static!? alternative?

[yanshay]

I can't upgrade the compiler at this time, so I changed the code to use mk_static! assuming that's the issue.
attached updated reproducer
esp32-memory-crash-try2.zip

btw - I didn't check that the issue takes place after that change assuming the resulting code is the same.