diff --git a/_posts/2024-05-01-stack-usage.markdown b/_posts/2024-05-01-stack-usage.markdown new file mode 100644 index 0000000..3269305 --- /dev/null +++ b/_posts/2024-05-01-stack-usage.markdown @@ -0,0 +1,73 @@ +--- +layout: post +title: "Avoiding stack overflows on CHERIoT" +date: 2024-05-01 +categories: rtos stack programming +author: "David Chisnall" +--- + +If you put code in a compartment and do nothing to protect the interfaces, there are a lot of ways that a caller can make that compartment crash. +In some cases, this doesn't matter. +The compartment exists to provide confidentiality and integrity, but not necessarily availability. +The TLS compartment in the network stack is a good example of this. +It provides strong flow isolation and so making it crash will impact only the TLS session of the caller. +The caller can also simply call the close function on a TLS session and so we don't worry about their ability to break a TLS session, only about their ability to send plain text over the network or extract keying material from the TLS session. + +In other contexts, availability is far more important. +In the core of the RTOS, the scheduler and allocator both have availability requirements. +If you can crash the allocator while it holds the lock over the heap, you can prevent any future memory allocation. +If you can crash the scheduler in the middle of updating run queues, you may be able to prevent certain threads ever running. + +In most platforms, it's easy to make a function crash by moving the stack pointer to near the end just before calling it. +The function will run off the end of the stack and hit a guard page on MMU / MPU systems or the end of the stack capability on CHERI systems. +This is even more crucial on embedded systems, where stacks tend to be small. +Large desktop or server systems often have stacks of 1-4 MiBs, which are large enough for most programs to treat as infinite. +Embedded systems may have stacks that are under 1 KiB. +Even without the security implications from distrust between compartments, having systems fail because they ran out of stack space is far from ideal. + +These problems are relevant on CHERIoT. +The caller can constrain the amount of space available on the stack before a cross-compartment call. +If the callee requires more stack space than the caller provides then this can cause problems. + +From the start, the CHERIoT ABI has provided some mitigation for this. +Every cross-compartment entry point is described by an entry in an export table. +One of the fields in an export-table entry indicates the amount of stack space that a call requires. +If the stack has less space than is available then the call will fail without invoking the callee at all. + +This leads to an obvious problem: How do you set this value to something sensible? +By default, the compiler sets it to the stack space required for the function that implements the entry point. +This means that any function that doesn't call any other functions is fine. +It also means that, if you put stack checks in the function *before* calling any other functions, then you can guarantee that they will work correctly. + +This still leaves a lot of work to determine how much stack space you actually need. +The compiler or other tooling could build a static control-flow graph for the current compilation unit, and possibly even the current compartment, but what happens if you call library functions? + +It turns out that we already had the building block for a good solution. +CHERIoT guarantees that you can't accidentally leak data left on the stack through a cross-compartment call. +This is done by zeroing the portion of the stack that is going to be shared before and after a cross-compartment call. +When we did this initially, we quickly realised that we were spending a lot of time zeroing memory that was already full of zeroes. +This got worse the larger stacks were. +If you had a 4 KiB stack and a function that used 128 bytes, you may still end up zeroing almost 4 KiB *twice* (once on call, once on return). + +To fix this, we introduced the stack high-water mark. +This is configured with the range of the stack and tracks all stores into that range. +Any store below the current high-water mark (remember, stacks grow down, so the 'high'-water mark is actually at the bottom of the memory for the stack) moves the mark. +The switcher can read this before and after the call and zero only memory that's used. +If you start a thread, do some stack allocation, and then call a new compartment, we don't need to do any zeroing. +If you call some internal functions, return, and then call another compartment, we zero only the bit of the stack that you've left data on. +If you call a compartment that uses 128 bytes of stack space then we will zero 128 bytes of stack on return, independent of the stack size. + +This means that we *already* had a mechanism for dynamically determining how much stack a particular compartment entry point needed. +All that we needed to do was expose it. +For C++, we've [wrapped this up in a class that lets you report the highest stack usage for a function across multiple invocations](https://cheriot.org/book/compartments.html#_ensuring_adequate_stack_space). +A lot of functions have data-dependent control flow and so data-dependent worst-case stack usage. +By running a set of tests over a function, you can find the worst-case stack usage. +This class can either log the highest stack usage that it sees or can crash the compartment if the expected amount is exceeded. + +Once you've made determined the maximum stack usage for a function, you can use the `__cheriot_minimum_stack` attribute to set the value in the export table. +We've now done this for the allocator and the scheduler, which should make both robust. + +Confidentiality and integrity were the primary goals for CHERIoT. +Our first adversarial security evaluation did not find any ways of violating confidentiality or integrity. +Since then, we've been working to add availability to the guarantees that we can provide. +This is just one of many steps in making the CHERIoT platform a solid foundation for high-availability embedded systems.