From 40e45a414f7963ced189a51b8888a56395b16c03 Mon Sep 17 00:00:00 2001 From: Jason Andrews Date: Thu, 2 Nov 2023 16:05:35 -0500 Subject: [PATCH] tested new Learning Path on dynamic memory allocation --- .../1_dynamic_memory_allocation.md | 80 ++++++++++--------- .../2_designing_a_dynamic_memory_allocator.md | 69 ++++++++-------- ...implementing_a_dynamic_memory_allocator.md | 59 ++++++++------ .../4_conclusions_further_work.md | 73 +++++++++-------- .../dynamic-memory-allocator/_index.md | 39 +++++++++ .../dynamic-memory-allocator/_next-steps.md | 2 +- .../dynamic-memory-allocator/_review.md | 4 +- .../dynamic-memory-allocator/_index.md | 31 ------- contributors.csv | 1 + 9 files changed, 194 insertions(+), 164 deletions(-) rename content/learning-paths/{laptops-and-desktops => cross-platform}/dynamic-memory-allocator/1_dynamic_memory_allocation.md (61%) rename content/learning-paths/{laptops-and-desktops => cross-platform}/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md (67%) rename content/learning-paths/{laptops-and-desktops => cross-platform}/dynamic-memory-allocator/3_implementing_a_dynamic_memory_allocator.md (87%) rename content/learning-paths/{laptops-and-desktops => cross-platform}/dynamic-memory-allocator/4_conclusions_further_work.md (72%) create mode 100644 content/learning-paths/cross-platform/dynamic-memory-allocator/_index.md rename content/learning-paths/{laptops-and-desktops => cross-platform}/dynamic-memory-allocator/_next-steps.md (88%) rename content/learning-paths/{laptops-and-desktops => cross-platform}/dynamic-memory-allocator/_review.md (95%) delete mode 100644 content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_index.md diff --git a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/1_dynamic_memory_allocation.md b/content/learning-paths/cross-platform/dynamic-memory-allocator/1_dynamic_memory_allocation.md similarity index 61% rename from content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/1_dynamic_memory_allocation.md rename to content/learning-paths/cross-platform/dynamic-memory-allocator/1_dynamic_memory_allocation.md index 1816354e9..a410bc996 100644 --- a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/1_dynamic_memory_allocation.md +++ b/content/learning-paths/cross-platform/dynamic-memory-allocator/1_dynamic_memory_allocation.md @@ -1,20 +1,22 @@ --- -title: Dynamic Memory Allocation +title: Dynamic memory allocation weight: 2 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Dynamic vs. Static Allocation +## Dynamic vs. static memory allocation -In this learning path you will learn how to implement dynamic memory allocation. -If you have used C's "heap" (`malloc`, `free`, etc.) before, that is one example +In this Learning Path you will learn how to implement dynamic memory allocation. +If you have used the C programming language "heap" (`malloc`, `free`, etc.) before, that is one example of dynamic memory allocation. -It allows programs to allocate memory while they are running without knowing -at build time what amount of memory they will need. In constrast to static -memory allocation where the amount is known at build time. +Dynamic memory allocation allows programs to allocate memory while they are running without knowing +at build time how much memory they will need. In contrast, static +memory allocation is used when the amount of memory is known at build time. + +The code sample below shows both dynamic and static memory allocation: ```C #include @@ -27,10 +29,10 @@ void fn() { } ``` -The example above shows the difference. The size and location of `a` is known +In the example above, the size and location of `a` is known when the program is built. The size of `b` is also known, but its location is not. -It may even never be allocated, as this pseudocode example shows: +Sometimes, memory may never be allocated, as in the pseudocode example below: ```C int main(...) { @@ -40,37 +42,37 @@ int main(...) { } ``` -If the user passes no arguments to the program, there's no need to allocate space -for `b`. If they do, `malloc` will find space for it. +The arguments passed to the program determine if memory is allocated or not. -## malloc +## The C library malloc function The C standard library provides a special function [`malloc`](https://en.cppreference.com/w/c/memory/malloc). `m` for "memory", -`alloc` for "allocate". This can be used to ask for a suitably sized memory -location while the program is running. +`alloc` for "allocate". This is used to ask for a suitably sized memory +location while a program is running. ```C void *malloc(size_t size); ``` -The C library will then look for a chunk of memory with size of at least `size` +The C library looks for a chunk of memory with size of at least `size` bytes in a large chunk of memory that it has reserved. For instance on Ubuntu -Linux, this will be done by GLIBC. +Linux, this is done by GLIBC. The example at the top of the page is trivial of course. As it is we could just statically allocate both integers like this: + ```C void fn() { int a, b = 0; } ``` -That's ok if this data is never be returned from this function. Or in other -words, if the lifetime of this data is equal to that of the function. +Variables `a` and `b` work fine if they are not needed outside of the function. Or in other +words, if the lifetime of the data is equal to that of the function. -A more complicated example will show you when that is not the case, and the value -lives longer than the function that created it. +A more complex example shows when this is not the case, and the values +live longer than the creating function. ```C #include @@ -93,7 +95,7 @@ void add_entry(Entry *entry, int data) { ``` What you see above is a struct `Entry` that defines a singly-linked-list entry. -Singly meaining that you can go forward via `next`, but you cannot go backwards +Singly meaning that you can go forward via `next`, but you cannot go backwards in the list. There is some data `data`, and each entry points to the next entry, `next`, assuming there is one (it will be `NULL` for the end of the list). @@ -111,8 +113,8 @@ Now you want to add another `Entry` to this list at runtime. So you do not know ahead of time what it will contain, or if we indeed will add it or not. Where would you put that entry? -* If it is another global variable, we would have to declare many empty `Entry`s - and hope we never needed more than that amount. +* If it is another global variable, we would have to declare many empty `Entry` +values and hope {{% notice Other Allocation Techniques%}} Although in this specific case global variables aren't a good solution, there are @@ -120,7 +122,7 @@ cases where large sets of pre-allocated objects can be beneficial. For example, it provides a known upper bound of memory usage and makes the timing of each allocation predictable. -However, we will not be covering these techniques in this learning path. It will +However, these techniques are not covered in this Learning Path. It will however be useful to think about them after you have completed this learning path. {{% /notice %}} @@ -128,38 +130,38 @@ path. * If it is in a function's stack frame, that stack frame will be reclaimed and modified by future functions, corrupting the new `Entry`. -So you can see, we must use dynamic memory allocation. Which is why the `add_entry` +So you can see, dynamic memory allocation is required. Which is why the `add_entry` shown above calls `malloc`. The resulting pointer points to somewhere not in the program's global data section or in any function's stack space, but in the -heap memory. Where it can live until we `free` it. +heap memory. It will stay in the heap until a call to `free` is made. -## free +## The C library free function -You cannot ask malloc for memory forever. Eventually that space behind the scenes -will run out. So you should give up your dynamic memory once it is not needed, +You cannot ask malloc for memory forever. Eventually the space behind the scenes +will run out. You should give up your dynamic memory once it is not needed, using [`free`](https://en.cppreference.com/w/c/memory/free). ```C void free(void *ptr); ``` -You call `free` with a pointer previously given to you by `malloc`, and this tells -the heap that we no longer need this memory. +You call `free` with a pointer previously returned by `malloc`, and this tells +the heap that the memory is no longer needed. -{{% notice Undefined Behaviour%}} -You may wonder what happens if you don't pass the exact pointer to `free`, as -`malloc` returned to you. The result varies as this is "undefined behaviour". +{{% notice Undefined Behavior%}} +You may wonder what happens if you don't pass the exact same pointer to `free` as +`malloc` returned. The result varies as this is "undefined behavior". Which essentially means a large variety of unexpected things can happen. In practice, many allocators will tolerate this difference or reject it outright -if it's not possible to do something sensbile with the pointer. +if it's not possible to do something sensible with the pointer. -Remember that just because one allocator handles this a certain way, does not -mean all will. Indeed, that same allocator may handle it differently for +Remember, just because one allocator handles this a certain way, does not +mean all allocators will be the same. Indeed, that same allocator may handle it differently for different allocations within the same program. {{% /notice %}} -So, you can use `free` to remove an item from your linked list. +You can use `free` to remove an item from your linked list. ```C void remove_entry(Entry* previous, Entry* entry) { @@ -183,5 +185,5 @@ to remove, so that the list skips over it. With `entry` now isolated we call [A]---------->[C] | [A] [C] ``` -That covers the high level how and why of using `malloc` and `free`, next you'll +That covers the high level how and why of using `malloc` and `free`, next you will see a possible implementation of a dynamic memory allocator. \ No newline at end of file diff --git a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md b/content/learning-paths/cross-platform/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md similarity index 67% rename from content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md rename to content/learning-paths/cross-platform/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md index 4b7b226d3..495351eb8 100644 --- a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md +++ b/content/learning-paths/cross-platform/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md @@ -1,41 +1,41 @@ --- -title: Designing a Dynamic Memory Allocator +title: Design a dynamic memory allocator weight: 3 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## High Level Design +## High level design To begin with, decide which functions your memory allocator will provide. We have described `malloc` and `free`, there are more provided by the [C library](https://en.cppreference.com/w/c/memory). -This will assume you just need `malloc` and `free`. Start with those and write -out their behaviours, as the programmer using your allocator will see. +This will assume you just need `malloc` and `free`. The new implementations will +be called `simple_malloc` and `simple_free`. Start with just two functions and write +out their behaviors. -There will be a function, `malloc`. It will: -* Take a size in bytes as a parameter. -* Try to allocate some memory. -* Return a pointer to that memory, NULL pointer otherwise. +The first function is `simple_malloc` and it will: +* Take a size in bytes as a parameter +* Try to allocate the requested memory +* Return a pointer to that memory or return a NULL pointer if the memory cannot be allocated -There will be a function `free`. It will: -* Take a pointer to some previously allocated memory as a parameter. -* Mark that memory as avaiable for future allocations. +The second function is `simple_free` and it will: +* Take a pointer to some previously allocated memory as a parameter +* Mark that memory as available for future allocations From this you can see that you will need: * Some large chunk of memory, the "backing storage". -* A way to mark parts of that memory as allocated, or available for allocation. +* A way to mark parts of that memory as allocated, or available for allocation -## Backing Storage +## Backing storage The memory can come from many sources. It can even change size throughout the -program's execution if you wish. For your allocator you'll keep it as simple -as possible. +program's execution if you wish. For your allocator you can keep it simple. A single, statically allocated global array of bytes will be your backing -storage. So you can do dynamic allocation of parts of a statically allocated +storage. You can do dynamic allocation of parts of a statically allocated piece of memory. ```C @@ -43,21 +43,20 @@ piece of memory. static char storage[STORAGE_SIZE]; ``` -## Record Keeping +## Record keeping This backing memory needs to be annotated somehow to record what has been -allocated so far. There are many, many ways to do this. With the biggest choice -here being whether to store these records in the heap itself, our outside of it. +allocated so far. There are many ways to do this. Te biggest choice +is whether to store these records in the heap itself or outside of it. -We will not go into those tradeoffs here, and instead you will put the records -in the heap, as this is relatively simple to do. +The easiest way is to put the records in the heap. -What should be in your records? Think about what question the software will ask -us. Can you give me a pointer to an area of free memory of at least this size? +What should be in the records? Think about the question the caller is asking. +Can you give me a pointer to an area of memory of at least this size? For this you will need to know: -* Which ranges of the backing storage have been allocated or not. -* How large each of ranges sections is. This includes free areas. +* The ranges of the backing storage that have already been allocated +* The size of each section, both free and allocated Where a "range" a pointer to a location, a size in bytes and a boolean to say whether the range is free or allocated. So a range from 0x123 of 345 bytes, @@ -67,7 +66,7 @@ that has been allocated would be: start: 0x123 size: 345 allocated: true ``` -For the intial state of a heap of size `N`, you will have one range of +For the initial state of a heap of size `N`, you will have one range of unallocated memory. ```text @@ -102,7 +101,7 @@ Pointer: 0x4 Size: N-4 Allocated: False range = 0x4 + (N-4) = 1 beyond the end of the heap, so the walk is finished. ``` -`free` uses the pointer given to it to find the range it needs to deallocate. +`simple_free` uses the pointer given to it to find the range it needs to deallocate. Let's say the 4 byte allocation was freed: ```text @@ -110,18 +109,18 @@ Pointer: 0x0 Size: 4 Allocated: False Pointer: 0x4 Size: N-4 Allocated: False ``` -Since `free` gets a pointer directly to the allocation you know exactly which +Since `simple_free` gets a pointer directly to the allocation you know exactly which range to modify. The only change made is to the boolean which marks it as allocated or not. The location and size of the range stay the same. {{% notice Merging Free Ranges%}} -The allocator presented here will not merge free ranges like the 2 above. This +The allocator presented here does not merge free ranges like the 2 above. This is a deliberate limitation and addressing this is discussed later. {{% /notice %}} -## Record Storage +## Record storage -You'll keep these records in heap which means using some of the allocated space +You will keep these records in the heap which means using some of the allocated space for them on top of the allocation itself. The simplest way to do this is to prepend each allocation with the range @@ -135,13 +134,13 @@ ease. <...and so on until the end of the heap...> ``` -Pointers returned by `malloc` are offset to just beyond the range information. -When `free` receives a pointer, it can get to the range information by +Pointers returned by `simple_malloc` are offset to just beyond the range information. +When `simple_free` receives a pointer, it can get to the range information by subtracting the size of that information from the pointer. Using the example above: ```text -free(my_ptr); +simple_free(my_ptr); 0x00: [ptr, size, allocated] <-- my_ptr - sizeof(range information) 0x08: <...> <-- my_ptr @@ -153,7 +152,7 @@ calculations above must be adjusted. The allocator presented here does not concern itself with alignment, which is why it can do a simple subtraction. {{% /notice %}} -## Running Out Of Space +## Running out of space The final thing an allocator must do is realise it has run out of space. This is simply achieved by knowing the bounds of the backing storage. diff --git a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/3_implementing_a_dynamic_memory_allocator.md b/content/learning-paths/cross-platform/dynamic-memory-allocator/3_implementing_a_dynamic_memory_allocator.md similarity index 87% rename from content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/3_implementing_a_dynamic_memory_allocator.md rename to content/learning-paths/cross-platform/dynamic-memory-allocator/3_implementing_a_dynamic_memory_allocator.md index 3169ecdce..b486463eb 100644 --- a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/3_implementing_a_dynamic_memory_allocator.md +++ b/content/learning-paths/cross-platform/dynamic-memory-allocator/3_implementing_a_dynamic_memory_allocator.md @@ -1,26 +1,34 @@ --- -title: Implementing a Dynamic Memory Allocator +title: Implement a dynamic memory allocator weight: 4 ### FIXED, DO NOT MODIFY layout: learningpathall --- -## Project Structure +The source code of the `simple_malloc` and `simple_free` memory allocation functions are below. +Everything required to build and run example allocations are also provided. -The file layout will be as follows: -* `CMakeLists.txt` - To tell `cmake` how to configure the project. +You will need a Linux machine to try the code and see how the allocation works. + +## Project structure + +The files used are: +* `CMakeLists.txt` - Tells `cmake` how to configure and build the project. * `heap.c` - The dynamic memory allocator implementation. * `heap.h` - Function declarations including your new `simple_malloc` and `simple_free` functions. -* `main.c` - A program that makes use of `simple_malloc` and `simple_free`. +* `main.c` - A test program that makes use of `simple_malloc` and `simple_free`. + +Building it will produce a single binary, `demo`, that you can run and see the results. + +## Source code -Building it will produce a single binary, `demo`, that you will run to see the -results. +The files are listed below. -## Sources +Use a text editor to copy and paste the contents of each file on a Linux machine. -### CMakeLists.txt +Contents of `CMakeLists.txt`: ``` {file_name="CMakeLists.txt"} cmake_minimum_required(VERSION 3.15) @@ -30,7 +38,7 @@ project(MemoryAllocatorDemo C) add_executable(demo main.c heap.c) ``` -#### heap.h +Contents of `heap.h`: ```C {file_name="heap.h"} #include @@ -45,20 +53,20 @@ void *simple_malloc(size_t size); void simple_free(void *ptr); ``` -## heap.c +## Information about heap.c Please refer to the comments in the source code here for detailed explanations -of each function. We will cover the key elements here up front. +of each function. You can identify a few key elements before studying the code. First is `storage`, this is the backing storage which is a global char array. This is where the ranges, represented by `Header`, are stored. Each `Header` is written to the start of the allocated range. This means that -`malloc` returns a pointer that points just beyond this location. `free` on the +`simple_malloc` returns a pointer that points just beyond this location. `simle_free` on the other hand, deducts the size of `Header` from the pointer parameter to find the range information. -When the heap is initialised with `simple_heap_init`, a single range is setup +When the heap is initialized with `simple_heap_init`, a single range is setup that covers the whole heap and marks it as unallocated. To find a free range, `find_free_space` walks the heap using these `Header` @@ -78,6 +86,8 @@ enabled. Generally run to run, the output addresses may change. Focus on the relative values of pointers in relation to where the heap starts and ends. {{% /notice %}} +Contents of `heap.c`: + ```C {file_name="heap.c"} #include #include @@ -243,7 +253,7 @@ void simple_free(void *ptr) { } ``` -### main.c +Contents of `main.c`: ```C { file_name="main.c"} #include "heap.h" @@ -263,38 +273,41 @@ int main() { } ``` -The code here does allocation and deallocation of memory. This tests the heap +The main code does allocation and deallocation of memory. This tests the heap code but also highlights an interesting problem that you'll see more about later. -## Building +## Build the source code -First install dependencies. +Install the required tools using the command: ```bash sudo apt install -y cmake ninja-build ``` -Then configure using CMake. We recomend a Debug build for the extra safety the +Next, configure using CMake. You can use a Debug build for the extra safety the asserts bring. ```bash cmake . -DCMAKE_BUILD_TYPE=Debug -G Ninja ``` -Then build with `ninja` +## Build and run a test + +Build the executable with `ninja`: ```bash ninja ``` -This should result in a `demo` executable in the same folder. Run this to see -the allocator in action. +You now have a `demo` executable in the same folder. + +Run `demo` to see the allocator in action: ```bash ./demo ``` -## Output +## Review the program output The output addresses will vary depending on where backing memory gets allocated by your system but this is the general form you should expect: diff --git a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/4_conclusions_further_work.md b/content/learning-paths/cross-platform/dynamic-memory-allocator/4_conclusions_further_work.md similarity index 72% rename from content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/4_conclusions_further_work.md rename to content/learning-paths/cross-platform/dynamic-memory-allocator/4_conclusions_further_work.md index f0d69fe3d..4d3545f99 100644 --- a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/4_conclusions_further_work.md +++ b/content/learning-paths/cross-platform/dynamic-memory-allocator/4_conclusions_further_work.md @@ -1,5 +1,5 @@ --- -title: Conclusions +title: Memory allocation summary weight: 5 ### FIXED, DO NOT MODIFY @@ -8,29 +8,29 @@ layout: learningpathall ## Conclusions -You've now had a glimpse into the world of dynamic memory allocation, and +You have a glimpse into the world of dynamic memory allocation, and probably have more questions than answers. You may have noticed some oversights -in the implementation presented, and you're almost certainly right, we'll get to -those shortly. +in the implementation presented, and you are almost certainly right. Overall your take away from this material is that "dynamic" memory allocation can mean many things. Sometimes it is all dynamic, sometimes it is a dynamic face with a static allocation behind it. This will change depending on the performance and complexity needs of the application. -Fundementally it provides a way to get memory you did not know whether you would -need when the program was written. You knew you would need some non-zero amount -and dynamic allocation lets you ask for it while the program is running. +Fundamentally, dynamic memory allocation provides a way to get memory you +did not know whether you would +need when the program was written. You likely know you need some amount +of memory and dynamic allocation lets you ask for it while the program is running. The implementation shown here is a "classic" heap, and a very simple one at that (not quite minimal, look up "bump allocator" for that). -Memory allocation is a whole field of study, and you can use this implementation -as a base for further research if you wish. +Memory allocation is an entire field of study, and you can use this implementation +as a basis for further research. -## Further Work +## Further work -### Merging Free Ranges +### Merging free ranges Look again at the last logging example on the previous page. @@ -44,31 +44,31 @@ Look again at the last logging example on the previous page. ``` What's wrong with these ranges? Nothing, until you allocate something >= 249 -bytes. We should be able to put that at address `0x55e68c41f0ac`, but because -we treat the 2 free ranges as separate, we can't put it there, or in the second -free range. +bytes. The allocator should be able to put that at address `0x55e68c41f0ac`, but because +the 2 free ranges are separated, the requested memory doesn't fit. ```text [0x55e68c41f0ac -> 0x55e68c41f1a4) : 0x00000000000000f8 (free, size = 248 bytes) [0x55e68c41f1a4 -> 0x55e68c41f2ac) : 0x0000000000000108 (free, size = 264 bytes) ``` -To solve this, you would need some kind of cleanup step after a free. Where -free ranges next to each other are merged into one. +To solve this, you would need a cleanup step after a free. Where +free ranges next to each other are merged into one free range. Then again, this does add some overhead. Perhaps it shouldn't be called on every -free. Think about the tradeoff there (and don't be afraid to change the data +free. Think about the tradeoff (and don't be afraid to change the data structures you've used, they are not perfect either). -### Memory Safety (Or Lack Of) +### Memory safety A big problem with memory in general is code accessing or changing memory that -it should not. The allocator presented here is certainly vunerable to all the +it should not. The allocator presented here is certainly vulnerable to all the classic memory exploits, which you can try out yourself. Replace the allocations in `main.c` with these to see what happens. -Use after free: +Here is a use after free: + ```C int *ptr = simple_malloc(sizeof(int)); *ptr = 123; @@ -81,7 +81,8 @@ There's a good chance `ptr2` will point to the same place as `ptr`. Meaning that someone could use `ptr` to modify the data now at `ptr2`. This can be even worse if the type of that data has changed in the meantime. -Double free: +Here is a double free: + ```C int *ptr = simple_malloc(sizeof(int)); simple_free(ptr); @@ -97,7 +98,7 @@ at the same place as `ptr`. When `ptr` is freed again, this would free the `ptr2 allocation as well. Meaning that instead of being its own allocation, `ptr3` also ends up pointing -to the same location as `ptr2`. So modiying one modifies the other. +to the same location as `ptr2`. So modifying one modifies the other. Another possibility is that memory that was previously freed is used as part of a larger allocation. So the original range header is now in the middle of the @@ -107,7 +108,8 @@ When free is called for the second time, the allocator may blindly write to wher it would have stored the metadata for the original allocation. In doing so, it will corrupt the original allocation. -Buffer overflow: +Here is buffer overflow: + ```C char *ptr = simple_malloc(4); char *ptr2 = simple_malloc(4); @@ -118,7 +120,7 @@ Buffer overflow: first one. Writing to `ptr[4]` overflows the array, because the maximum index is only 3. -This would corrupt the header attached to the `ptr2` allocation. In the case +This will corrupt the header attached to the `ptr2` allocation. In the case of your allocator, it would likely change the size of the allocation to just 1 byte. @@ -127,39 +129,41 @@ That's a selection of the many, many, possible attacks on the heap. You could consider how they might be mitigated, or even try applying some of them to the heap you have just written. -### Special Case Allocators +### Special case allocators Imagine you are writing a video game with a fixed memory budget and need predictable performance. Do you think a heap that has to walk a variable number of ranges would be able to achieve that? -If you think it wouldn't, you could look into +If you think it wouldn't, you can look into [Region-Based Memory Management](https://en.wikipedia.org/wiki/Region-based_memory_management). -(whether it would or not depends enitrely on your application's requirements) +(whether it would or not depends entirely on your application's requirements) -This takes advantages of scenarios where you know the upper limit of objects you +This takes advantage of scenarios where you know the upper limit of objects you will need, along with their types and sizes. -For the video game, maybe you are making a menu that will have at most 256 +For a video game, maybe you are making a menu that will have at most 256 entries. Why not statically allocate an array of 256 menu item objects on start up? Then simply construct a new item in place in the array as you need them. It is more overhead if the menu is always small, but it's very predictable. Maximum memory use is known and there is no variable time taken to walk the heap. -You could also mix this approach into a traditional heap, using areas of memory +You can also mix this approach into a traditional heap, using areas of memory only for certain types or sizes of data. For example, could it reduce the metadata overhead for small allocations (e.g. a 4 byte allocation that may require > 4 bytes of metadata)? -### LD_PRELOAD +### The LD_PRELOAD environment variable If your allocator grows to support all the C standard library functions, you can try using it instead of the one your system C library provides. On Linux this is done using the environment variable `LD_PRELOAD`. +Set the environment variable to point to your allocator instead of the one provided by Linux: + ``` LD_PRELOAD= ``` @@ -173,9 +177,12 @@ prefix from the functions to do this. Note that if you only implement a subset of the memory management functions, the program being run will get the rest from the system C library. This will -almost certianly lead to a crash when it tries to, for example, `realloc` a +almost certainly lead to a crash when it tries to, for example, `realloc` a pointer that your heap produced, but instead asks the system heap to do it. Finally, you will likely need a lot more storage for the heap. Either increase the size of the static allocation, or consider using `mmap` to ask the kernel -for memory, as C libraries tend to do instead. \ No newline at end of file +for memory, as C libraries tend to do instead. + +There are many things to learn about dynamic memory allocation, but it helps +to have a good understanding of the basics. \ No newline at end of file diff --git a/content/learning-paths/cross-platform/dynamic-memory-allocator/_index.md b/content/learning-paths/cross-platform/dynamic-memory-allocator/_index.md new file mode 100644 index 000000000..0faf2fcd4 --- /dev/null +++ b/content/learning-paths/cross-platform/dynamic-memory-allocator/_index.md @@ -0,0 +1,39 @@ +--- +armips: null +author_primary: David Spickett +layout: learningpathall +learning_objectives: +- Explain how dynamic memory allocation and the C heap works +- Write a simple dynamic memory allocator +- Explain some of the flaws and risks of heap allocation in general, and the specific + implementation you have studied +learning_path_main_page: 'yes' +minutes_to_complete: 120 +operatingsystems: +- Linux +prerequisites: +- Familiarity with C programming, with a good understanding of pointers. +- A Linux machine to run the example code. +skilllevels: Introductory +subjects: Performance and Architecture +armips: + - Cortex-A + - Neoverse +test_images: +- ubuntu:latest +test_link: null +test_maintenance: true +test_status: +- passed +title: Write a Dynamic Memory Allocator +tools_software_languages: +- C +- Coding +weight: 1 +who_is_this_for: This is an introductory topic for software developers learning about dynamic memory allocation for the first time, + and may have used malloc and free in C programming. It also provides a starting point to explore advanced memory allocation topics. +shared_path: true +shared_between: + - laptops-and-desktops + - embedded-systems +--- diff --git a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_next-steps.md b/content/learning-paths/cross-platform/dynamic-memory-allocator/_next-steps.md similarity index 88% rename from content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_next-steps.md rename to content/learning-paths/cross-platform/dynamic-memory-allocator/_next-steps.md index 9a6645c9c..a42a93bb7 100644 --- a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_next-steps.md +++ b/content/learning-paths/cross-platform/dynamic-memory-allocator/_next-steps.md @@ -1,7 +1,7 @@ --- next_step_guidance: -recommended_path: /learning-paths/PLACEHOLDER_CATEGORY/PLACEHOLDER_LEARNING_PATH/ +recommended_path: /learning-paths/servers-and-cloud-computing/exploiting-stack-buffer-overflow-aarch64/ further_reading: - resource: diff --git a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_review.md b/content/learning-paths/cross-platform/dynamic-memory-allocator/_review.md similarity index 95% rename from content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_review.md rename to content/learning-paths/cross-platform/dynamic-memory-allocator/_review.md index aa63ecc1a..2f21591b2 100644 --- a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_review.md +++ b/content/learning-paths/cross-platform/dynamic-memory-allocator/_review.md @@ -16,7 +16,7 @@ review: complexity of the dynamic allocator may change. Dynamic allocation is done using runtime calls, so the program can - react to what's needed at the time. Static alloation is decided ahead + react to what's needed at the time. Static allocation is decided ahead of time instead. Both types of allocation have the same memory constraints as the @@ -62,7 +62,7 @@ review: reduce the time taken to find a free range, and the overhead of recording the information about the ranges. - In our case, the performance of the heap would be ok to begin with. + In this case, the performance of the heap would be ok to begin with. As the program continues, more and more small ranges pile up. Leading to poorer performance later. diff --git a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_index.md b/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_index.md deleted file mode 100644 index e2fa6a7c0..000000000 --- a/content/learning-paths/laptops-and-desktops/dynamic-memory-allocator/_index.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -armips: null -author_primary: David Spickett -layout: learningpathall -learning_objectives: -- Explain how dynamic memory allocation and the C heap works. -- Write a simple dynamic memory allocator. -- Explain some of the flaws and risks of heap allocation in general, and the specific - implementation you have written. -learning_path_main_page: 'yes' -minutes_to_complete: 120 -operatingsystems: -- Linux -prerequisites: -- Familiarity with C programming, with a good understanding of pointers. -skilllevels: Introductory -subjects: Memory Allocation -test_images: -- ubuntu:latest -test_link: null -test_maintenance: true -test_status: -- passed -title: Writing a Dynamic Memory Allocator -tools_software_languages: -- C Programming -weight: 1 -who_is_this_for: Those learning about dynamic memory allocation for the first time, - who may have used C's malloc and free before. Also suitable for those looking for - a simple template from which to explore more advanced topics. ---- diff --git a/contributors.csv b/contributors.csv index 0683434a1..9043db3fa 100644 --- a/contributors.csv +++ b/contributors.csv @@ -16,4 +16,5 @@ Pranay Bakre,Arm,,,, Elham Harirpoush,Arm,,,, Frédéric -lefred- Descamps,OCI,,,,lefred.be Kristof Beyls,Arm,,,, +David Spickett,Arm,,,, Uma Ramalingam,Arm,uma-ramalingam,,,