Merge pull request #572 from ArmDeveloperEcosystem/main

Merge to production

.wordlist.txt

@@ -1627,4 +1627,25 @@ Dawid
 rg
 pds
 WebApp
-AllowAnyCustom
+AllowAnyCustom
+Konstantinos
+Margaritis
+VectorCamp
+ASIMD
+enum
+solaris
+Spickett
+Allocator
+allocator
+pseudocode
+struct
+allocators
+tradeoff
+tradeoffs
+unallocated
+deallocate
+prepend
+initialise
+ptr
+LLSoftSecBook
+nodeSelector

content/learning-paths/cross-platform/_example-learning-path/contribute.md

@@ -55,7 +55,8 @@ After you have reviewed the new material using `hugo server` and there are no is
 You can now submit a GitHub pull request. 
 
 {{% notice Note%}}
-If you are new to GitHub, find a tutorial about how to create a pull request from a GitHub fork.
+If you are new to GitHub, please go to [GitHub's documentation](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request)
+to learn how to create a pull request from a GitHub fork.
 {{% /notice %}}
 
 Optionally, if you would like to add your new Learning Path content to the automated testing framework, follow the guidelines in the [Appendix: How to test your code](/learning-paths/cross-platform/_example-learning-path/appendix-3-test).

content/learning-paths/cross-platform/dynamic-memory-allocator/1_dynamic_memory_allocation.md

@@ -0,0 +1,189 @@
+---
+title: Dynamic memory allocation
+weight: 2
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## Dynamic vs. static memory allocation
+
+In this Learning Path you will learn how to implement dynamic memory allocation.
+If you have used the C programming language "heap" (`malloc`, `free`, etc.) before, that is one example
+of dynamic memory allocation.
+
+Dynamic memory allocation allows programs to allocate memory while they are running without knowing
+at build time how much memory they will need. In contrast, static
+memory allocation is used when the amount of memory is known at build time.
+
+The code sample below shows both dynamic and static memory allocation:
+
+```C
+#include <stdlib.h>
+
+void fn() {
+    // Static allocation
+    int a = 0;
+    // Dynamic allocation
+    int *b = malloc(sizeof(int));
+}
+```
+
+In the example above, the size and location of `a` is known
+when the program is built. The size of `b` is also known, but its location is not.
+
+Sometimes, memory may never be allocated, as in the pseudocode example below:
+
+```C
+int main(...) {
+    if (/*user has passed some argument*/) {
+        int *b = malloc(sizeof(int));
+    }
+}
+```
+
+The arguments passed to the program determine if memory is allocated or not. 
+
+## The C library malloc function
+
+The C standard library provides a special function
+[`malloc`](https://en.cppreference.com/w/c/memory/malloc). `m` for "memory",
+`alloc` for "allocate". This is used to ask for a suitably sized memory
+location while a program is running.
+
+```C
+void *malloc(size_t size);
+```
+
+The C library looks for a chunk of memory with size of at least `size`
+bytes in a large chunk of memory that it has reserved. For instance on Ubuntu
+Linux, this is done by GLIBC.
+
+The example at the top of the page is trivial of course. As it is we could just
+statically allocate both integers like this:
+
+```C
+void fn() {
+    int a, b = 0;
+}
+```
+
+Variables `a` and `b` work fine if they are not needed outside of the function. Or in other
+words, if the lifetime of the data is equal to that of the function.
+
+A more complex example shows when this is not the case, and the values
+live longer than the creating function.
+
+```C
+#include <stdlib.h>
+
+typedef struct Entry {
+    int data;
+    // NULL if end of list, next entry otherwise.
+    struct Entry* next;
+} Entry;
+
+void add_entry(Entry *entry, int data) {
+    // New entry, which becomes the end of the list.
+    Entry *new_entry = malloc(sizeof(Entry));
+    new_entry->data = data;
+    new_entry->next = NULL;
+
+    // Previous tail now points to the newly allocated entry.
+    entry->next = new_entry;
+}
+```
+
+What you see above is a struct `Entry` that defines a singly-linked-list entry.
+Singly meaning that you can go forward via `next`, but you cannot go backwards
+in the list. There is some data `data`, and each entry points to the next entry,
+`next`, assuming there is one (it will be `NULL` for the end of the list).
+
+`add_entry` makes a new entry and adds it to the end of the list.
+
+Think about how you would use these functions. You could start with some known
+size of list, like a global variable for the head (first entry)
+of our list.
+
+```C
+Entry head = {.data = 123, .next=NULL};
+```
+
+Now you want to add another `Entry` to this list at runtime. So you do not know
+ahead of time what it will contain, or if we indeed will add it or not. Where
+would you put that entry?
+
+* If it is another global variable, we would have to declare many empty `Entry`
+values and hope 
+
+{{% notice Other Allocation Techniques%}}
+Although in this specific case global variables aren't a good solution, there are
+cases where large sets of pre-allocated objects can be beneficial. For example,
+it provides a known upper bound of memory usage and makes the timing of each
+allocation predictable.
+
+However, these techniques are not covered in this Learning Path. It will
+however be useful to think about them after you have completed this learning
+path.
+{{% /notice %}}
+
+* If it is in a function's stack frame, that stack frame will be reclaimed and
+  modified by future functions, corrupting the new `Entry`.
+
+So you can see, dynamic memory allocation is required. Which is why the `add_entry`
+shown above calls `malloc`. The resulting pointer points to somewhere not in
+the program's global data section or in any function's stack space, but in the
+heap memory. It will stay in the heap until a call to `free` is made. 
+
+## The C library free function
+
+You cannot ask malloc for memory forever. Eventually the space behind the scenes
+will run out. You should give up your dynamic memory once it is not needed,
+using [`free`](https://en.cppreference.com/w/c/memory/free).
+
+```C
+void free(void *ptr);
+```
+
+You call `free` with a pointer previously returned by `malloc`, and this tells
+the heap that the memory is no longer needed. 
+
+{{% notice Undefined Behavior%}}
+You may wonder what happens if you don't pass the exact same pointer to `free` as
+`malloc` returned. The result varies as this is "undefined behavior".
+Which essentially means a large variety of unexpected things can happen.
+
+In practice, many allocators will tolerate this difference or reject it outright
+if it's not possible to do something sensible with the pointer.
+
+Remember, just because one allocator handles this a certain way, does not
+mean all allocators will be the same. Indeed, that same allocator may handle it differently for
+different allocations within the same program.
+{{% /notice %}}
+
+You can use `free` to remove an item from your linked list.
+
+```C
+void remove_entry(Entry* previous, Entry* entry) {
+    // NULL checks skipped for brevity.
+    previous->next = entry->next;
+    free(entry);
+}
+```
+
+`remove_entry` makes the previous entry point to the entry after the one we want
+to remove, so that the list skips over it. With `entry` now isolated we call
+`free` to give up the memory it occupies.
+
+```text
+----- List ------ | - Heap --
+[A] -> [B] -> [C] | [A][B][C]
+                  |
+[A]    [B]    [C] | [A][B][C]
+ |-------------^  |
+                  |
+[A]---------->[C] | [A]   [C]
+```
+
+That covers the high level how and why of using `malloc` and `free`, next you will
+see a possible implementation of a dynamic memory allocator.

content/learning-paths/cross-platform/dynamic-memory-allocator/2_designing_a_dynamic_memory_allocator.md

@@ -0,0 +1,168 @@
+---
+title: Design a dynamic memory allocator
+weight: 3
+
+### FIXED, DO NOT MODIFY
+layout: learningpathall
+---
+
+## High level design
+
+To begin with, decide which functions your memory allocator will provide. We
+have described `malloc` and `free`, there are more provided by the
+[C library](https://en.cppreference.com/w/c/memory).
+
+This will assume you just need `malloc` and `free`. The new implementations will
+be called `simple_malloc` and `simple_free`. Start with just two functions and write
+out their behaviors.
+
+The first function is `simple_malloc` and it will: 
+* Take a size in bytes as a parameter
+* Try to allocate the requested memory
+* Return a pointer to that memory or return a NULL pointer if the memory cannot be allocated
+
+The second function is `simple_free` and it will: 
+* Take a pointer to some previously allocated memory as a parameter
+* Mark that memory as available for future allocations
+
+From this you can see that you will need:
+* Some large chunk of memory, the "backing storage".
+* A way to mark parts of that memory as allocated, or available for allocation
+
+## Backing storage
+
+The memory can come from many sources. It can even change size throughout the
+program's execution if you wish. For your allocator you can keep it simple.
+
+A single, statically allocated global array of bytes will be your backing
+storage. You can do dynamic allocation of parts of a statically allocated
+piece of memory.
+
+```C
+#define STORAGE_SIZE 4096
+static char storage[STORAGE_SIZE];
+```
+
+## Record keeping
+
+This backing memory needs to be annotated somehow to record what has been
+allocated so far. There are many ways to do this. Te biggest choice
+is whether to store these records in the heap itself or outside of it.
+
+The easiest way is to put the records in the heap.
+
+What should be in the records? Think about the question the caller is asking.
+Can you give me a pointer to an area of memory of at least this size?
+
+For this you will need to know:
+* The ranges of the backing storage that have already been allocated
+* The size of each section, both free and allocated
+
+Where a "range" a pointer to a location, a size in bytes and a boolean to say
+whether the range is free or allocated. So a range from 0x123 of 345 bytes,
+that has been allocated would be:
+
+```text
+start: 0x123 size: 345 allocated: true
+```
+
+For the initial state of a heap of size `N`, you will have one range of
+unallocated memory.
+
+```text
+Pointer: 0x0 Size: N Allocated: False
+```
+
+When an allocation is made you will split this free range into 2 ranges. The
+first part the new allocation, the second the remaining free space. If 4 bytes
+were to be allocated:
+
+```text
+Pointer: 0x0 Size: 4   Allocated: True
+Pointer: 0x4 Size: N-4 Allocated: False
+```
+
+The next time you need to allocate, you will walk these ranges until you find
+one with enough free space, and repeat the splitting process.
+
+The walk works like this. Starting from the first range, add the size of that
+range to the address of that range. This new address is the start of the next
+range. Repeat until the resulting address is beyond the end of the heap.
+
+```text
+range = 0x0;
+
+Pointer: 0x0 Size: 4   Allocated: False
+
+range = 0x0 + 4 = 0x4;
+
+Pointer: 0x4 Size: N-4 Allocated: False
+
+range = 0x4 + (N-4) = 1 beyond the end of the heap, so the walk is finished.
+```
+
+`simple_free` uses the pointer given to it to find the range it needs to deallocate.
+Let's say the 4 byte allocation was freed:
+
+```text
+Pointer: 0x0 Size: 4   Allocated: False
+Pointer: 0x4 Size: N-4 Allocated: False
+```
+
+Since `simple_free` gets a pointer directly to the allocation you know exactly which
+range to modify. The only change made is to the boolean which marks it as
+allocated or not. The location and size of the range stay the same.
+
+{{% notice Merging Free Ranges%}}
+The allocator presented here does not merge free ranges like the 2 above. This
+is a deliberate limitation and addressing this is discussed later.
+{{% /notice %}}
+
+## Record storage
+
+You will keep these records in the heap which means using some of the allocated space
+for them on top of the allocation itself.
+
+The simplest way to do this is to prepend each allocation with the range
+information. This way you can skip from the start of one range to another with
+ease.
+
+```text
+0x00: [ptr, size, allocated] <-- The range information
+0x08: <...>                  <-- The pointer malloc returns
+0x10: [ptr, size, allocated] <-- Information about the second range
+<...and so on until the end of the heap...>
+```
+
+Pointers returned by `simple_malloc` are offset to just beyond the range information.
+When `simple_free` receives a pointer, it can get to the range information by
+subtracting the size of that information from the pointer. Using the example
+above:
+
+```text
+simple_free(my_ptr);
+
+0x00: [ptr, size, allocated] <-- my_ptr - sizeof(range information)
+0x08: <...>                  <-- my_ptr
+```
+
+{{% notice Data Alignment%}}
+When an allocator needs to produce addresses with a specific alignment, the
+calculations above must be adjusted. The allocator presented here does not
+concern itself with alignment, which is why it can do a simple subtraction.
+{{% /notice %}}
+
+## Running out of space
+
+The final thing an allocator must do is realize it has run out of space. This is
+simply achieved by knowing the bounds of the backing storage.
+
+```C
+#define STORAGE_SIZE 4096
+static char storage[STORAGE_SIZE];
+// If our search reaches this point, there is no free space to allocate.
+static const char *storage_end = storage + STORAGE_SIZE;
+```
+
+If you are walking the heap and the start of the next range would be greater
+than or equal to `storage_end`, you have run out of memory to allocate.