How it works
This library works differently from other Python libraries to handle differences in memory management.
In this library, General types have ownership of their respective types. For example, instances created with the Config constructor have ownership of the Config object in Rust.
In addition, there is a separate type named ConfigRef, which acts as a "Container Object" that holds a reference to the Config type.
Most of the APIs in rraft-py handle both "General types" and "Container Object types," but some APIs require only types with ownership. You can find information about this in rraft.pyi.
It is important to note that passing a reference pointing an invalid object in Python to rraft-py's API can cause a Segmentation fault that is difficult to debug.
Understanding Rust's ownership concept can help you avoid such problems.
This section explain how RefMutOwner and RefMutContainer types work, focusing on the events that occur when a few lines of example code are executed in Python. I will use the Config type as an example to illustrate this concept.
In the Python code, an object of Config type is created, and three references are generated from that object. The Config type is owned by the PyConfig type, and the reference types are owned by the PyConfigRef type.
The references to PyConfigRef are stored in a hash table, which is owned by PyConfig.
The ownership of Config lies with PyConfig, and when cfg is dropped, it invalidates all references by traversing the hash table.
In this diagram, the white arrows represent strong references that ensure atomic access, while the red arrows represent weak cross-references that also guarantee atomic access.
In the previous section, I assumed that references are managed only in Rust. However, the issue of how to handle references is actually more complex. We can categorize various smart pointer types used internally by pyo3 for reference management into two groups.
This slide explain the smart pointers provided from pyo3 for reference management and how RefMutContainer extends them.
The diagram in this section represents the relationship between the smart pointer types used by pyo3 for reference management, based on my understanding.
pyo3 provides both smart pointer types bound to the Global Interpreter Lock (GIL) for memory management and pointer types independent of the GIL.
The smart pointer types bound to the GIL, which are located at the top, are PyRef and PyRefMut. They are wrappers around PyCell, which internally contains a Rust type compatible with C named PyTypeObject. As the name suggests, it provides internal mutability pattern similar to RefCell.
The pointer types independent of the GIL are represented by Py at the bottom. Py<T> may resemble Arc, but it is implemented in pyo3 using GILPool and operates by acquiring the GIL and creating data on the Python heap.
Here, a GIL-independent pointer implies that the lifetime of this reference can be longer than the lifetime of the GIL. It is designed to make the pyo3 API more flexible.
The pymethods are designed to accept only reference types as arguments, allowing both PyRef types at the top and Py types at the bottom of the image.
Similarly, rraft-py implements bindings for both types of references for each reference type. Since I'm not expert about the internal workings of pyo3, I recommend referring to the pyo3 documentation and source code for more details.
First, we write a Python type that implements the methods required by Storage Trait and name it LMDBStorageCore type. After creating the storage, we pass it as an argument when creating a Raft object.
Next, let's assume that we obtain logs through the method called get_logs and then call a method named term on the logs.
Internally, the term method calls a method named first_index at the raft-rs level, which is implemented in Python. If the first call to first_index returns None, it is a situation where an error should be thrown.
In this case, we have written Python bindings for the StoreError and Unavailable error types and throw them accordingly.
Then, in raft-rs, when catching this error, if it is either a CompactionError or an Unavailable error, it returns an error. In other types of errors, it panics and forcefully terminates the program.
Therefore, if the Python bindings for these two errors are not written here, the term will always be called, and when an error occurs, it will panic as mentioned.
If term is executed successfully, it returns a u64 value converted to a Python's int. If not, as mentioned, the StoreError returned by term will be thrown back to the Python side, allowing Python users to receive and handle this error.
