PooledStackAllocator helps improve performance of thread creating

[Coldwings]

In typical scenarios, we utilize photon::thread_pool primarily due to the efficiency gains from reusing existing photon threads rather than creating new ones.

From my observation, the most resource-intensive aspect of creating a photon thread is the allocation of its stack memory. It is notably more straightforward and offers better performance to reuse pre-allocated stack memory directly compared to reusing it through a thread pool mechanism.

Although the current IOAlloc toolset isn't compatible with stack allocation operations that cannot rely on photon thread utilities, I have developed a thread-local stack pool, enabling the practical reuse of already allocated stacks.

In our tests, the photon::WorkPool utilizing a pooled stack allocator outperforms the version using a traditional ThreadPool, and both these approaches significantly surpass direct thread creation in terms of performance.

[lihuiba]

Can we set capacity = 0 instead?

[beef9999]

Why the pooled stack allocator is faster than the thread pool?

[Coldwings]

Why the pooled stack allocator is faster than the thread pool?

The most heavy step of creating new photon thread is stack allocation, both thread pool and pooled stack allocator try to reuse allocated stack, but in different way.

In ThreadPool, all photon threads will not be put into DONE state when task finish, but keep in a indentity-pool to wait for reuse. The thread state is always kept and have to carefully deal with. It is ok for using ThreadPool in single vcpu. but using over multple-vcpus is quite stupid. Every step with locks and cross-vcpu notifications are always slow and too heavy.

The pooled-based allocator kept only stack memory allocation, never keep thread state at all. That makes it do not need to deal with locks and thread states, just a simple thread-local allocator. That is much simpler and efficient in tests. All thread create and die have no cross-vcpu related works.

Though, if workload always create new thread from one vcpu and then migrate to another, and let photon thread finished in vcpus that never create new threads, the thread local pool will never helps. (For example: always thread create in vcpu-A, then migrate to B, when work done, it will return stack to PooledStackAllocator in B. if B never create new threads, those stacks will never be reused.)