chore: reorganise buffers behaviour

[hauleth]

No description provided.

[Ziinc]

the MemoryBuffer actually needs to get rewritten to how the v1 pipeline works, such that we insert directly into Broadway's internal queue. Having the separate buffer leads to situation where we have 2 sequential queues.

[hauleth]

Ok, so we can close it and I can checkout how to do that. The whole point of all these changes is to have behaviour callbacks calls in single place, so this will allow me to add telemetry in one place instead of tracing through whole codebase to see where I need add them, or to be wary of adding them to each implementation independently. So that is why you can see some of these appearing, as it is all preparation to Telemetry addition.

[Ziinc]

It's fine to merge this first, just wanted to give you more context. I'll have a final once over on Monday, but I like the api changes. We had the idea of a persistent disk-backed buffer previously to reduce risk of dropping events on node failure/shutdown, hence the pluggable design.