Lambda producer should be run in the main thread

[kapcod]

Currently, if the lambda provided instead of the queue/array/enumerable it's executed in one of the threads.
As far as I can see, the only reason to provide the producing lambda separately from the processing block is to separate the producer to run in the main thread.
Example (from real life, but simplified):
suppose user.generate_update takes 0.1s and MUST happen in the main thread due to DB connections limit
But send_update_api(params) takes 1s and so I want to parallelize the net calls in this case, so my idea for the code would be:

Parallel.each( -> { user = users.shift; user&.generate_update || Parallel::Stop }, in_threads: 10) do |params|
  send_update_api(params)
end

But the generate_update also runs in the threads and so I need to use another implementation, such as FixedThreadPool, or write my own solution.

[grosser]

yeah, sounds good if you can make a PR that's not super complicated I'd merge it

…

On Wed, Jun 9, 2021 at 1:12 AM Ilya ***@***.***> wrote: Currently, if the lambda provided instead of the queue/array/enumerable it's executed in one of the threads. As far as I can see, the only reason to provide the lambda separately from the processing block is to separate the producer to run in the main thread. Example (from real life, but simplified): suppose user.generate_update takes 0.1s and MUST happen in the main thread due to DB connections limit But send_update_api(params) takes 1s and so I want to parallelize the net calls in this case, so my idea for the code would be: Parallel.each( -> { user = users.shift; user&.generate_update || Parallel::Stop }, in_threads: 10) do |params| send_update_api(params) end But the generate_update also runs in the threads and so I need to use another implementation, such as FixedThreadPool <https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/thread_pools.md>, or write my own solution. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#296>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACYZZQEHAT5F6ILSIZY2TTR4PADANCNFSM46LQNXDA> .