Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inter-process messaging issue #36

Closed
27medkamal opened this issue Oct 18, 2024 · 3 comments · Fixed by #37
Closed

Inter-process messaging issue #36

27medkamal opened this issue Oct 18, 2024 · 3 comments · Fixed by #37

Comments

@27medkamal
Copy link

First of all, thanks for venomous.

I might have found an issue that seems to uncover weird messaging behaviour between different processes. I managed to re-produce this with the examples repo

  1. Set up the repo according to the instructions.
  2. Run the following in iex -S mix run
Interactive Elixir (1.17.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Venomous.SnakeArgs.from_params(:my_module, :factorial, [5]) |> Venomous.python!()
120
iex(2)> Task.async(fn -> Venomous.SnakeArgs.from_params(:my_module, :factorial, [5]) |> Venomous.python!() end) |> Task.await()
120
iex(3)> Venomous.SnakeArgs.from_params(:my_module, :factorial, [5]) |> Venomous.python!()
{:killed, :normal}
iex(4)> Venomous.SnakeArgs.from_params(:my_module, :factorial, [5]) |> Venomous.python!()
120
iex(5)> Task.async(fn -> Venomous.SnakeArgs.from_params(:my_module, :factorial, [5]) |> Venomous.python!() end) |> Task.await()
120
iex(6)> flush()
{:EXIT, #Port<0.5>, :normal}
{:EXIT, #PID<0.197.0>, :normal}
:ok
iex(7)> Venomous.SnakeArgs.from_params(:my_module, :factorial, [5]) |> Venomous.python!()
120

The issue here seems to be in iex(3). It seems that it's receiving a message meant for the previously run Task.async process. Note that the one right after runs fine because the message was received.

In addition, if you run flush() between the 2 of them as in iex(6), the next call works fine which highlights the fact that it's related to messaging.

This can multiply massively if you have multiple async processes awaited such as the one in the docs. The only sensible way out of the following is to run flush() given the number of messages in the mailbox.

args = SnakeArgs.from_params(:time, :sleep, [0.5])
Enum.map(1..100, fn _ -> 
    Task.async(fn ->
        python!(args)
    end)
end) |> Task.await_many(5_000)
@RustySnek
Copy link
Owner

Hey, this is caused by Task.async() |> Task.await() leaving :EXIT signal in the mailbox after it finishes. Venomous traps exits and it so happens that whenever you call python later on these signals are being instantly captured and python process is killed. This can be solved by running :c.flush() At the end of Venomous return, however I'm not really sure if clearing out the whole process mailbox is a good idea, I will have to think about it.

@27medkamal
Copy link
Author

I’m definitely not suggesting clearing out the mailbox. I’m only using it to highlight the issue.

@RustySnek RustySnek linked a pull request Oct 18, 2024 that will close this issue
@RustySnek
Copy link
Owner

I decided to just filter out any :EXITs signals that happen prior to running receive block. This fixes it and shouldn't be destructive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants