Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libsubprocess: do not spin on large lines #6281

Merged
merged 5 commits into from
Sep 25, 2024
Merged

libsubprocess: do not spin on large lines #6281

merged 5 commits into from
Sep 25, 2024

Commits on Sep 25, 2024

  1. libsubprocess: clarify some comments 

    Problem: Some comments are a bit unclear because the word "reactor"
was used in place of "watcher".

Update comments.
    @chu11
    chu11 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    2961d11 View commit details
    Browse the repository at this point in the history

  2. libsubprocess/test: add output count 

    Problem: It'd be nice to know how many times the output callback
is called, but that is not tracked.

Add an output count to the output cb and output its result in
diagnostics.
    @chu11
    chu11 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    23c21d5 View commit details
    Browse the repository at this point in the history

  3. libsubprocess: document corner case 

    Problem: The flux_subprocess_read_line() function may return
an incomplete line if the last output of the stream is not a line.
This is not documented.

Document this in subprocess.h.
    @chu11
    chu11 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    2063039 View commit details
    Browse the repository at this point in the history

  4. libsubprocess: do not spin on large line 

    Problem: libsubprocess can hang/spin if the output buffer is line
buffered and a line exceeds the current output buffer size.  The
buffer can never be emptied because output callbacks are never called
(i.e. the buffer never contains a line).

Solution: If output is line buffered and the buffer is full AND no line
exists, call the output callback for the user to get the current data.
flux_subprocess_read_line() and similar functions will return data that
is not a full line.

Fixes flux-framework#6262
    @chu11
    chu11 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    252dcd2 View commit details
    Browse the repository at this point in the history

  5. libsubprocess/test: cover line buffer overflow 

    Problem: There are no unit tests for when a single line exceeds
the size of an output buffer.

Add unit tests.
    @chu11
    chu11 committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    3302805 View commit details
    Browse the repository at this point in the history