Skip to content

Commit

Permalink
add basic feature docs, examples still WIP
Browse files Browse the repository at this point in the history
  • Loading branch information
sweikenb committed Sep 29, 2023
1 parent da76d8a commit a16eed1
Show file tree
Hide file tree
Showing 8 changed files with 306 additions and 1 deletion.
6 changes: 5 additions & 1 deletion docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,11 @@
- [Process Queue](features/process-queue.md)
- [Inter Process Communication (IPC)](features/ipc.md)

# Examples
# Troubleshoot

- [Common Pitfalls](common-pitfalls-and-workarounds.md)

# Real-World Examples

- [Async Workloads](examples/async-workloads.md)
- [Queued Parallel-Processing](examples/queued-processing.md)
Expand Down
45 changes: 45 additions & 0 deletions docs/src/common-pitfalls-and-workarounds.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Common Pitfalls and Workarounds

## Database Connections and File-Pointer

When working with database-connections, file-pointer or any other kind of ressource, you have to be careful when forking
processes.

A fork is a copy of the original thread that contains a copy of the memory too. The pointer and resources referenced by
the copied memory still belong to the main thread and when accessing them it wil cause errors and a lot of problems.

Fortunately, there are some simple workarounds:

1. Do not open any connection of pointer before you fork your process _(easy enough)_
2. If you need to open a connection or resource, close it and create a new instance inside the fork:

```php
use Sweikenb\Library\Pcntl\ProcessManager;

require __DIR__ . '/vendor/autoload.php';

$pm = new ProcessManager();

// get the connection
$connection = new Connection();

// load results
$results = $connection->getReults();

// close connection
$connection->close();
foreach ($results as $result) {
$pm->runProcess(function () use ($result) {
// close connection
$connection = new Connection();

// TODO process data

// update data
$connection->update($result);

// close connection
$connection->close();
});
}
```
2 changes: 2 additions & 0 deletions docs/src/examples/async-workloads.md
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
# Async Workloads

_TODO_
2 changes: 2 additions & 0 deletions docs/src/examples/ipc-examples.md
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
# IPC Examples

_TODO_
2 changes: 2 additions & 0 deletions docs/src/examples/queued-processing.md
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
# Queued Processing

_TODO_
74 changes: 74 additions & 0 deletions docs/src/features/ipc.md
Original file line number Diff line number Diff line change
@@ -1 +1,75 @@
# Inter Process Communication (IPC)

When working with threads, you might want to send data between the parent- and the child-process _(e.g. to update data
or return the result of an asynchronous workload)_.

In order to do so, the threads have a direct socket-connection which can be used to send messages with custom payloads.

## Topic and Payload

The **topic** is intended to be used as description, intention or routing of the message. Beside the fact that it must
be a string, the topic can be anything you like.

The **payload** on the other hand is intended to carry the actual data you want to transfer between the threads.
Please beware that you can only send payloads that are [serializable](https://www.php.net/serialize). Any kind of
file-pointer or resource _(e.g.
[database-connections](../common-pitfalls-and-workarounds.md#database-connections-and-file-pointer))_ will **NOT** work!

You might also want to refer to the [Common Pitfalls and Workarounds](../common-pitfalls-and-workarounds.md) section if
you run into trouble.

## Basic Usage

```php
use Sweikenb\Library\Pcntl\Api\ChildProcessInterface as ChildProcess;
use Sweikenb\Library\Pcntl\Api\ParentProcessInterface as ParentProcess;
use Sweikenb\Library\Pcntl\Factory\MessageFactory;
use Sweikenb\Library\Pcntl\ProcessManager;

require __DIR__ . '/vendor/autoload.php';

$pm = new ProcessManager();
$factory = new MessageFactory();

$child = $pm->runProcess(
function (ChildProcess $child, ParentProcess $parent) use ($factory) {
$messageFromParent = $parent->getNextMessage();
$parent->sendMessage(
$factory->create(
'hello parent',
[
'pid' => $child->getId(),
'lastMessage' => $messageFromParent->getTopic()
]
)
);
}
);

$child->sendMessage($factory->create('hello child', null));
$messageFromChild = $child->getNextMessage();

var_dump($messageFromChild);
```

This will output something like this:

```
class Sweikenb\Library\Pcntl\Model\Ipc\MessageModel#14 (2) {
private string $topic =>
string(12) "hello parent"
private mixed $payload =>
array(2) {
'pid' =>
int(54723)
'lastMessage' =>
string(11) "hello child"
}
}
Process finished with exit code 0
```

## Further Examples

You can find some real-world examples [here](../examples/ipc-examples.md).
145 changes: 145 additions & 0 deletions docs/src/features/process-manager.md
Original file line number Diff line number Diff line change
@@ -1 +1,146 @@
# Process Manager

The process manager is the core of this library and provides the basic threading functionality by utilizing the native
functions of the [pcntl](https://www.php.net/pcntl) and [posix](https://www.php.net/posix) modules.

## Basic Usage

```php
use Sweikenb\Library\Pcntl\Api\ChildProcessInterface as ChildProcess;
use Sweikenb\Library\Pcntl\Api\ParentProcessInterface as ParentProcess;
use Sweikenb\Library\Pcntl\Api\ProcessOutputInterface as Output;
use Sweikenb\Library\Pcntl\ProcessManager;

require __DIR__ . '/vendor/autoload.php';

$pm = new ProcessManager();

// inline callback
$pm->runProcess(fn() => sleep(3));

// example with all available callback parameters
$pm->runProcess(function (ChildProcess $child, ParentProcess $parent, Output $output) {
$output->stdout(sprintf("Parent PID: %s\n", $parent->getId()));
$output->stdout(sprintf("Child PID: %s\n", $child->getId()));
});

// return 'true' or 'null'/'void' to indicate that the execution was successful
$pm->runProcess(function () {
//...
return true; // this will exit the child process with exit-code `0`
});

// return 'false' to indicate that the execution failed
$pm->runProcess(function () {
//...
return false; // this will exit the child process with exit-code `1`
});

// prints the PID of the main process
var_dump($pm->getMainProcess()->getId());
```

### Wait for Children

If you want to continue synchronously after creating child-threads, simply call the `wait()`-method of the process
manager. By default, the method is called automatically at the end of each script-execution but this can be configured
using the `$autoWait`-[setting](#settings) of the `ProcessManager`.

```php
use Sweikenb\Library\Pcntl\ProcessManager;

require __DIR__ . '/vendor/autoload.php';

$pm = new ProcessManager();

// run in sync
// ...

// run the next three lines async
$pm->runProcess(fn() => sleep(3));
$pm->runProcess(fn() => sleep(1));
$pm->runProcess(fn() => sleep(2));

// wait for all threads to finish
$pm->wait();

// continue to run in sync from here
// ...
```

If you wat to know which children exited, you can provide a callback function for the `wait()`-method:

```php
// wait for all threads to finish
$pm->wait(function (int $status, int $pid) {
echo sprintf("The child with pid %s exited with status code %s\n", $pid, $status);
});
```

By default, the `wait()`-method will wait for **ALL** children to exit before it continues the programm. If you wish to
only wait for a specific children to exit, you can modify this behavior by returning `false` in the callback:

```php
$pm->wait(function (int $status, int $pid) {
if ($status === 1) {
// the child failed, lets stop waiting and continue with the programm-flow
return false;
}

// info
echo sprintf("The child with pid %s exited with status code %s\n", $pid, $status);

// continue to wait
return true;
});
```

Beside the callback in the wait()-method itself, there are also [thread-hooks](#thread-hooks) that can be used to get
notified when a
thread is created or finished.

### Thread-Hooks

You can register hooks that gets triggered during the lifetime of a thread. Note that you can register multiple hooks
for the same lifetime-event and that the callbacks are executed in the order of registration.

#### onThreadCreate

Registers a callback that gets called whenever a thread is created:

```php
$pm->onThreadCreate(function (ChildProcessInterface $child) {
echo sprintf("The child with pid %s was created.", $child->getId());
});
```

#### onThreadExit

Registers a callback that gets called whenever a thread exits:

```php
$pm->onThreadExit(function (int $status, int $pid) {
echo sprintf("The child with pid %s exited with status %s", $pid, $status);
});
```

Please note that the callback of the `wait()` method gets called BEFORE the lifecycle hooks.

## Settings

- `$autoWait`
- enables or disables the automatic wait at the end of the script
- default: `true` _RECOMMENDED!_
- `$propagateSignals`
- list of signals that should be propagated to the child-processes
- default: [`SIGTERM`, `SIGHUP`, `SIGALRM`, `SIGUSR1`, `SIGUSR2`]
- `$processFactory`
- factory instance that should be used to create the process models
- default: `Sweikenb\Library\Pcntl\Factory\ProcessFactory`
- `$processOutput`
- output instance that should be used as proxy for writing data to `STDOUT` and `STDERR`
- default: `Sweikenb\Library\Pcntl\ProcessOutput`

## Further Examples

You can find some real-world examples [here](../examples/async-workloads.md).
31 changes: 31 additions & 0 deletions docs/src/features/process-queue.md
Original file line number Diff line number Diff line change
@@ -1 +1,32 @@
# Process Queue

If you do not know how many threads you might need, but you want to limit the amount of threads that will be forked
simultaneously, you can use the `ProcessQueue` which internally ensures that your thread-limit is never exceeded.

## Basic Usage

```php
use Sweikenb\Library\Pcntl\ProcessQueue;

require __DIR__ . '/vendor/autoload.php';

// only 4 threads will be forked and further callbacks must wait until free slots are available
$maxThreads = 4;

$queue = new ProcessQueue($maxThreads);
for ($i = 0; $i < 100; $i++) {
$queue->addToQueue(fn() => sleep(3));
}
```

## Settings

- `$maxThreads`
- the maximum number of threads that might be forked _(min. `1`)_
- `$processManager`
- instance of the process manager to be used
- default: `Sweikenb\Library\Pcntl\ProcessManager`

## Further Examples

You can find some real-world examples [here](../examples/queued-processing.md).

0 comments on commit a16eed1

Please sign in to comment.