Converting Examples To Go Workers

gobyexample.com (Worker Pools)

Absolutely nothing wrong with building out workers like this example. It is a great example. In fact they have many great examples for anything your curious about with the Go language. Let us see how we can replace it with go-workers though. Notice how it becomes more human readable? Initialize your workers, send to the workers, close and wait. Think about how you would organize each example differently in your own project. Also think about how you are going to bubble up errors from your workers. Use an ErrGroup and you will have to refactor the entire thing. Add another channel but for errors? go-workers wait method convienently returns an error bubbled up.

package main

import (
	"fmt"
	"time"
)

func main() {
	start := time.Now()
        const exampleNumJobs = 5

	exampleWorkerFunction := func(id int, jobs <-chan int, results chan<- int) {
		for j := range jobs {
			fmt.Println("worker", id, "started  job", j)
			time.Sleep(time.Second)
			fmt.Println("worker", id, "finished job", j)
			results <- j * 2
		}
	}

	jobs := make(chan int, exampleNumJobs)
	results := make(chan int, exampleNumJobs)

	for w := 1; w <= 3; w++ {
		go exampleWorkerFunction(w, jobs, results)
	}

	for j := 1; j <= exampleNumJobs; j++ {
		jobs <- j
	}
	close(jobs)

	for a := 1; a <= exampleNumJobs; a++ {
		<-results
	}

	exampleTimeToRun := time.Since(start).Milliseconds()
	fmt.Println("Example time to run", exampleTimeToRun, "ms")
}

Output:

worker 3 started  job 1
worker 2 started  job 3
worker 1 started  job 2
worker 1 finished job 2
worker 1 started  job 4
worker 3 finished job 1
worker 3 started  job 5
worker 2 finished job 3
worker 1 finished job 4
worker 3 finished job 5
Example time to run 2000 ms

Now the go-workers example (Since worker initialization is handled by the library we cannot give a worker an ID efficiently.)

package main

import (
	"context"
	"fmt"
	goworker "github.com/catmullet/go-workers"
	"time"
)

func main() {
	start := time.Now()
	const numJobsForGoWorkers = 5

	ctx := context.Background()
	worker := goworker.NewWorker(ctx, NewExampleWorker(), 3).Work()

	for j := 1; j <= numJobsForGoWorkers; j++ {
		worker.Send(j)
	}

	if err := worker.Close(); err != nil {
		fmt.Println(err)
	}

	goWorkersTimeToRun := time.Since(start).Milliseconds()
	fmt.Println("Go Workers time to run", goWorkersTimeToRun, "ms")
}

type ExampleWorker struct{}

func NewExampleWorker() *ExampleWorker {
	return &ExampleWorker{}
}

func (ew *ExampleWorker) Work(w *goworker.Worker, in interface{}) error {
	fmt.Println("worker", " started job", in)
	time.Sleep(time.Second)
	fmt.Println("worker", " finished job", in)
	return nil
}

Output:

worker started job 1
worker started job 2
worker started job 3
worker finished job 3
worker started job 4
worker finished job 2
worker started job 5
worker finished job 1
worker finished job 4
worker finished job 5
Go Workers time to run 2000 ms