Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: thread autoscaling #1266

Open
wants to merge 125 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
fe1158f
Decouple workers.
Alliballibaba2 Nov 1, 2024
ad34140
Moves code to separate file.
Alliballibaba2 Nov 1, 2024
89b211d
Cleans up the exponential backoff.
Alliballibaba2 Nov 2, 2024
7d2ab8c
Initial working implementation.
Alliballibaba2 Nov 2, 2024
f7e7d41
Refactors php threads to take callbacks.
Alliballibaba2 Nov 2, 2024
c03c59b
Cleanup.
Alliballibaba2 Nov 2, 2024
a9857dc
Cleanup.
Alliballibaba2 Nov 2, 2024
bac9555
Cleanup.
Alliballibaba2 Nov 2, 2024
a2f8d59
Cleanup.
Alliballibaba2 Nov 2, 2024
279924c
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 3, 2024
0825453
Adjusts watcher logic.
Alliballibaba2 Nov 3, 2024
17d5cbe
Adjusts the watcher logic.
Alliballibaba2 Nov 3, 2024
09e0ca6
Fix opcache_reset race condition.
Alliballibaba2 Nov 4, 2024
a726a2c
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 4, 2024
7f13ada
Fixing merge conflicts and formatting.
Alliballibaba2 Nov 4, 2024
13fb4bb
Prevents overlapping of TSRM reservation and script execution.
Alliballibaba2 Nov 5, 2024
a8a00c8
Adjustments as suggested by @dunglas.
Alliballibaba2 Nov 5, 2024
b4dd138
Adds error assertions.
Alliballibaba2 Nov 5, 2024
03f98fa
Adds comments.
Alliballibaba2 Nov 5, 2024
e52dd0f
Removes logs and explicitly compares to C.false.
Alliballibaba2 Nov 5, 2024
cd98e33
Resets check.
Alliballibaba2 Nov 5, 2024
4e2a2c6
Adds cast for safety.
Alliballibaba2 Nov 5, 2024
c51eb93
Fixes waitgroup overflow.
Alliballibaba2 Nov 5, 2024
89d8e26
Resolves waitgroup race condition on startup.
Alliballibaba2 Nov 6, 2024
3587243
Moves worker request logic to worker.go.
Alliballibaba2 Nov 7, 2024
ec32f0c
Removes defer.
Alliballibaba2 Nov 7, 2024
4e35698
Removes call from go to c.
Alliballibaba2 Nov 11, 2024
740fac7
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 15, 2024
8a272cb
Fixes merge conflict.
Alliballibaba2 Nov 15, 2024
ecce5d5
Adds fibers test back in.
Alliballibaba2 Nov 15, 2024
06ebd67
Refactors new thread loop approach.
Alliballibaba2 Nov 15, 2024
c811f4a
Removes redundant check.
Alliballibaba2 Nov 16, 2024
6bd047a
Adds compareAndSwap.
Alliballibaba2 Nov 16, 2024
55ad8ba
Refactor: removes global waitgroups and uses a 'thread state' abstrac…
Alliballibaba2 Nov 17, 2024
3ffbe06
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Nov 17, 2024
01ed92b
Removes unnecessary method.
Alliballibaba2 Nov 17, 2024
790cccc
Updates comment.
Alliballibaba2 Nov 17, 2024
0dd2605
Removes unnecessary booleans.
Alliballibaba2 Nov 18, 2024
60a66b4
test
Alliballibaba2 Nov 24, 2024
4719fa8
First state machine steps.
Alliballibaba2 Nov 25, 2024
f72e8cb
Splits threads.
Alliballibaba2 Dec 6, 2024
d20e706
Minimal working implementation with broken tests.
Alliballibaba2 Dec 6, 2024
6747d15
Fixes tests.
Alliballibaba2 Dec 6, 2024
54dc267
Refactoring.
Alliballibaba2 Dec 7, 2024
660efcd
Merge branch 'main' into refactor/start-worker-threads-directly
Alliballibaba2 Dec 7, 2024
6214754
Fixes merge conflicts.
Alliballibaba2 Dec 7, 2024
00eb834
Formatting
Alliballibaba2 Dec 7, 2024
02b73b1
C formatting.
Alliballibaba2 Dec 7, 2024
421904e
More cleanup.
Alliballibaba2 Dec 7, 2024
cca2a00
Allows for clean state transitions.
Alliballibaba2 Dec 7, 2024
ec8aeb7
Adds state tests.
Alliballibaba2 Dec 7, 2024
b598bd3
Adds support for thread transitioning.
Alliballibaba2 Dec 7, 2024
66cedbd
Fixes the testdata path.
Alliballibaba2 Dec 7, 2024
06af5d5
Formatting.
Alliballibaba2 Dec 7, 2024
71c16bc
Allows transitioning back to inactive state.
Alliballibaba2 Dec 7, 2024
5095342
Fixes go linting.
Alliballibaba2 Dec 7, 2024
4b18059
Formatting.
Alliballibaba2 Dec 7, 2024
15429d9
Removes duplication.
Alliballibaba2 Dec 7, 2024
c080608
Applies suggestions by @dunglas
Alliballibaba2 Dec 7, 2024
a166b87
Removes redundant check.
Alliballibaba2 Dec 7, 2024
9491e6b
Locks the handler on restart.
Alliballibaba2 Dec 7, 2024
e795c86
Removes unnecessary log.
Alliballibaba2 Dec 8, 2024
68fa124
Adds frankenphp admin api.
Alliballibaba2 Dec 8, 2024
b6cbfae
Allows booting threads at runtime.
Alliballibaba2 Dec 8, 2024
f185279
Adds proper admin status codes and tests.
Alliballibaba2 Dec 8, 2024
ea0a4fe
Makes config smaller.
Alliballibaba2 Dec 8, 2024
fcb5f8c
Adds max threads option and debug status.
Alliballibaba2 Dec 8, 2024
a43ecbe
Adds test with debug message.
Alliballibaba2 Dec 8, 2024
b117bff
Formatting and comments.
Alliballibaba2 Dec 8, 2024
ef1bd0d
Changes Unpin() logic as suggested by @withinboredom
Alliballibaba2 Dec 9, 2024
8cd9061
Allows scaling regular threads.
Alliballibaba2 Dec 9, 2024
9e8d8f0
Only allows POST requests.
Alliballibaba2 Dec 9, 2024
a8a4545
Adds suggestions by @dunglas and resolves TODO.
Alliballibaba2 Dec 10, 2024
23a6362
Makes restarts fully safe.
Alliballibaba2 Dec 10, 2024
62224bf
Merge branch 'refactor/start-worker-threads-directly' into feat/worke…
Alliballibaba2 Dec 10, 2024
18e3e58
Will make the initial startup fail even if the watcher is enabled (as…
Alliballibaba2 Dec 10, 2024
3672c60
Also adds compareAndSwap to the test.
Alliballibaba2 Dec 10, 2024
38f87b7
Adds comment.
Alliballibaba2 Dec 10, 2024
96265d3
Merge branch 'refactor/start-worker-threads-directly' into feat/worke…
Alliballibaba2 Dec 10, 2024
d97ebfe
Prevents panic on initial watcher startup.
Alliballibaba2 Dec 10, 2024
950a509
Merge branch 'refactor/start-worker-threads-directly' into feat/worke…
Alliballibaba2 Dec 11, 2024
5f1ec1f
Cleans up admin endpoints.
Alliballibaba2 Dec 11, 2024
7c61dfa
Fixes admin test.
Alliballibaba2 Dec 11, 2024
2af993e
Boots a thread in a test.
Alliballibaba2 Dec 11, 2024
547139f
Sets more explicit max_threads.
Alliballibaba2 Dec 11, 2024
c8bf1ec
Adjusts naming.
Alliballibaba2 Dec 12, 2024
7f2b94e
Adds docs.
Alliballibaba2 Dec 12, 2024
df78254
Changes logic to actually terminate the thread.
Alliballibaba2 Dec 12, 2024
ec0bc0f
Removes the test's randomness.
Alliballibaba2 Dec 13, 2024
8f10407
Adds comments.
Alliballibaba2 Dec 13, 2024
91c324d
Adds comments.
Alliballibaba2 Dec 13, 2024
ff06bd7
Scaling v1.
Alliballibaba2 Dec 15, 2024
50ba106
Scaling v2.
Alliballibaba2 Dec 15, 2024
bfe3de1
Allows regular thread scaling.
Alliballibaba2 Dec 15, 2024
e9f62b9
Refactors wait-time.
Alliballibaba2 Dec 15, 2024
21949dd
Explicitly requires setting max_threads.
Alliballibaba2 Dec 15, 2024
39a7fc9
Removes redundant check.
Alliballibaba2 Dec 15, 2024
442a558
Removes unnecessary import.
Alliballibaba2 Dec 15, 2024
c213fc9
Records clock time.
Alliballibaba2 Dec 17, 2024
58daaa5
Saves CPU metrics of last 100 requests.
Alliballibaba2 Dec 17, 2024
14925f6
Integrates CPU tracking.
Alliballibaba2 Dec 18, 2024
d408bdd
Replaces clock with probing.
Alliballibaba2 Dec 19, 2024
8fc3293
fmt.
Alliballibaba2 Dec 19, 2024
0314247
Adds autoscale tests.
Alliballibaba2 Dec 19, 2024
bd4af11
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Dec 19, 2024
dc10546
Merges main.
Alliballibaba2 Dec 19, 2024
3b9f577
Fixes alpine (probably)
Alliballibaba2 Dec 19, 2024
790ce4e
Fixes alpine (definitely)
Alliballibaba2 Dec 19, 2024
29de62a
go fmt
Alliballibaba2 Dec 19, 2024
b447412
Removes unnecessary 'isProtected'
Alliballibaba2 Dec 19, 2024
6fa90d6
Adds perf tests.
Alliballibaba2 Dec 20, 2024
3bd7c76
Adds request status message to thread debug status.
Alliballibaba2 Dec 21, 2024
45cd915
Adjusts performance tests.
Alliballibaba2 Dec 21, 2024
af40470
Adds an exponential backoff on request overflow.
Alliballibaba2 Dec 21, 2024
d5e8f86
Merge branch 'main' into feat/auto-scale-clock-time
Alliballibaba2 Dec 21, 2024
c7acb25
changes dir.
Alliballibaba2 Dec 21, 2024
8c22cbf
Linting and formatting.
Alliballibaba2 Dec 22, 2024
745b29b
Linting and formatting.
Alliballibaba2 Dec 22, 2024
68ae2e4
Adds explicit scaling tests.
Alliballibaba2 Dec 22, 2024
09a5caf
Adjusts perf tests.
Alliballibaba2 Dec 22, 2024
3cfcb11
Uses different worker in removal test.
Alliballibaba2 Dec 22, 2024
cbe45fc
More formatting fixes.
Alliballibaba2 Dec 22, 2024
1d8e973
Replaces inline errors and adjusts comments.
Alliballibaba2 Dec 22, 2024
bf48b14
Formatting.
Alliballibaba2 Dec 22, 2024
4f0cc8a
Formatting.
Alliballibaba2 Dec 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 133 additions & 0 deletions caddy/admin.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
package caddy

import (
"fmt"
"github.com/caddyserver/caddy/v2"
"github.com/dunglas/frankenphp"
"net/http"
"strconv"
"strings"
)

type FrankenPHPAdmin struct{}

// if the id starts with "admin.api" the module will register AdminRoutes via module.Routes()
func (FrankenPHPAdmin) CaddyModule() caddy.ModuleInfo {
return caddy.ModuleInfo{
ID: "admin.api.frankenphp",
New: func() caddy.Module { return new(FrankenPHPAdmin) },
}
}

func (admin FrankenPHPAdmin) Routes() []caddy.AdminRoute {
return []caddy.AdminRoute{
{
Pattern: "/frankenphp/workers/restart",
Handler: caddy.AdminHandlerFunc(admin.restartWorkers),
},
{
Pattern: "/frankenphp/threads",
Handler: caddy.AdminHandlerFunc(admin.threads),
},
}
}

func (admin *FrankenPHPAdmin) restartWorkers(w http.ResponseWriter, r *http.Request) error {
if r.Method != http.MethodPost {
return admin.error(http.StatusMethodNotAllowed, fmt.Errorf("method not allowed"))
}

frankenphp.RestartWorkers()
caddy.Log().Info("workers restarted from admin api")
admin.success(w, "workers restarted successfully\n")

return nil
}

func (admin *FrankenPHPAdmin) threads(w http.ResponseWriter, r *http.Request) error {
if r.Method == http.MethodPut {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: you could use a switch here.

return admin.changeThreads(w, r, admin.getCountFromRequest(r))
}
if r.Method == http.MethodDelete {
return admin.changeThreads(w, r, -admin.getCountFromRequest(r))
}
if r.Method == http.MethodGet {
return admin.success(w, frankenphp.ThreadDebugStatus())
}

return admin.error(http.StatusMethodNotAllowed, fmt.Errorf("method not allowed, try: GET,PUT,DELETE"))
}

func (admin *FrankenPHPAdmin) changeThreads(w http.ResponseWriter, r *http.Request, count int) error {
if !r.URL.Query().Has("worker") {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: you could store the result of Query() in a variable to prevent parsing the query two times.

You could even directly get the value and check if it is the zero value here.

return admin.changeRegularThreads(w, count)
}
workerFilename := admin.getWorkerBySuffix(r.URL.Query().Get("worker"))

return admin.changeWorkerThreads(w, count, workerFilename)
}

func (admin *FrankenPHPAdmin) changeWorkerThreads(w http.ResponseWriter, num int, workerFilename string) error {
method := frankenphp.AddWorkerThread
if num < 0 {
num = -num
method = frankenphp.RemoveWorkerThread
}
message := ""
for i := 0; i < num; i++ {
threadCount, err := method(workerFilename)
if err != nil {
return admin.error(http.StatusBadRequest, err)
}
message = fmt.Sprintf("New thread count: %d %s\n", threadCount, workerFilename)
}
return admin.success(w, message)
}

func (admin *FrankenPHPAdmin) changeRegularThreads(w http.ResponseWriter, num int) error {
method := frankenphp.AddRegularThread
if num < 0 {
num = -num
method = frankenphp.RemoveRegularThread
}
message := ""
for i := 0; i < num; i++ {
threadCount, err := method()
if err != nil {
return admin.error(http.StatusBadRequest, err)
}
message = fmt.Sprintf("New thread count: %d Regular Threads\n", threadCount)
}
return admin.success(w, message)
}

func (admin *FrankenPHPAdmin) success(w http.ResponseWriter, message string) error {
w.WriteHeader(http.StatusOK)
_, err := w.Write([]byte(message))
return err
}

func (admin *FrankenPHPAdmin) error(statusCode int, err error) error {
return caddy.APIError{HTTPStatus: statusCode, Err: err}
}

func (admin *FrankenPHPAdmin) getCountFromRequest(r *http.Request) int {
value := r.URL.Query().Get("count")
if value == "" {
return 1
}
i, err := strconv.Atoi(value)
if err != nil {
return 1
}
return i
}

func (admin *FrankenPHPAdmin) getWorkerBySuffix(pattern string) string {
for _, workerFilename := range frankenphp.WorkerFileNames() {
if strings.HasSuffix(workerFilename, pattern) {
return workerFilename
}
}
return ""
}
Loading
Loading