Performance is a bit strange

[xstar2091]

I test Crow performance using the blowing code

#include <crow.h>

int main()
{
    crow::SimpleApp app;
    app.loglevel(crow::LogLevel::Warning);
    CROW_ROUTE(app, "/hello")
    ([]() {
        return "hello world";
    });
    app.port(18080)
      .multithreaded()
      .run();
    return 0;
}

Then run wrk
wrk -t12 -c400 -d30s http://127.0.0.1:18080/hello

The result shows

Running 30s test @ http://127.0.0.1:18080/hello
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.89ms    1.15ms  17.84ms   68.33%
    Req/Sec    17.55k     6.81k   42.22k    83.19%
  6298733 requests in 30.10s, 792.92MB read
Requests/sec: 209269.72
Transfer/sec:     26.34MB

I changed the code like blow

#include <crow.h>

int main()
{
    crow::SimpleApp app;
    app.loglevel(crow::LogLevel::Warning);
    CROW_ROUTE(app, "/hello")
    ([](const crow::request&, crow::response& res) {
        res.add_header("Content-Type", "plain/text");
        res.body = "hello world";
        res.end();
    });
    app.port(18080)
      .multithreaded()
      .run();
    return 0;
}

Then run the same wrk command
wrk -t12 -c400 -d30s http://127.0.0.1:18080/hello

The result shows
Running 30s test @ http://127.0.0.1:18080/hello
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    41.24ms    2.19ms  43.96ms   99.72%
    Req/Sec   802.78     85.53     1.20k    66.75%
  287830 requests in 30.04s, 43.42MB read
Requests/sec:   9581.56
Transfer/sec:      1.45MB

Why is there such a big difference in performance?

[gittiver]

There could be two reasons - assigning the strings or res.end().
Could you check with
CROW_ROUTE(app, "/hello")
([](const crow::request&) {
crow::response res;
res.add_header("Content-Type", "plain/text");
res.body = "hello world";
return res;
});

[dr3mro]

Why res.end() is so expensive?

[gittiver]

I don't know, if it is. Did you test it?

[dr3mro]

I will test and report back

…

On Mon, Aug 26, 2024, 10:00 AM gittiver ***@***.***> wrote: I don't know, if it is. Did you test it? — Reply to this email directly, view it on GitHub <#875 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJQHCMXL37RUPUQAP7WJSTZTLHCTAVCNFSM6AAAAABMZQIYUGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBZGQ3TSNBVHA> . You are receiving this because you commented.Message ID: ***@***.***>

[xstar2091]

There could be two reasons - assigning the strings or res.end(). Could you check with CROW_ROUTE(app, "/hello") ([](const crow::request&) { crow::response res; res.add_header("Content-Type", "plain/text"); res.body = "hello world"; return res; });

I test the following two codes:

CROW_ROUTE(app, "/hello")
([](const crow::request&) {
    crow::response res;
    res.add_header("Content-Type", "plain/text");
    res.body = "hello world";
    return res;
});

and

CROW_ROUTE(app, "/hello")
([](const crow::request&) {
    crow::json::wvalue x({{"zmessage", "Hello, World!"},
                          {"amessage", "Hello, World2!"}});
    return x;
});

The Request/Sec is just 9580

Running 30s test @ http://127.0.0.1:18080/hello
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    41.24ms    2.16ms  43.05ms   99.72%
    Req/Sec   802.88     79.81     1.29k    70.67%
  287816 requests in 30.04s, 57.42MB read
Requests/sec:   9580.85
Transfer/sec:      1.91MB

I have no idea about the test result.

[xstar2091]

There could be two reasons - assigning the strings or res.end(). Could you check with CROW_ROUTE(app, "/hello") ([](const crow::request&) { crow::response res; res.add_header("Content-Type", "plain/text"); res.body = "hello world"; return res; });

I did another test, use code

CROW_ROUTE(app, "/hello")
([](const crow::request&) {
    return "hello world";
});

The Requests/sec is 337253.80, the cpu usage is 485%
Use code

  CROW_ROUTE(app, "/hello")
    ([](const crow::request&) {
        crow::response res;
        res.add_header("Content-Type", "plain/text");
        res.body = "hello world";
        return res;
    });

The Requests/sec is 9586.31, the cpu usage is 25%

Is there some locks to block?

[gittiver]

For me, it looks like res.add_header(...) let the handler run much longer.
That's even true for the json object created and returned.
Could you repeat with

CROW_ROUTE(app, "/hello")
    ([](const crow::request&) {
        crow::response res;
        res.body = "hello world";
        return res;
    });

Just to get sure for the function.

[gittiver]

btw. what is the used hardware? In the logs should be visible how many concurrent accesses are possible.

[xstar2091]

For me, it looks like res.add_header(...) let the handler run much longer. That's even true for the json object created and returned. Could you repeat with

CROW_ROUTE(app, "/hello")
    ([](const crow::request&) {
        crow::response res;
        res.body = "hello world";
        return res;
    });

Just to get sure for the function.

That is. I test this code, and the Requests/sec is 334266.41.
As http protocol, not setting a header is not a good idea, right?

[xstar2091]

btw. what is the used hardware? In the logs should be visible how many concurrent accesses are possible.

I've set the logging level to Warning for the purpose of stress testing. The test results should have nothing to do with the hardware, I think.

[gittiver]

self_t& multithreaded()
{
return concurrency(std::thread::hardware_concurrency());
}

multithreaded is using the hardware_concurrency level, thats why I am asking.

[xstar2091]

btw. what is the used hardware? In the logs should be visible how many concurrent accesses are possible.

I have 8 concurrency threads. 11th Gen Intel® Core™ i7-1165G7 × 8, 40GiB RAM, Ubuntu 24.04