Build all frameworks with docker

Update: Docker support has been deprecated.

Build all frameworks with docker

Building and serving the frameworks with docker allows us have a build process that works for everyone. Building all those frameworks isn't that easy. Some frameworks can't cope with the most recent node.js version, need a special java or a rust version. The docker build takes care of that without clobbering your machine with the prerequisites for building. The docker build runs on linux, OSX and Windows 10 with Hyper-V.

Although the test driver can run headless (or even with an x11 server) we run the test driver from the local machine such that the "normal" browser speed is measured.

Architectural overview

The docker container allows us to build all frameworks on a defined platform whilst supporting Windows 10 Professional, OSX and Linux as the host platform. For all frameworks the build process happens inside the docker container. There's even a headless chrome and the benchmarks test driver (webdriver-ts) that can perform basic checks inside the container. A node http-server publishes the bundled benchmark implementations and serves them available on port 8080.

The real benchmark run is the performed on the host machine with the system browser installed.

Note: Building locally

Building locally is still supported and might be preferred for benchmark implementation contributors.

Instructions

Building all implementations

1. Install dependencies on host machine npm install

2. Install webdriver-ts and webdriver-ts-results dependencies on host. Both will be executed on your host machine. npm run install-local

3. Make sure docker is running. The following command must not return an error: docker ps

4. Build the docker image [maybe one day pull image from registry] npm run docker-build-image

5. Start the docker container (it'll automatically create a docker volume named js-framework-benchmark) npm run docker-start

6. Build all frameworks with docker npm run docker-build-frameworks

Tips:

• If building all implementations fails you can fix the build for that implementation and restart building from that implementation with npm run docker-build-frameworks non-keyed/san

• The build installs dependencies with npm ci. If you want to use npm install use the docker-build-frameworks-non-ci script

Running from the local machine

1. Make sure that the docker container runs and check that the http-server is up. Opening http://localhost:8080/index.html must now show the index page for all implementations.

2. Run the benchmark from the local machine: npm run bench

3. Create the results file from the local machine: npm run results

4. The results file can now be opened

• on Linux: google-chrome webdriver-ts-results/table.html
• on Windows: webdriver-ts-results\table.html

Tips:

• run a single framework framework you want npm run bench keyed/vanillajs
• or use more options npm run bench -- keyed/vanillajs --benchmark 01_ --count 3
• Remember to update the results file after running a benchmark npm run results

Updating / Rebuilding a single implementation

1. Edit the implementation's file as you like on your local machine

2. Rebuild the framework within the docker container

• The easiest way is to call npm run docker-rebuild [keyed|non-keyed]/[framework]. This will delete the directory in the docker container, sync the source code, invoke npm install and npm run build-prod and perform some checks: It'll run all benchmarks for the implementation with minimal count and run a check whether the keyed categorization is correct. And finally it'll update the index.html for all implementations. The script npm run docker-rebuild-ci works the same way, but uses npm ci.

Get rid of it again

Stop the docker container npm run docker-stop Delete the volume docker volume rm js-framework-benchmark