docs: Start documenting testing section

docs/source/index.rst

@@ -15,6 +15,12 @@
 
  core/data-structures/index
 
+.. toctree::
+ :caption: Tests
+ :hidden:
+
+ tests/index
+
 Welcome to the php-src documentation!
 
 .. warning::

docs/source/tests/index.rst

@@ -0,0 +1,22 @@
+#################
+ Testing
+#################
+
+.. toctree::
+ :hidden:
+
+ running-tests
+ writing-tests
+
+This section provides an overview of the testing infrastructure used for php-src.
+
+*******
+ TODOS
+*******
+
+- Talk about bless-tests
+- Writing conflict free tests
+- Describe all sections from https://qa.php.net/phpt_details.php
+- How to invoke the test runner directly
+- Files generated when a test fails
+- Invoke GDB directly via the generated ``.sh`` file.

docs/source/tests/phpt-sections/file.rst

@@ -0,0 +1,19 @@
+#################
+ --FILE--
+#################
+
+The most common way to provide the script of the test being tested.
+The script within the ``--FILE--`` section should be wrapped by an opening *and* closing PHP tag.
+As this allows to execute the ``.phpt`` directly without invoking the test runner.
+
+*********
+ Example
+*********
+
+.. code:: php
+
+ --FILE--
+ <?php
+ $r = 20 + 36;
+ var_dump($r);
+ ?>

docs/source/tests/phpt-sections/test.rst

@@ -0,0 +1,16 @@
+#################
+ --TEST--
+#################
+
+The mandatory ``--TEST--`` section that must be at the beginning of a PHPT.
+
+It should contain the title of the test as plain text on a single line.
+
+*********
+ Example
+*********
+
+.. code:: plaintext
+
+ --TEST--
+ Test filter_input() with GET and POST data.

docs/source/tests/running-tests.rst

@@ -0,0 +1,43 @@
+########################
+ Running the test suite
+########################
+
+php-src has a custom test runner that is the ``run-tests.php`` file at the root of the repository.
+
+There are two ways of executing the test runner, either via the makefile, or by invoking the test runner directly.
+
+******************
+ Via the makefile
+******************
+
+After having compiled PHP a prompt to execute ``make test`` appears,
+doing so will run all the tests (a bit less than 18 000 at the time of writing)
+with a single thread.
+This is slow, but there are various ways to speed this up!
+
+First of all it is possible to set a ``TEST_PHP_ARGS`` variable with the make command
+that will pass the defined arguments to the test runner.
+One important argument is the ``-jN`` argument which allows the tests to be run in parallel (if possible).
+Some other important arguments are ``-q``, which makes the test runner silent (i.e. stops prompting to send the test results via email),
+and the ``--asan`` argument which is required when running tests with a build of PHP compiled with ASAN.
+
+Secondly, one can reduce the number of tests being run by specifying which individual tests, or test folders
+should be run via the ``TESTS`` variable.
+
+******************************************
+ Via direct invocation of the test runner
+******************************************
+
+TODO
+
+*********************
+ Test runner options
+*********************
+
+The test runner accepts various options that can be discovered by invoking it with ``--help``
+Here is a list of the most common and useful options:
+
+- ``-g``: to filter which tests are shown, e.g. ``-g FAIL`` to only show failing tests
+- ``-x``: to skip tests marked as slow
+- ``--offline``: to skip tests which require an internet connection
+- ``-d``: to set an INI option, useful to enable/disable opcache and the JIT, e.g. ``-d opcache.enable=0``

docs/source/tests/writing-tests.rst

@@ -0,0 +1,79 @@
+########################
+ Writing tests
+########################
+
+php-src's test runner uses a custom format for tests, named PHPT.
+Each PHPT test is composed of multiple sections,
+however each test must have at least 3 sections:
+
+- A title
+- A script to test
+- The expected output of the test
+
+A complete description of the available section is available TODO.
+
+.. toctree::
+
+ phpt-sections/test
+ phpt-sections/file
+
+*******************
+ Naming convention
+*******************
+
+The name of a test file should ideally be descriptive,
+however a lot of old tests files do not follow this convention as the old naming convention was:
+
+:Tests for a function's basic behaviour: ``<function_name>_basic.phpt`` (e.g. ``dba_open_basic.phpt``)
+
+:Tests for a function's error behaviour: ``<function_name>_error.phpt`` (e.g. ``dba_open_error.phpt``)
+
+:Tests for variations in a function's behaviour: ``<function_name>_variation.phpt`` (e.g. ``dba_open_variation.phpt``)
+
+:General tests for extensions: ``<extname><no>.phpt`` (e.g. ``dba_003.phpt``)
+
+For bugs the naming convention is as follows:
+
+- If the bug is a GitHub issue: ``gh<issue_no>.phpt`` (e.g. ``gh9032.phpt``)
+- If the bug is a bugsnet bug: ``bug<bug_no>.phpt`` (e.g. ``bug20528.phpt``)
+
+**********************
+ Writing a basic test
+**********************
+
+
+.. code:: php
+
+ --TEST--
+ echo - basic test for echo language construct
+ --FILE--
+ <?php
+ echo 'This works ', 'and takes args!';
+ ?>
+ --EXPECT--
+ This works and takes args!
+
+******************************************
+ An example of a more complicated test
+******************************************
+
+Let's write a test which checks that 64 bit integers are usable with the X function of the DateTime extension
+
+TODO
+
+.. code:: php
+
+ --TEST--
+ Test that X works with 64 bit integers
+ --EXTENSIONS--
+ date
+ --SKIPIF--
+ <?php
+ if (PHP_INT_SIZE != 8) die("skip this test is for 64bit platform only");
+ ?>
+ --FILE--
+ <?php
+ echo 'TODO';
+ ?>
+ --EXPECT--
+ TODO