Update README

tests/README.md

@@ -1,45 +1,53 @@
 # Testing the adapters library
 
-This README gives an overview of how the test directory is organized and the possibilities to group and execute different kinds of tests.
-## Test directory structure
+This README gives an overview of how the test directory is organized and the possibilities of grouping and executing different kinds of tests.
+## Overview test directory structure
 
 ```
 tests/
 ├── __init__.py
-├── fixtures/                               # Datasets, samples, ...
+├── fixtures/                       # Datasets, test samples, ...
 |   └── ...
-├── test_impl/                              # Test Implementations
+├── test_methods/                   # Dynamic adapter method tests (all models)
 │   ├── __init__.py
-│   ├── composition/
+│   ├── method_test_impl/               # Implementation of tests
 │   │   ├── __init__.py
-│   │   ├── test_adapter_composition.py
-│   │   └── test_parallel.py
-│   ├── core/
-│   │   ├── __init__.py
-│   │   ├── test_adapter_config.py
-│   │   ├── test_adapter_conversion.py
+│   │   ├── core/
+│   │   ├── composition/
 │   │   └── ...
-│   ├── embeddings/
+│   ├── base.py                         # Base from which model test bases inherit from                           
+│   ├── generator.py                    # Testcase generation and registration
+│   ├── test_albert.py                  # Example model test base testing adapter methods on the model
+│   ├── test_beit.py 
 │   └── ...
-├── test_methods/                           # Test entry points 
-│   └── __init__.py
-├── test_models/                            # Test entry points 
+├── test_misc/                      # Miscellaneous adapter method tests (single model)
+│   ├── test_adapter_config.py 
+│   └── ...
+├── test_models/                    # Adapter model tests with Hugging Face test suite
 │   └── __init__.py
+│   │   ├── base.py
+│   │   ├── test_albert.py
+│   │   └── ...
 ```
 
-## Test Types
+We differentiate between three kinds of tests:
 
-1. Adapter method tests: test the **implementation of the adapter methods**, such as the different kind of adapters or costum heads.
-    - These tests are exectued for each model, hence there is a testfile for each model, e.g. `test_albert.py`
-    - Each model test file is organized in various test classes to group similar tests
-    - While this results in a little bit more boilerplate code, it allows for an organized view in the test viewer, which in return also allows to conviniently execute subgroups of test, e.g. like this:
-    ![alt text](image.png)
-2. Adapter model tests: test the **implementation of the adapter models** on which the adapter methods can be used. 
-    - We resort to the thorough test suite of Hugging Face and test our models on it.
+1. Dynamic adapter method tests: These tests cover most functionalities of the adapters library, e.g. the individual adapter methods (LoRA, prompt tuning) or head functionalities and **are executed on every model**
+2. Miscellaneous adapter method tests: These are the remaining tests not covered by the dynamic tests and are **only executed on a single model** to spare ressources as repeated execution on every model would not provide additional value
+3. Adapter model tests: These tests **check the implementation of the adapter models** themselves, by applying the Hugging Face model test suite
 
-## Utilizing pytest markers
+## Test Generator $ Pytest Markers
+
+This chapter zooms in on the test_methods directory. The main actor here is the file `generator.py` which is used by every model test base to generate the appropriate set of adapter method tests. Those tests are then registered in the respective model test file, like this:
 
-Each class in each model test file in `tests/test_methods` is decorated with a marker of a certain type, e.g.:
+``` python
+method_tests = generate_method_tests(AlbertAdapterTestBase)
+
+for test_class_name, test_class in method_tests.items():
+    globals()[test_class_name] = test_class
+```
+
+Each generatable class in `tests/test_methods` is decorated with a marker of a certain type, e.g.:
 ``` python
 @require_torch
 @pytest.mark.lora
@@ -51,15 +59,15 @@ class LoRA(
     pass
 ```
 
-These markers can be used to execute a certain type of test **for every model**:
-- e.g.: for executing the compacter tests for every model we can write:
+These markers can be used to execute a certain type of test **for every model**. To use them you have two options:
+1. Use `make` command:
+    ```bash
+    make test-adapter-method-subset subset=lora
+    ```
+
+2. Navigate to directory and directly execute:
     ```bash
     cd tests/test_methods
     pytest -m lora
     ```
-    This command will execute all lora tests for every model in the adapters libray
-
-Alternatively to navigating to `tests/test_methods` in the terminal you can select a command from the `Makefile` in the root directory and launch such a subset of test via e.g.:
-```bash
-make test-adapter-lora
-```
+Both versions will execute all LoRA tests for every model in the adapters library.