From 9236f28650081ba7199114bfd5fd0be622b07abd Mon Sep 17 00:00:00 2001 From: Juan Leaniz Date: Fri, 13 Oct 2023 10:26:31 -0500 Subject: [PATCH] Update readme (#1374) * Update readme * add link --- README.md | 215 ++++++++++++++---------------------------------------- 1 file changed, 53 insertions(+), 162 deletions(-) diff --git a/README.md b/README.md index 91c29e9ec..6a15f3fb6 100644 --- a/README.md +++ b/README.md @@ -52,177 +52,65 @@ configuration are: * Start Turbinia server component with `turbiniactl server` command * Start Turbinia API server component with `turbiniactl api_server` command if using Celery * Start one or more Turbinia workers with `turbiniactl celeryworker` if using Celery, or `turbiniactl psqworker` if using PSQ -* Send evidence to be processed from the turbinia client with `turbiniactl - ${evidencetype}` -* Check status of running tasks with `turbiniactl status` +* Install `turbinia-client` via `pip install turbinia-client` +* Send evidence to be processed from the turbinia client with `turbinia-client submit ${evidencetype}` +* Check status of running tasks with `turbinia-client status` -turbiniactl can be used to start the different components, and here is the basic +turbinia-client can be used to interact with Turbinia through the API server component, and here is the basic usage: ``` -$ turbiniactl -h -usage: turbiniactl [-h] [-q] [-v] [-d] [-a] [-c CONFIG_FILE] [-C RECIPE_CONFIG] - [-f] [-k DECRYPTION_KEYS] [-o OUTPUT_DIR] [-L LOG_FILE] - [-r REQUEST_ID] [-R] [-V] [-D] [-F FILTER_PATTERNS_FILE] - [-Y YARA_RULES_FILE] [-j JOBS_ALLOWLIST] [-J JOBS_DENYLIST] - [-p POLL_INTERVAL] [-t TASK] [-T] [-w] - ... - -optional arguments: - -h, --help show this help message and exit - -q, --quiet Show minimal output - -v, --verbose Show verbose output - -d, --debug Show debug output - -a, --all_fields Show all task status fields in output - -c CONFIG_FILE, --config_file CONFIG_FILE - Load explicit config file. If specified it will ignore - config files in other default locations - (/etc/turbinia.conf, ~/.turbiniarc, or in paths - referenced in environment variable TURBINIA_CONFIG_PATH) - -C RECIPE_CONFIG, --recipe_config RECIPE_CONFIG - Recipe configuration data passed in as comma separated - key=value pairs (e.g. "-C - key=value,otherkey=othervalue"). These will get passed - to tasks as evidence config, and will also be written to - the metadata.json file for Evidence types that write it - -f, --force_evidence Force evidence processing request in potentially unsafe - conditions - -k DECRYPTION_KEYS, --decryption_keys DECRYPTION_KEYS - Decryption keys to be passed in as comma separated list. - Each entry should be in the form type=key. (e.g. "-k - password=123456,recovery_password=XXXX-XXXX-XXXX-XXXX") - -o OUTPUT_DIR, --output_dir OUTPUT_DIR - Directory path for output - -L LOG_FILE, --log_file LOG_FILE - Log file - -r REQUEST_ID, --request_id REQUEST_ID - Create new requests with this Request ID - -V, --version Show the version - -D, --dump_json Dump JSON output of Turbinia Request instead of sending - it - -F FILTER_PATTERNS_FILE, --filter_patterns_file FILTER_PATTERNS_FILE - A file containing newline separated string patterns to - filter text based evidence files with (in extended grep - regex format). This filtered output will be in addition - to the complete output - -Y YARA_RULES_FILE, --yara_rules_file YARA_RULES_FILE - A file containing Yara rules. - -j JOBS_ALLOWLIST, --jobs_allowlist JOBS_ALLOWLIST - An allowlist for Jobs that will be allowed to run (in - CSV format, no spaces). This will not force them to run - if they are not configured to. This is applied both at - server start time and when the client makes a processing - request. When applied at server start time the change is - persistent while the server is running. When applied by - the client, it will only affect that processing request. - -J JOBS_DENYLIST, --jobs_denylist JOBS_DENYLIST - A denylist for Jobs we will not allow to run. See - --jobs_allowlist help for details on format and when it - is applied. - -p POLL_INTERVAL, --poll_interval POLL_INTERVAL - Number of seconds to wait between polling for task state - info - -T, --debug_tasks Show debug output for all supported tasks - -w, --wait Wait to exit until all tasks for the given request have - completed +$ turbinia-client -h +Usage: turbinia-client [OPTIONS] COMMAND [ARGS]... + + Turbinia API command-line tool (turbinia-client). + + *** *** + * * + *** ****** + * * + ** * * ** ,* + ******* * ******** + * * * + * * * + %%%%%% + %%%%%% + %%%%%%%%%%%%%%% %%%%%% + %%%%%%%%%%%%%%%%%%%%% %%%%%%% + %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ** ******* + %% %% *************** + %% (%%%%%%%%%%%%%%%%%%% ***** ** + %%%%% %%%%%%%%%%%%%%% + %%%%%%%%%% %% ** *** + %%% %% %% %%% %%%%, + %%% %%% %%% %%%%% %%% %%% %% %%% %%% %%% (%% + %%% %%% %%% %%% %% %%/ %% %%% %%% %%% %%%%%%%% + %%% %%% %%% %%% %%% %%% %% %%% %%% %%% %%% %%% + %%% %%%%% %%% %%%%% %% %%% %% %%% %%%%% + + This command-line tool interacts with Turbinia's API server. + + You can specify the API server location in ~/.turbinia_api_config.json + +Options: + -c, --config_instance TEXT A Turbinia instance configuration name. + [default: (dynamic)] + -p, --config_path TEXT Path to the .turbinia_api_config.json file.. + [default: (dynamic)] + -h, --help Show this message and exit. Commands: - - config Print out config file - testnotify Sends test notification - rawdisk Process RawDisk as Evidence - googleclouddisk Process Google Cloud Persistent Disk as Evidence - googleclouddiskembedded - Process Google Cloud Persistent Disk with an embedded - raw disk image as Evidence - rawmemory Process RawMemory as Evidence - directory Process a directory as Evidence - compresseddirectory - Process a compressed tar file as Evidence - hindsight Process ChromiumProfile as Evidence - listjobs List all available Jobs. These Job names can be used by - --jobs_allowlist and --jobs_denylist - psqworker Run PSQ worker - celeryworker Run Celery worker - status Get Turbinia Task status - gcplogs Collects Turbinia logs from Stackdriver. - dumpgcs Get Turbinia results from Google Cloud Storage. - server Run Turbinia Server + config Get Turbinia configuration. + evidence Get or upload Turbinia evidence. + jobs Get a list of enabled Turbinia jobs. + result Get Turbinia request or task results. + status Get Turbinia request or task status. + submit Submit new requests to the Turbinia API server. ``` -The commands for processing evidence specify the metadata about that evidence -for Turbinia to process. By default, when adding new evidence to be processed, -turbiniactl will act as a client and send a request to the configured Turbinia -server, otherwise if `server` is specified, it will start up its own Turbinia -server process. Here's the turbiniactl usage for adding a raw disk type of -evidence to be processed by Turbinia: +Check out the `turbinia-client` documentation [page]((https://turbinia.readthedocs.io/en/latest/user/turbinia-client.html#turbinia-api-cli-tool-turbinia-client)) for a detailed user guide. -``` -$ turbiniactl rawdisk -h -usage: turbiniactl rawdisk [-h] -l SOURCE_PATH [-P MOUNT_PARTITION] - [-s SOURCE] [-n NAME] - -optional arguments: - -h, --help show this help message and exit - -l SOURCE_PATH, --source_path SOURCE_PATH - Local path to the evidence - -P MOUNT_PARTITION, --mount_partition MOUNT_PARTITION - The partition number to use when mounting this disk. - Defaults to the entire raw disk. Only affects - mounting, and not what gets processed. - -s SOURCE, --source SOURCE - Description of the source of the evidence - -n NAME, --name NAME Descriptive name of the evidence -``` - -Sample rawdisk command: -`turbiniactl rawdisk -l {PATH_TO_THE_IMAGE}` - -Status information about the requests that are being or have been processed can -be viewed with the `turbiniactl status` command. You can specify the request ID -that was generated, or other filters like the username of the requester, or how -many days of processing history you want to view. You can also generate -statistics and reports (in markdown format) with other flags. - -``` -$ turbiniactl status -h -usage: turbiniactl status [-h] [-c] [-C] [-d DAYS_HISTORY] [-f] - [-r REQUEST_ID] [-p PRIORITY_FILTER] [-R] [-s] - [-t TASK_ID] [-u USER] [-i] [-w] - -optional arguments: - -h, --help show this help message and exit - -c, --close_tasks Close tasks based on Request ID or Task ID - -C, --csv When used with --statistics, the output will be in CSV - format - -d DAYS_HISTORY, --days_history DAYS_HISTORY - Number of days of history to show - -f, --force Gatekeeper for --close_tasks - -r REQUEST_ID, --request_id REQUEST_ID - Show tasks with this Request ID - -p PRIORITY_FILTER, --priority_filter PRIORITY_FILTER - This sets what report sections are shown in full - detail in report output. Any tasks that have set a - report_priority value equal to or lower than this - setting will be shown in full detail, and tasks with a - higher value will only have a summary shown. To see - all tasks report output in full detail, set - --priority_filter=100 - -R, --full_report Generate full markdown report instead of just a - summary - -s, --statistics Generate statistics only - -t TASK_ID, --task_id TASK_ID - Show task for given Task ID - -u USER, --user USER Show task for given user - -i, --requests Show all requests from a specified timeframe. The - default timeframe is 7 days. Please use the -d flag to - extend this. - -w, --workers Show Worker status information from a specified - timeframe. The default timeframe is 7 days. Please use - the -d flag to extend this. -``` -Sample status command: -`turbiniactl -a status -r {REQUEST_ID}` +You can also interact with Turbinia directly from Python by using the API library. We provide some examples [here](https://github.com/google/turbinia/tree/master/turbinia/api/client) ## Other documentation @@ -230,6 +118,9 @@ Sample status command: * [Installation](https://turbinia.readthedocs.io/en/latest/user/install.html) * [How it works](https://turbinia.readthedocs.io/en/latest/user/how-it-works.html) * [Operational Details](https://turbinia.readthedocs.io/en/latest/user/operational-details.html) +* [Turbinia client CLI tool](https://turbinia.readthedocs.io/en/latest/user/turbinia-client.html#turbinia-api-cli-tool-turbinia-client) +* [Turbinia API server](https://turbinia.readthedocs.io/en/latest/user/api-server.html) +* [Turbinia Python API library](https://github.com/google/turbinia/tree/master/turbinia/api/client) * [Contributing to Turbinia](https://turbinia.readthedocs.io/en/latest/developer/contributing.html) * [Developing new Tasks](https://turbinia.readthedocs.io/en/latest/developer/developing-new-tasks.html) * [FAQ](https://turbinia.readthedocs.io/en/latest/user/faq.html)