Skip to content

Commit

Permalink
Additional links "next steps" in the working with schedules manuals.
Browse files Browse the repository at this point in the history
  • Loading branch information
piotrczarnas committed Jan 17, 2024
1 parent b164d34 commit e0467ae
Show file tree
Hide file tree
Showing 3 changed files with 28 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@ Instructions on how to configure the schedule using the user interface can be fo

## Configuring a schedule at the connection level

To schedule a check on an entire connection, you first need to open the YAML file of that connection. To do this, simply
use the [connection edit](../../command-line-interface/connection.md#dqo-connection-edit) command, which will launch the
YAML file in Visual Studio Code. To make working with the YAML file even easier, we recommend installing the YAML
extension by RedHat and the Better Jinja extension by Samuel Colvin.
To schedule a check on an entire connection, you first need to open the YAML file of that connection.
You can edit the YAML file directly on the disk, or let DQOps open the right file in the text editor using
the [connection edit](../../command-line-interface/connection.md#dqo-connection-edit) command from the [DQOps shell](../working-with-dqo-shell.md). This command will open Visual Studio Code
to edit the file. You will find more information about [configuring code-completion and schema validation in Visual Studio Code](../../integrations/visual-studio-code/index.md)
in the [DQOps integrations](../../integrations/index.md) section.

For example, to edit the connection named "testconnection" just run

```
connection edit -c=testconnection
```
Expand Down Expand Up @@ -107,6 +109,7 @@ the YAML files will look like the ones provided below:


=== "Schedule on table level row_count check"

``` yaml hl_lines="11-12"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
Expand Down Expand Up @@ -136,7 +139,9 @@ the YAML files will look like the ones provided below:
column_type: FLOAT64
nullable: true
```

=== "Schedule on column level nulls_count check"

```yaml hl_lines="16-17"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
Expand All @@ -161,4 +166,8 @@ the YAML files will look like the ones provided below:
type_snapshot:
column_type: STRING
nullable: true
```
```

## What's next
- Learn how DQOps [stores the configuration of data sources](../../dqo-concepts/configuring-data-sources.md).
- Learn how to configure [scheduling data quality checks](index.md) from the DQOps user interface.
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,8 @@ Cron expression usage examples:
`0 0 * * SUN` At 12:00 AM, only on Sunday

`0 22 * * 1-5` Every weekday (Mon-Fri) at 10:00 PM


## What's next
- Learn how to configure [scheduling data quality checks](index.md) from the DQOps user interface.
- Learn how to configure [CRON schedules for running data quality checks](configuring-schedules-by-modifying-yaml-file.md) in YAML files.
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,11 @@ command `scheduler stop`. For further information on the `scheduler` commands, p

Scheduler can also be started in a server mode that continuously run a job scheduler and synchronize the data every 10 minutes.
To do this, simply enter the command below in your terminal:

```
$ dqo run
```

To terminate dqo running in the background, simply use the Ctrl+C.

For more information on the `run` command, please refer to the [Command-line interface section](../../command-line-interface/run.md).
Expand All @@ -115,6 +117,7 @@ new schedules, start dqo with the following parameter:
```
$ dqo --dqo.scheduler.scan-metadata-cron-schedule=<Unix cron expression>
```

Please use quotation marks when defining a frequency in cron format.

You can also configure this parameter by setting `DQO_SCHEDULER_SCAN_METADATA_CRON_SCHEDULE=<Unix cron expression>`
Expand All @@ -127,6 +130,11 @@ with the following parameter:
```
$ dqo --dqo.scheduler.enable-cloud-sync=<true/false>
```

To enable synchronization type `true`, to disable it type `false`.

You can also configure this parameter by setting `DQO_SCHEDULER_ENABLE_CLOUD_SYNC=<TRUE/FALSE>` environment variable.
You can also configure this parameter by setting `DQO_SCHEDULER_ENABLE_CLOUD_SYNC=<TRUE/FALSE>` environment variable.

## What's next
- Learn how to configure [CRON schedules for running data quality checks](configuring-schedules-by-modifying-yaml-file.md) in YAML files.
- The format of expression and sample schedules are documented in the [cron formatting](cron-formatting.md) reference.

0 comments on commit e0467ae

Please sign in to comment.