Expression-based job filters (#9046)

* add section to config ref for expression job filters and use partial for operators table

* add example using pipeline params too

* add missing full stop

* fix lint error

* update wording after review

* add missing to

* update filters requirements definition to include string or map

* remove link to paragraph

jekyll/_cci2/configuration-reference.adoc

@@ -1761,7 +1761,7 @@ While this improves performance in most cases, if a downstream step requires tho
 ----
 ====
 
-A special step used to check out source code to the configured `path` (defaults to the `working_directory`). The reason this is a special step is because it is more of a helper function designed to make checking out code easy for you. If you require doing git over HTTPS you should not use this step as it configures git to checkout over SSH.
+A special step used to check out source code to the configured `path` (defaults to the `working_directory`). The reason this is a special step is because it is more of a helper function designed to simplify the process of checking out code. If you require doing git over HTTPS you should not use this step as it configures git to checkout over SSH.
 
 [.table.table-striped]
 [cols=4*, options="header", stripes=even]
@@ -2700,6 +2700,11 @@ An approval job can have any name. In the example above the approval job is name
 
 [#jobfilters]
 ====== *`filters`*
+Filter job execution within a workflow based on the following:
+
+* Branch
+* Tag
+* Expression-based condition
 
 Job filters can have the keys `branches` or `tags`.
 
@@ -2713,10 +2718,10 @@ NOTE: Workflows will ignore job-level branching. If you use job-level branching
 | `filters`
 | N
 | Map
-| A map defining rules for execution on specific branches
+| A map or string to define rules for job execution. Branch and tag filters require a map. Expression-based filters require a string.
 |===
 
-The following is an example of how the CircleCI documentation uses a regex to filter running a workflow for building PDF documentation:
+The following is an example of how the CircleCI documentation project uses a regular expression to filter running a job in a workflow only on a specific branch:
 
 [,yaml]
 ----
@@ -2728,19 +2733,128 @@ workflows:
       - build_server_pdfs: # << the job to conditionally run based on the filter-by-branch-name.
           filters:
             branches:
-              only: /server\/.*/
+              only: /server\/.*/ # the job build_server_pdfs will only run when the branch being built starts with server/
+----
+
+You can read more about using regular expressions in your config in the xref:workflows#using-regular-expressions-to-filter-tags-and-branches[Using workflows to schedule jobs] page.
+
+'''
+
+====== Expression-based job filters
+Expression-based job filters allow you to conditionally run jobs based on the following:
+
+* xref:variables#pipeline-values[Pipeline values]
+* xref:pipeline-variables#pipeline-parameters-in-configuration[Pipeline parameters]
+
+An expression-based job filter is a rule that is evaluated against pipeline values and parameters to decide whether a job should run.
+
+Using expression-based job filters is one way to optimize your pipelines to lower costs, decrease time to feedback, or run specific jobs based on the context of the source of change.
+
+[,yml]
+----
+workflows:
+  deploy:
+    jobs:
+      - init-service
+      - build-service-image:
+          requires:
+            - init-service
+      - dry-run-service:
+          requires:
+            - init-service
+          filters: pipeline.git.branch != "main" and pipeline.git.branch != "canary"
+      - publish-service:
+          requires:
+            - build-service-image
+            - test-service
+          filters: pipeline.git.branch == "main" or pipeline.git.tag starts-with "release"
+      - deploy-service:
+          context:
+            - org-global
+          requires:
+            - publish-service
+          filters: pipeline.git.branch == "main" and pipeline.git.commit.subject starts-with "DEPLOY:"
+----
+
+'''
+
+**Examples**
+
+Only run the job on the project's `main` branch:
+
+[source,yml]
+----
+filters: pipeline.git.branch == "main"
+----
+
+Only run the job on the project's `main` branch, or branches starting with `integration-test`:
+
+[source,yml]
+----
+filters: pipeline.git.branch == "main" or pipeline.git.branch starts-with "integration-test"
 ----
 
-The above snippet causes the job  `build_server_pdfs` to only be run when the branch being built starts with "server/".
+Only run the job on the `main` branch, and disallow use with pipelines xref:vs-code-extension-overview#test-run-your-config-from-vs-code[triggered with unversioned configuration]:
 
-You can read more about using regex in your config in the xref:workflows#using-regular-expressions-to-filter-tags-and-branches[Using workflows to schedule jobs] page.
+[source,yml]
+----
+filters: pipeline.git.branch == "main" and not (pipeline.config_source starts-with "api")
+----
+
+Use pipeline parameters and the pipeline value `pipeline.git.branch` to run a job only on specific branches **or** when triggered via the API with a pipeline parameter set to true:
+
+[source,yml]
+----
+version: 2.1
+
+parameters:
+  run-storybook-tests:
+    type: boolean
+    default: false
+
+...
+# jobs configuration ommitted for brevity
+
+workflows:
+  build:
+    jobs:
+      - setup
+      - storybook-tests:
+          requires:
+            - setup
+          filters: |
+            pipeline.parameters.run-storybook-tests
+            or pipeline.git.branch == "dry-run-deploy"
+            or pipeline.git.branch starts-with "deploy"
+----
+
+You can use the API to trigger a pipeline with a pipeline parameter set to true:
+
+[source,yml]
+----
+curl -X POST https://circleci.com/api/v2/project/circleci/<org-id>/<project-id>/pipeline/run \
+  --header "Circle-Token: $CIRCLE_TOKEN" \
+  --header "content-type: application/json" \
+  --data {
+  "definition_id": "<pipeline-definition-id>",
+  "config": {"branch": "<your-branch-name>"},
+  "checkout": {"branch": "<your-branch-name>"},
+  "parameters": {"run-storybook-tests": "true"}
+  }
+----
+
+**Operators**
+
+The operators you can use for expression-based job filters are described in the following table. You can also group sub-expressions with parentheses `(`, `)`. as in the examples above.
+
+include::../_includes/partials/using-expressions/operators.adoc[]
 
 '''
 
 [#branches]
 ====== *`branches`*
 
-Branches can have the keys `only` and `ignore`, which either map to a single string naming a branch. You may also use regular expressions to match against branches by enclosing them with slashes, or map to a list of such strings. Regular expressions must match the *entire* string.
+The branches filter can have the keys `only` and `ignore`, which map to a single string naming a branch. You may also use regular expressions to match against branches by enclosing them with slashes, or map to a list of such strings. Regular expressions must match the *entire* string.
 
 * Any branches that match `only` will run the job.
 * Any branches that match `ignore` will not run the job.
@@ -2755,17 +2869,17 @@ Branches can have the keys `only` and `ignore`, which either map to a single str
 | `branches`
 | N
 | Map
-| A map defining rules for execution on specific branches
+| A map defining rules for execution on specific branches.
 
 | `only`
 | N
-| String, or List of Strings
-| Either a single branch specifier, or a list of branch specifiers
+| String, or list of strings
+| Either a single branch specifier, or a list of branch specifiers.
 
 | `ignore`
 | N
-| String, or List of Strings
-| Either a single branch specifier, or a list of branch specifiers
+| String, or list of strings
+| Either a single branch specifier, or a list of branch specifiers.
 |===
 
 [source,yaml]

jekyll/_cci2/contexts.adoc

@@ -362,36 +362,7 @@ Any errors evaluating an expression will _fail closed_ and prevent use of the co
 
 The operators you can use are described in the following table. You can also group sub-expressions with parentheses `(`, `)`. as in the example above.
 
-[.table.table-striped]
-[cols=3*, options="header", stripes=even]
-|===
-| Operator type | Operators | Description
-
-| Logical
-|`and`, `or`
-| These are short-circuiting boolean operators.
-
-| Equality
-| `==`, `!=`
-| String, numeric, and boolean equality. If the operands are of different types then `==` will evaluate `false`, and `!=` will evaluate `true`.
-
-| Equality
-| `starts-with`
-| String prefix equality, `"hello world" starts-with "hello"` evaluates as `true`. It is an error to use a non-string type as an operand.
-
-| Numeric comparison
-| `>=`, `>`, `<=`, `<`
-| Numeric comparisons. It is an error to use a non-numeric type as an operand.
-
-| Negation
-| `not`
-a| Boolean negation.
-
-Note that `not` has very high precedence and so binds very tightly. Use sub-expressions to apply `not` to more complex expressions. For example, with `foo` being `true` and `bar` being `false`:
-
-  * `not foo and bar` evaluates to `false`
-  * `not (foo and bar)` evaluates to `true`
-|===
+include::../_includes/partials/using-expressions/operators.adoc[]
 
 === Precedence
 The following table shows operator precedence table, from weakest to strongest binding.

jekyll/_includes/partials/using-expressions/operators.adoc

@@ -0,0 +1,30 @@
+[.table.table-striped]
+[cols=3*, options="header", stripes=even]
+|===
+| Operator type | Operators | Description
+
+| Logical
+|`and`, `or`
+| These are short-circuiting boolean operators.
+
+| Equality
+| `==`, `!=`
+| String, numeric, and boolean equality. If the operands are of different types then `==` will evaluate `false`, and `!=` will evaluate `true`.
+
+| Equality
+| `starts-with`
+| String prefix equality, `"hello world" starts-with "hello"` evaluates as `true`. It is an error to use a non-string type as an operand.
+
+| Numeric comparison
+| `>=`, `>`, `<=`, `<`
+| Numeric comparisons. It is an error to use a non-numeric type as an operand.
+
+| Negation
+| `not`
+a| Boolean negation.
+
+Note that `not` has very high precedence and so binds very tightly. Use sub-expressions to apply `not` to more complex expressions. For example, with `foo` being `true` and `bar` being `false`:
+
+  * `not foo and bar` evaluates to `false`
+  * `not (foo and bar)` evaluates to `true`
+|===