Merge branch '2.8.x' of github.com:RasaHQ/rasa into 2.8.x

RasaHQ · Jul 12, 2021 · ebc24f6 · ebc24f6
2 parents 297f944 + f627371
commit ebc24f6
Show file tree

Hide file tree

Showing 17 changed files with 118 additions and 110 deletions.
diff --git a/CHANGELOG.mdx b/CHANGELOG.mdx
@@ -16,6 +16,122 @@ https://github.com/RasaHQ/rasa/tree/main/changelog/ . -->
 
 <!-- TOWNCRIER -->
 
+## [2.8.0] - 2021-07-12
+### Deprecations and Removals
+- [#9045](https://github.com/rasahq/rasa/issues/9045): The option `model_confidence=linear_norm` is deprecated and will be removed in Rasa Open Source `3.0.0`.
+
+  Rasa Open Source `2.3.0` introduced `linear_norm` as a possible value for `model_confidence`
+  parameter in machine learning components such as `DIETClassifier`, `ResponseSelector` and `TEDPolicy`.
+  Based on user feedback, we have identified multiple problems with this option.
+  Therefore, `model_confidence=linear_norm` is now deprecated and
+  will be removed in Rasa Open Source `3.0.0`. If you were using `model_confidence=linear_norm` for any of the mentioned components,
+  we recommend to revert it back to `model_confidence=softmax` and re-train the assistant. After re-training,
+  we also recommend to [re-tune the thresholds for fallback components](./fallback-handoff.mdx#fallbacks).
+- [#9091](https://github.com/rasahq/rasa/issues/9091): The fallback mechanism for spaCy models has now been removed in Rasa `3.0.0`. 
+
+  Rasa Open Source `2.5.0` introduced support for spaCy 3.0. This introduced a 
+  breaking feature because models would no longer be manually linked. To make
+  the transition smooth Rasa would rely on the `language` parameter in the
+  `config.yml` to fallback to a medium spaCy model if no model was configured
+  for the `SpacyNLP` component. In Rasa Open Source `3.0.0` and onwards the 
+  `SpacyNLP` component will require the model name (like `"en_core_web_md"`) 
+  to be passed explicitly.
+
+### Features
+- [#8724](https://github.com/rasahq/rasa/issues/8724): Added `sasl_mechanism` as an optional configurable parameters for the [Kafka Producer](event-brokers.mdx#kafka-event-broker).
+- [#8913](https://github.com/rasahq/rasa/issues/8913): Introduces a new policy called [`UnexpecTEDIntentPolicy`](./policies.mdx#unexpected-intent-policy).
+
+  `UnexpecTEDIntentPolicy` helps you [review conversations](./conversation-driven-development.mdx#review)
+   and also allows your bot to react to unexpected user turns in conversations.
+   It is an auxiliary policy that should only be used in conjunction with
+   at least one other policy, as the only action that it can trigger
+   is the special and newly introduced
+   [`action_unlikely_intent`](./default-actions.mdx#action_unlikely_intent) action.
+
+   The auto-configuration will include `UnexpecTEDIntentPolicy` in your
+   configuration automatically, but you can also include it yourself
+   in the `policies` section of the configuration:
+
+   ```
+   policies:
+     - name: UnexpecTEDIntentPolicy
+       epochs: 200
+       max_history: 5
+   ```
+
+  As part of the feature, it also introduces:
+
+  - [`IntentMaxHistoryTrackerFeaturizer`](./policies.mdx#3-intent-max-history)
+    to featurize the trackers for `UnexpecTEDIntentPolicy`.
+  - `MultiLabelDotProductLoss` to support `UnexpecTEDIntentPolicy`'s multi-label training objective.
+  - A new default action called [`action_unlikely_intent`](./default-actions.mdx#action_unlikely_intent).
+
+
+  `rasa test` command has also been adapted to `UnexpecTEDIntentPolicy`:
+
+  - If a test story contains `action_unlikely_intent` and the policy ensemble does not trigger it, this leads to
+    a test error (wrongly predicted action) and the corresponding story will be logged in `failed_test_stories.yml`.
+  - If the story does not contain `action_unlikely_intent` and Rasa Open Source does predict it then
+    the prediction of `action_unlikely_intent` will be ignored for the evaluation (and hence not lead
+    to a prediction error) but the story will be logged in a file called `stories_with_warnings.yml`.
+
+
+  The `rasa data validate` command will warn if `action_unlikely_intent` is
+  included in the training stories. Accordingly, `YAMLStoryWriter` and `MarkdownStoryWriter` have been updated to not dump `action_unlikely_intent` when writing stories to a file.
+
+  :::caution
+  The introduction of a new default action **breaks backward compatibility of previously trained models**.
+  It is not possible to load models trained with previous versions of Rasa Open Source. Please re-train
+  your assistant before trying to use this version.
+
+  :::
+
+### Improvements
+- [#8127](https://github.com/rasahq/rasa/issues/8127): Added detailed json schema validation for `UserUttered`, `SlotSet`, `ActionExecuted` and `EntitiesAdded` events both sent and received from the action server, as well as covered at high-level the validation of the rest of the 20 events.
+  In case the events are invalid, a `ValidationError` will be raised.
+- [#8232](https://github.com/rasahq/rasa/issues/8232): Users don't need to specify an additional buffer size for sparse featurizers anymore during incremental training.
+
+  Space for new sparse features are created dynamically inside the downstream machine learning 
+  models - `DIETClassifier`, `ResponseSelector`. In other words, no extra buffer is created in 
+  advance for additional vocabulary items and space will be dynamically allocated for them inside the model.
+
+  This means there's no need to specify `additional_vocabulary_size` for [`CountVectorsFeaturizer`](./components.mdx#countvectorsfeaturizer) or
+  `number_additional_patterns` for [`RegexFeaturizer`](./components.mdx#regexfeaturizer). These parameters are now deprecated.
+
+  **Before**
+  ```yaml
+  pipeline:
+    - name: "WhitespaceTokenizer"
+    - name: "RegexFeaturizer"
+      number_additional_patterns: 100
+    - name: "CountVectorsFeaturizer"
+      additional_vocabulary_size: {text: 100, response: 20}
+  ```
+
+  **Now**
+  ```yaml
+  pipeline:
+    - name: "WhitespaceTokenizer"
+    - name: "RegexFeaturizer"
+    - name: "CountVectorsFeaturizer"
+  ```
+
+  Also, all custom layers specifically built for machine learning models - `RasaSequenceLayer`, `RasaFeatureCombiningLayer` 
+  and `ConcatenateSparseDenseFeatures` now inherit from `RasaCustomLayer` so that they support flexible incremental training out of the box.
+- [#8295](https://github.com/rasahq/rasa/issues/8295): Speed up the contradiction check of the [`RulePolicy`](policies.mdx#rule-policy)
+  by a factor of 3.
+- [#8801](https://github.com/rasahq/rasa/issues/8801): Change the confidence score assigned by [`FallbackClassifier`](components.mdx#fallbackclassifier) to fallback intent to be the same as the fallback threshold.
+- [#8926](https://github.com/rasahq/rasa/issues/8926): Issue a UserWarning if a specified **domain folder** contains files that look like YML files but cannot be parsed successfully. 
+  Only invoked if user specifies a folder path in `--domain` paramater. Previously those invalid files in the specified folder were silently ignored.  
+  **Does not apply** to individually specified domain YAML files, e.g. `--domain /some/path/domain.yml`, those being invalid will still raise an exception.
+
+### Bugfixes
+- [#8711](https://github.com/rasahq/rasa/issues/8711): Fix for unnecessary retrain and duplication of folders in the model
+
+### Miscellaneous internal changes
+- [#8241](https://github.com/rasahq/rasa/issues/8241), [#8525](https://github.com/rasahq/rasa/issues/8525), [#8694](https://github.com/rasahq/rasa/issues/8694), [#8704](https://github.com/rasahq/rasa/issues/8704)
+
+
 ## [2.7.1] - 2021-06-16
 
 

diff --git a/changelog/8127.improvement.md b/changelog/8127.improvement.md
diff --git a/changelog/8232.improvement.md b/changelog/8232.improvement.md
diff --git a/changelog/8241.misc.md b/changelog/8241.misc.md
diff --git a/changelog/8295.improvement.md b/changelog/8295.improvement.md
diff --git a/changelog/8525.misc.md b/changelog/8525.misc.md
diff --git a/changelog/8694.misc.md b/changelog/8694.misc.md
diff --git a/changelog/8704.misc.md b/changelog/8704.misc.md
diff --git a/changelog/8711.bugfix.md b/changelog/8711.bugfix.md
diff --git a/changelog/8724.feature.md b/changelog/8724.feature.md
diff --git a/changelog/8801.improvement.md b/changelog/8801.improvement.md
diff --git a/changelog/8913.feature.md b/changelog/8913.feature.md
diff --git a/changelog/8926.improvement.md b/changelog/8926.improvement.md
diff --git a/changelog/9045.removal.md b/changelog/9045.removal.md
diff --git a/changelog/9091.removal.md b/changelog/9091.removal.md
diff --git a/pyproject.toml b/pyproject.toml
@@ -9,7 +9,7 @@ exclude = "((.eggs | .git | .pytest_cache | build | dist))"
 
 [tool.poetry]
 name = "rasa"
-version = "2.7.1"
+version = "2.8.0"
 description = "Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants"
 authors = [ "Rasa Technologies GmbH <hi@rasa.com>",]
 maintainers = [ "Tom Bocklisch <tom@rasa.com>",]

diff --git a/rasa/version.py b/rasa/version.py
@@ -1,3 +1,3 @@
 # this file will automatically be changed,
 # do not add anything but the version number here!
-__version__ = "2.7.1"
+__version__ = "2.8.0"