Add doc page about upgrading

docs/index.rst

@@ -46,8 +46,7 @@ Topics
    contributing
    release_notes
    versioning
-   upgrading_binary
-   upgrading_unicodeset
+   upgrading
 
 API docs
 ========

docs/release_notes.rst

@@ -1,3 +1,5 @@
+.. _release_notes:
+
 Release Notes
 =============
 
@@ -102,8 +104,13 @@ v5.0.3
 
 :date: 2021-02-14
 
+This version has an unintentional breaking change:
+
 * Propagate ``Model.serialize``'s ``null_check`` parameter to nested MapAttributes (:pr:`908`)
 
+  Previously null errors (persisting ``None`` into an attribute defined as ``null=False``)
+  were ignored for attributes in map attributes that were nested in maps or lists. After upgrade,
+  these will resulted in an :py:class:`~pynamodb.exceptions.AttributeNullError` being raised.
 
 v5.0.2
 ----------
@@ -137,7 +144,7 @@ Breaking changes
   In previous versions, assigning an empty value to would be akin to assigning ``None``: if the attribute was defined with ``null=True`` then it would be omitted, otherwise an error would be raised.
 
   As of May 2020, DynamoDB `supports <https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-dynamodb-now-supports-empty-values-for-non-key-string-and-binary-attributes-in-dynamodb-tables/>`_ empty values for String and Binary attributes. This release of PynamoDB starts treating empty values like any other values. If existing code unintentionally assigns empty values to StringAttribute or BinaryAttribute, this may be a breaking change: for example, the code may rely on the fact that in previous versions empty strings would be "read back" as ``None`` values when reloaded from the database.
-* :py:class:`~pynamodb.attributes.UTCDateTimeAttribute` now strictly requires the date string format '%Y-%m-%dT%H:%M:%S.%f%z' to ensure proper ordering.
+* :py:class:`~pynamodb.attributes.UTCDateTimeAttribute` now strictly requires the date string format ``'%Y-%m-%dT%H:%M:%S.%f%z'`` to ensure proper ordering.
   PynamoDB has always written values with this format but previously would accept reading other formats.
   Items written using other formats must be rewritten before upgrading.
 * Table backup functionality (``Model.dump[s]`` and ``Model.load[s]``) has been removed.
@@ -321,20 +328,34 @@ of all attributes including the key attributes.
 Support for `Legacy Conditional Parameters <https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LegacyConditionalParameters.html>`_ has been
 removed. See a complete list of affected ``Model`` methods below:
 
-* ``update_item``: removed in favor of ``update``.
-* ``rate_limited_scan``: removed in favor of ``scan`` and ``ResultIterator``.
+.. list-table::
+   :widths: 10 90
+   :header-rows: 1
+
+   * - Method
+     - Changes
+   * - ``update_item``
+     - removed in favor of ``update``
+   * - ``rate_limited_scan``
+     - removed in favor of ``scan`` and ``ResultIterator``
+   * - ``delete``
+     - ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``condition`` instead.
+   * - ``update``
+     - ``attributes``, ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``actions`` and ``condition`` instead.
+   * - ``save``
+     - ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``condition`` instead.
+   * - ``count``
+     - ``**filters`` kwargs removed. Use ``range_key_condition``/``filter_condition`` instead.
+   * - ``query``
+     - ``conditional_operator`` and ``**filters`` kwargs removed. Use ``range_key_condition``/``filter_condition`` instead.
+   * - ``scan``
+     -
+       - ``conditional_operator`` and ``**filters`` kwargs removed. Use ``filter_condition`` instead.
+       - ``allow_rate_limited_scan_without_consumed_capacity`` was removed
 
-  + Relatedly, the ``allow_rate_limited_scan_without_consumed_capacity`` option has been removed.
-* ``delete``: ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``condition`` instead.
-* ``update``: ``attributes``, ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``actions`` and ``condition`` instead.
-* ``save``: ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``condition`` instead.
-* ``count``: ``**filters`` kwargs removed. Use ``range_key_condition``/``filter_condition`` instead.
-* ``query``: ``conditional_operator`` and ``**filters`` kwargs removed. Use ``range_key_condition``/``filter_condition`` instead.
-* ``scan``: ``conditional_operator`` and ``**filters`` kwargs removed. Use ``filter_condition`` instead.
 
 When upgrading, pay special attention to use of ``**filters`` and ``**expected_values``, as you'll need to check for arbitrary names that correspond to
-attribute names. Also keep an eye out for kwargs like ``user_id__eq=5`` or ``email__null=True``, which are no longer supported. If you're not already using
-``mypy`` to type check your code, it can help you catch cases like these.
+attribute names. Also keep an eye out for kwargs like ``user_id__eq=5`` or ``email__null=True``, which are no longer supported. A type check can help you catch cases like these.
 
 New features in this release:
 

docs/upgrading.rst

@@ -0,0 +1,140 @@
+Upgrading
+=========
+
+This file complements the :ref:`release notes <release_notes>`, focusing on helping safe upgrades of the library
+in production scenarios.
+
+PynamoDB 5.x to 6.x
+-------------------
+
+BinaryAttribute is no longer double base64-encoded
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+See :ref:`upgrading_binary` for details.
+
+PynamoDB 4.x to 5.x
+-------------------
+
+Null checks enforced where they weren't previously
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Previously null errors (persisting ``None`` into an attribute defined as ``null=False``) were ignored inside **nested** map attributes, e.g.
+
+.. code-block:: python
+
+    from pynamodb.models import Model
+    from pynamodb.attributes import ListAttribute, MapAttribute, UnicodeAttribute
+
+    class Employee(MapAttribute):
+      name = UnicodeAttribute(null=False)
+
+    class Team(Model):
+      employees = ListAttribute(of=Employee)
+
+
+    team = Team()
+    team.employees = [Employee(name=None)]
+    team.save()  # this will raise now
+
+
+Now these will resulted in an :py:class:`~pynamodb.exceptions.AttributeNullError` being raised.
+
+This was an unintentional breaking change introduced in 5.0.3.
+
+Empty values are now meaningful
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:py:class:`~pynamodb.attributes.UnicodeAttribute` and :py:class:`~pynamodb.attributes.BinaryAttribute` now support empty values (:pr:`830`)
+
+In previous versions, assigning an empty value to would be akin to assigning ``None``: if the attribute was defined with ``null=True`` then it would be omitted, otherwise an error would be raised.
+DynamoDB `added support <https://aws.amazon.com/about-aws/whats-new/2020/05/amazon-dynamodb-now-supports-empty-values-for-non-key-string-and-binary-attributes-in-dynamodb-tables/>`_ empty values
+for String and Binary attributes. This release of PynamoDB starts treating empty values like any other values. If existing code unintentionally assigns empty values to StringAttribute or BinaryAttribute,
+this may be a breaking change: for example, the code may rely on the fact that in previous versions empty strings would be "read back" as ``None`` values when reloaded from the database.
+
+No longer parsing date-time strings leniently
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+:py:class:`~pynamodb.attributes.UTCDateTimeAttribute` now strictly requires the date string format ``'%Y-%m-%dT%H:%M:%S.%f%z'`` to ensure proper ordering.
+PynamoDB has always written values with this format but previously would accept reading other formats.
+Items written using other formats must be rewritten before upgrading.
+
+Removed functionality
+~~~~~~~~~~~~~~~~~~~~~
+
+The following changes are breaking but are less likely to go unnoticed:
+
+* Python 2 is no longer supported. Python 3.6 or greater is now required.
+* Table backup functionality (``Model.dump[s]`` and ``Model.load[s]``) has been removed.
+* ``Model.query`` no longer converts unsupported range key conditions into filter conditions.
+* Internal attribute type constants are replaced with their "short" DynamoDB version (:pr:`827`)
+* Remove ``ListAttribute.remove_indexes`` (added in v4.3.2) and document usage of remove for list elements (:pr:`838`)
+* Remove ``pynamodb.connection.util.pythonic`` (:pr:`753`) and (:pr:`865`)
+* Remove ``ModelContextManager`` class (:pr:`861`)
+
+PynamoDB 3.x to 4.x
+-------------------
+
+Requests Removal
+~~~~~~~~~~~~~~~~
+
+Given that ``botocore`` has moved to using ``urllib3`` directly for making HTTP requests, we'll be doing the same (via ``botocore``). This means the following:
+
+* The ``session_cls`` option is no longer supported.
+* The ``request_timeout_seconds`` parameter is no longer supported. ``connect_timeout_seconds`` and ``read_timeout_seconds`` are available instead.
+
+  + Note that the timeouts for connection and read are now ``15`` and ``30`` seconds respectively. This represents a change from the previous ``60`` second combined ``requests`` timeout.
+* *Wrapped* exceptions (i.e ``exc.cause``) that were from ``requests.exceptions`` will now be comparable ones from ``botocore.exceptions`` instead.
+
+Key attribute types must match table
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The previous release would call `DescribeTable` to discover table metadata
+and would use the key types as defined in the DynamoDB table. This could obscure
+type mismatches e.g. where a table's hash key is a number (`N`) in DynamoDB,
+but defined in PynamoDB as a `UnicodeAttribute`.
+
+With this release, we're always using the PynamoDB model's definition
+of all attributes including the key attributes.
+
+Deprecation of old APIs
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Support for `Legacy Conditional Parameters <https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/LegacyConditionalParameters.html>`_ has been
+removed. See a complete list of affected ``Model`` methods below:
+
+.. list-table::
+   :widths: 10 90
+   :header-rows: 1
+
+   * - Method
+     - Changes
+   * - ``update_item``
+     - removed in favor of ``update``
+   * - ``rate_limited_scan``
+     - removed in favor of ``scan`` and ``ResultIterator``
+   * - ``delete``
+     - ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``condition`` instead.
+   * - ``update``
+     - ``attributes``, ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``actions`` and ``condition`` instead.
+   * - ``save``
+     - ``conditional_operator`` and ``**expected_values`` kwargs removed. Use ``condition`` instead.
+   * - ``count``
+     - ``**filters`` kwargs removed. Use ``range_key_condition``/``filter_condition`` instead.
+   * - ``query``
+     - ``conditional_operator`` and ``**filters`` kwargs removed. Use ``range_key_condition``/``filter_condition`` instead.
+   * - ``scan``
+     -
+       - ``conditional_operator`` and ``**filters`` kwargs removed. Use ``filter_condition`` instead.
+       - ``allow_rate_limited_scan_without_consumed_capacity`` was removed
+
+
+When upgrading, pay special attention to use of ``**filters`` and ``**expected_values``, as you'll need to check for arbitrary names that correspond to
+attribute names. Also keep an eye out for kwargs like ``user_id__eq=5`` or ``email__null=True``, which are no longer supported. A type check can help you catch cases like these.
+
+PynamoDB 2.x to 3.x
+--------------------
+
+Changes to UnicodeSetAttribute
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+See :ref:`upgrading_unicodeset` for details.

docs/upgrading_binary.rst

@@ -1,3 +1,7 @@
+:orphan:
+
+.. _upgrading_binary:
+
 Upgrading Binary(Set)Attribute
 ==============================
 

docs/upgrading_unicodeset.rst

@@ -1,3 +1,7 @@
+:orphan:
+
+.. _upgrading_unicodeset:
+
 Upgrading UnicodeSetAttribute
 =============================