Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@cap-js/audit-logging #261

Closed
wants to merge 25 commits into from
Closed
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions guides/data-privacy/assets/Data-Privacy.drawio.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
137 changes: 137 additions & 0 deletions guides/data-privacy/assets/Data-Subjects.drawio.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
170 changes: 170 additions & 0 deletions guides/data-privacy/assets/Transactional-Outbox.drawio.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
658 changes: 658 additions & 0 deletions guides/data-privacy/audit-logging.md

Large diffs are not rendered by default.

58 changes: 58 additions & 0 deletions guides/data-privacy/drm.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
layout: cookbook
shorty: Data Retention Management
synopsis: >
Use the SAP Data Retention Manager (DRM) with a CAP application.
breadcrumbs:
- Cookbook
- Data Privacy
- Data Retention
#status: released
---



# Data Retention Management

Under construction.



<!-- Build that as own guide as soon as it's ready

## Retention Manager

Goal – find out which personal data has to be deleted at a certain point in time.

When?
Find out the correct time for deletion.

What?
Find out the correct amount of data to be deleted.


To support this, we’ll invent some new CDS annotation to mark all possible candidates for 'End of Business' indicating time fields in each legal ground (like Consent, Order etc.).

This new CDS annotation for "End of Business" indicators will serve as input for the retention manager.

An additional configuration at the customer site per type of transactional document defines the actual retention time (like 2 years, 5 years, etc.).
Finally, the retention manager searches all candidates (of natural persons) for possible deletion (across all object types).

Finally, the search results will be cross checked:
Check all legal grounds, if deletion of certain data really is allowed. (One active Legal ground is sufficient to stop the deletion!)

CDS could support this process by building certain queries - based on annotations - to find out which legal ground is invalid at a certain point in time (tt.mm.yyyy) and no other legal ground (of the same type) per person (DataSubject) exists.

Static implementation for such queries already exists. We try to bring this on a dynamic meta-data-driven level with help of CDS annotations and CDS queries.

## Consent Repository

The consent repository is already built with help of CAP and therefore with CDS and with full OData support.

See the [Concent Management Documentation](https://github.../foundation-apps/ConsentManagementDocumentation) for more details.

## Central Business Partner

To reuse the Business Partner from an SAP S/4HANA system, a central Business Partner service is created. If your application makes use of this Business Partner service, you only have to annotate the relation to the Business Partner and your application can make use of the service. In addition, all settings that are necessary to integrate all DPP processes will be performed automatically.

-->
237 changes: 176 additions & 61 deletions guides/data-privacy/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,12 @@ status: released
---
<!--- Migrated: @external/guides/67-Data-Privacy/01-intro.md -> @external/guides/data-privacy/introduction.md -->

# Basics of Data Privacy with CAP
# Basics of Data Privacy

{{ $frontmatter.synopsis }}



## Introduction


Expand All @@ -25,62 +26,193 @@ SAP provides specific features and functions to support compliance regarding the

CAP supports applications in their obligations to comply to data privacy regulations, by automating tedious tasks as much as possible based on annotated models. Using annotations and configurations, CAP supports you using SAP BTP services, which enable you to fulfill specific data privacy requirements in your application. This means at first, personal data management, with the help of annotations and configurations and the SAP Personal Data Manager service.

ADDED FROM GUIDE #4:

Compliance to data privacy regulations is an important requirement for all busines applications nowadays. CAP provides easy ways to designate personal data, as well as out-of-the-box integration with SAP BTP services, like SAP Personal Data Manager service. This greatly relieves application developers these tedious tasks and related efforts.

::: danger _TODO_
linie zu DRM sollte gestrichelt sein
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
line zu DRM sollte gestrichelt sein

Or maybe one of these: Line, linier, linin, linde?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM solute gestrichelt sein

Or maybe one of these: sollas, solutes, solita, solace?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the spelling mistake in "gestrichelt" or add it to the project-words.txt list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM sollte gestrichelt seine

Or maybe one of these: seil, Seine, sean, seen?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
line zu DRM sollte gestrichelt sein

Or maybe one of these: Line, linier, linin, linde?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM solute gestrichelt sein

Or maybe one of these: sollas, solutes, solita, solace?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the spelling mistake in "gestrichelt" or add it to the project-words.txt list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM sollte gestrichelt seine

Or maybe one of these: seil, Seine, sean, seen?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
line zu DRM sollte gestrichelt sein

Or maybe one of these: Line, linier, linin, linde?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM solute gestrichelt sein

Or maybe one of these: sollas, solutes, solita, solace?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the spelling mistake in "gestrichelt" or add it to the project-words.txt list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM sollte gestrichelt seine

Or maybe one of these: seil, Seine, sean, seen?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
line zu DRM sollte gestrichelt sein

Or maybe one of these: Line, linier, linin, linde?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM solute gestrichelt sein

Or maybe one of these: sollas, solutes, solita, solace?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix the spelling mistake in "gestrichelt" or add it to the project-words.txt list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
linie zu DRM sollte gestrichelt sein
linie zu DRM sollte gestrichelt seine

Or maybe one of these: seil, Seine, sean, seen?

:::

<img src="./assets/Data-Privacy.drawio.svg" alt="Data Privacy.drawio.svg" style="zoom:111%;" />

::: tip
DRM integration in progress
:::

<!--
::: danger _TODO_
keep?
:::

<span id="inintroduction" />
-->



### Data Protection and Privacy Requirements

EU regulation etc. -> [Personal Data](https://en.wikipedia.org/wiki/Personal_data)

#### Right of access to personal data

See [Right of access to personal data](https://en.wikipedia.org/wiki/Right_of_access_to_personal_data) -> SAP Personal Data Manager

#### Right to be forgotten

See [Right to be forgotten](https://en.wikipedia.org/wiki/Right_to_be_forgotten) -> SAP Data Rentention Manager



### Addressed Requirements

The most essential requests you have to answer are those in the table below, with the job to be done in response to that given on the right-hand side:

| Question / Request | Discipline |
| ------------------------------------------- | -------------------------------------------------------- |
| *When was personal data stored/changed?* | → [Audit Logging](#audit-logging) |
| *What data about me do you have stored?* | → [Personal Data Management](#sap-personal-data-manager) |
| → "Right of access to personal data" | |
| *Please delete all personal data about me!* | → [Retention Management](#sap-data-retention-manager) |
| → "Right to be forgotten" | |

<br>

::: warning
**PLEASE NOTE:** Full compliance is more than that! <br>
While CAP and SAP BTP services greatly facilitate fulfilling the obligations related to data privacy, there are usually numerous **additional regulations** you have comply to, such as from industry-specific legislation in different countries.
:::



<span id="sdfgew343244" />



## Indicate Personal Data in Your Domain Model { #indicate-privacy }

See full sample in [cloud-cap-samples](https://github.com/SAP-samples/cloud-cap-samples/tree/gdpr/gdpr).

### Base Model

In the remainder of this guide, we'll use this domain model as the base to add data privacy and audit logging.

db/schema.cds
{.sample-label}

```cds
using { Country, managed, cuid } from '@sap/cds/common';
namespace sap.capire.bookshop;

entity Customers : cuid, managed {
emailAddress : String;
firstName : String;
lastName : String;
dateOfBirth : Date;
addresses : Composition of Addresses on addresses.customer = $self;
billingData : Composition of BillingData on billingData.customer = $self;
}

entity Addresses : cuid, managed {
customer : Association to Customers;
street : String(128);
town : String(128);
country : Country;
someOtherField : String(128);
}

entity BillingData : cuid, managed {
customer : Association to Customers;
creditCardNo : String(16);
}

entity Orders : cuid, managed {
orderNo : String(111); // human-readable key
customer : Association to Customers;
personalNote : String;
dateOfOrder : Date;
Items : Composition of many { … }
}
```

### Annotating Personal Data

Let's annotate our data model to identify personal data. In essence, in all our entities we search for elements which carry personal data, such as person names, birth dates, etc., and tag them accordingly. All found entities are classified as either *Data Subjects*, *Data Subject Details* or *Other*.

Use `@PersonalData` annotations to indicate entities and elements in your domain model, which will contain personal data.
::: tip
The best practice is to do that in separate files. <br>
See also: [Using Aspects for Separation of Concerns](../../guides/domain-modeling#separation-of-concerns).

For more details on the `@PersonalData` vocabulary, see [this](https://github.com/SAP/odata-vocabularies/blob/main/vocabularies/PersonalData.md).

::: danger _TODO_
- add BillingData to diagram
- adjust types to `DataSubject`, `DataSubjectDetails`, and `Other (e.g., Transactional Data)`
:::

Let's have a look at our [sample](https://github.com/SAP-samples/cloud-cap-samples/tree/gdpr/gdpr).
<img src="./assets/Data-Subjects.drawio.svg" alt="Data Subjects.drawio" style="zoom:111%;" />

Open the _db/data-privacy.cds_ file, which contains our data privacy-related annotations.
Following the [best practice of separation of concerns](../../domain-modeling#separation-of-concerns), we do that in a separate file `db/data-privacy.cds`:

db/data-privacy.cds
{.sample-label}

```cds
// Proxy for importing schema from bookshop sample
using {sap.capire.bookshop} from './schema';

// annotations for Data Privacy
annotate bookshop.Customers with @PersonalData : {
DataSubjectRole : 'Customer',
EntitySemantics : 'DataSubject'
}
{
} {
ID @PersonalData.FieldSemantics : 'DataSubjectID';
emailAddress @PersonalData.IsPotentiallyPersonal;
firstName @PersonalData.IsPotentiallyPersonal;
lastName @PersonalData.IsPotentiallyPersonal;
creditCardNo @PersonalData.IsPotentiallySensitive;
dateOfBirth @PersonalData.IsPotentiallyPersonal;
}

annotate bookshop.CustomerPostalAddress with @PersonalData : {
annotate bookshop.Addresses with @PersonalData : {
DataSubjectRole : 'Customer',
EntitySemantics : 'DataSubjectDetails'
}
{
Customer @PersonalData.FieldSemantics : 'DataSubjectID';
} {
customer @PersonalData.FieldSemantics : 'DataSubjectID';
street @PersonalData.IsPotentiallyPersonal;
town @PersonalData.IsPotentiallyPersonal;
country @PersonalData.IsPotentiallyPersonal;
}

annotate bookshop.BillingData with @PersonalData : {
DataSubjectRole : 'Customer',
EntitySemantics : 'DataSubjectDetails'
} {
customer @PersonalData.FieldSemantics : 'DataSubjectID';
creditCardNo @PersonalData.IsPotentiallySensitive;
}

annotate bookshop.Orders with @PersonalData : {
DataSubjectRole : 'Customer',
EntitySemantics : 'Other'
} {
customer @PersonalData.FieldSemantics : 'DataSubjectID';
personalNote @PersonalData.IsPotentiallyPersonal;
}
```
It is important to annotate the data privacy-relevant entities as `DataSubject`, `DataSubjectDetails`, or `Other`.

It is important to annotate the data privacy-relevant entities as `DataSubject`, `DataSubjectDetails`, or `Other`.

You can annotate different CDS artifacts, such as entities or fields. The data privacy annotations work on different levels - from the entity level to the field level, as described below.

- The **entity-level annotations** signify relevant entities as *Data Subject*, *Data Subject Details*, or *Other* in data privacy terms, as depicted in the graphic below.

- The **key-level annotations** signify object primary keys, as well as references to data subjects (which have to be present on each object).

- The **field-level annotations** identify elements containing personal data.

### Entity-Level Annotations

Entity-level annotations indicate which entities are relevant for data privacy. The most important annotations are:

<!-- cds-mode: ignore, because it's the same annotation repeated -->
#### EntitySemantics

Entity-level annotations indicate which entities are relevant for data privacy.

```cds
@PersonalData.EntitySemantics: 'DataSubject'
@PersonalData.EntitySemantics: 'DataSubjectDetails'
Expand All @@ -95,70 +227,53 @@ Annotation | Description

::: warning _❗ Data Subject and Data Object_<br>
For each specific personal data operation on a data object (like a Sales Order) a valid data subject (like a Customer) is needed.
The application has to clarify that this link between data object and data subject - which is typically induced by an annotation like
`Customer @PersonalData.FieldSemantics : 'DataSubjectID';` - is never broken. Thus, semantically correct personal data operation logs can only be written on top of a semantical correctly built application.

The application has to clarify that this link between data object and data subject - which is typically induced by an annotation like `Customer @PersonalData.FieldSemantics : 'DataSubjectID'` - is never broken. Thus, semantically correct personal data operation logs can only be written on top of a semantical correctly built application.

Make sure that the data subject is a valid CAP entity, otherwise the metadata-driven automatism will not work.
:::

### Key-Level Annotations

Key-level annotations indicate the corresponding key information.
#### DataSubjectRole

```cds
@PersonalData.FieldSemantics: 'DataSubjectID'
@PersonalData.DataSubjectRole: '<Role>'
```

This key information consists of the `DataSubject` (= Person) and its identifiers and the corresponding personal documents (such as Order, Consent, ...) and its identifiers. The latter is always captured implicitly, so we mainly have to specify the type and the key of the `DataSubject`.

### Field-Level Annotations
Can be added to `@PersonalData.EntitySemantics: 'DataSubject'`. User-chosen string designing the role name to use. Default is the entity name.

Field-level annotations tag which fields are relevant for data privacy in detail.
Example:

```cds
@PersonalData.IsPotentiallyPersonal
annotate Customers with @PersonalData: {
EntitySemantics: 'DataSubject',
DataSubjectRole: 'Customer'
};
```

This allows you to manage the data privacy-related actions on a fine granular level only using metadata definitions with annotations and without any need of implementation.

<div id="field-annos-more" />

<!-- Build that as own guide as soon as it's ready


## Retention Manager

Goal – find out which personal data has to be deleted at a certain point in time.

When?
Find out the correct time for deletion.

What?
Find out the correct amount of data to be deleted.


To support this, we’ll invent some new CDS annotation to mark all possible candidates for 'End of Business' indicating time fields in each legal ground (like Consent, Order etc.).

This new CDS annotation for "End of Business" indicators will serve as input for the retention manager.
### Key-Level Annotations

An additional configuration at the customer site per type of transactional document defines the actual retention time (like 2 years, 5 years, etc.).
Finally, the retention manager searches all candidates (of natural persons) for possible deletion (across all object types).
Key-level annotations indicate the corresponding key information.

Finally, the search results will be cross checked:
Check all legal grounds, if deletion of certain data really is allowed. (One active Legal ground is sufficient to stop the deletion!)
```cds
@PersonalData.FieldSemantics: 'DataSubjectID'
```

CDS could support this process by building certain queries - based on annotations - to find out which legal ground is invalid at a certain point in time (tt.mm.yyyy) and no other legal ground (of the same type) per person (DataSubject) exists.
This key information consists of the `DataSubject` (= Person) and its identifiers and the corresponding personal documents (such as Order, Consent, ...) and its identifiers. The latter is always captured implicitly, so we mainly have to specify the type and the key of the `DataSubject`.

Static implementation for such queries already exists. We try to bring this on a dynamic meta-data-driven level with help of CDS annotations and CDS queries.

## Consent Repository

The consent repository is already built with help of CAP and therefore with CDS and with full OData support.
### Field-Level Annotations

See the [Concent Management Documentation](https://github.../foundation-apps/ConsentManagementDocumentation) for more details.
Field-level annotations tag which fields are relevant for data privacy in detail.

## Central Business Partner
```cds
@PersonalData.IsPotentiallyPersonal
@PersonalData.IsPotentiallySensitive
```

To reuse the Business Partner from an SAP S/4HANA system, a central Business Partner service is created. If your application makes use of this Business Partner service, you only have to annotate the relation to the Business Partner and your application can make use of the service. In addition, all settings that are necessary to integrate all DPP processes will be performed automatically.
This allows you to manage the data privacy-related actions on a fine granular level only using metadata definitions with annotations and without any need of implementation.

-->
::: warning _Warning_
Please see [Audit Logging](./audit-logging.md) for implications before marking data as sensitive.
:::
Loading