fix: modification after review of the reference

README.md

@@ -60,7 +60,7 @@ further to fit a particular use case.
 - [reCAPTCHA enterprise](references/recaptcha-enterprise) - A reference for
   API protection against bot leveraging reCAPTCHA enterprise
 - [Firestore Facade](references/firestore-facade) - Reference implementation
-  for a storage/long term caching solution based on Firestore
+  for a long term caching/storage solution based on Cloud Firestore
 
 ## Tools
 

references/firestore-facade/.gitignore

@@ -1,3 +1 @@
-.DS_Store
-node_modules
 AM-SetFirestoreMock.xml

references/firestore-facade/README.md

@@ -6,13 +6,13 @@ from Firebase and Google Cloud.
 Apigee can act as a facade in front of Cloud Firestore, to implement the
 following use cases:
 
-- Long term storage: using Cloud Firestore to cache data on long term
+- Long term storage: using Cloud Firestore to cache documents on long term
 - Data as a Service (DaaS) pattern: some data of a Cloud Firestore database
 (db) are exposed as an API
 
 The use case that is proposed in the Firestore facade reference is the one
 based on long term storage.
-Indeed, in situations where you need a caching mechanim for data,
+Indeed, in situations where you need a caching mechanism for data,
 which must be cached for more than 30 days (max TTL for caching data in
 Apigee X) a storage solution is required.
 Cloud Firestore is the perfect solution to consider in case of long
@@ -21,12 +21,12 @@ term storage needs.
 ## How it works?
 
 Two Apigee **sharedflows** are used as a facade in front of a Cloud Firestore
-db. A **key cache** is used to lookup and populate data into the Cloud
+db. A **cache key** is used to lookup and populate data into the Cloud
 Firestore db.
 
-By default, the key cache is defined as the following:
+By default, the cache key is defined as the following:
 
-```keyCache = base64Encoding(basePath + '/' + pathSuffix)```
+```cacheKey = base64Encoding(basePath + '/' + pathSuffix)```
 
 This can be modified depending on the use case you need to implement
 
@@ -51,6 +51,7 @@ API endpoint (```sf-firestore-facade-lookup-v1``` and
 - [NodeJS](https://nodejs.org/en/) LTS version or above
 - An Apigee organization
 - [Google Cloud Platform](https://cloud.google.com/) (GCP)
+- **plantuml.1.2020.23.jar** (only if you want to execute ```generate-docs.sh```)
 
 This reference leverages Apigee and Cloud Firestore.
 Therefore, it is important to note that:
@@ -99,6 +100,10 @@ With this option, it is not possible to execute functional tests.
 Nevertheless, you can request the Firestore facade API using the
 HTTP client of your choice.
 
+Note that this reference implementation assumes the same GCP Project
+is used for Firestore. If that's not the case, please
+update the ```<URL>``` in the target endpoint.
+
 ## Script outputs
 
 The pipeline script deploys on Apigee X / hybrid two

references/firestore-facade/diagram/sequence-firestore-facade.txt

@@ -19,28 +19,28 @@ b -> fdp: Access the firestore facade api
 note over gfd,ffp: "Apigee API proxy and shared flows acting as a facade in front of Cloud Firestore db" 
 
 fdp -> ffl: Lookup data from the Cloud Firestore db based on base path, path suffix and encoding type (base64 only)
-ffl -> ffl: calculate the document key (key cache)\nkeyCache = encodingType( basePath + pathSuffix)
-ffl -> gfd: Lookup shared flow acting as a facade with Cloud Firestore, using the key cache\ncall is executed using an ID token
-gfd -> gfd: lookup in the Cloud Firestore db using key cache
+ffl -> ffl: calculate the document key (cache key)\ncacheKey = encodingType( basePath + pathSuffix)
+ffl -> gfd: Lookup shared flow acting as a facade with Cloud Firestore, using the cache key\ncall is executed using an ID token
+gfd -> gfd: lookup in the Cloud Firestore db using cache key
 
 opt Data retrieved from Cloud Firestore
     gfd -> ffl: data is retrieved from Cloud Firestore (lookup status)
-    ffl -> ffl: set context variables:\nflow.lookup.iscontent.cached = true \nflow.lookup.content.cached = "<json content retrieved from cache>" \nflow.lookup.status.code = 200
+    ffl -> ffl: set context variables:\nflow.lookup.hit = true \nflow.lookup.content = "<json content retrieved from cache>" \nflow.lookup.status.code = 200
     ffl -> fdp: shared flow response
 end
 
 opt Data is NOT retrieved from Cloud Firestore
     gfd -> ffl: data is not retrieved from Cloud Firestore (lookp status)
-    ffl -> ffl: set context variables:\nflow.lookup.iscontent.cached = false \nflow.lookup.content.cached = "none" \nflow.lookup.status.code = 404
+    ffl -> ffl: set context variables:\nflow.lookup.hit = false \nflow.lookup.content = "none" \nflow.lookup.status.code > 399
     ffl -> fdp: shared flow response
     fdp -> backend: request is forwarded to the backend API
     backend -> fdp: backend response
     fdp -> ffp: Populate data from the Cloud Firestore db based on base path, path suffix and encoding type (base64 only)
-    ffp -> ffp: calculate the key cache\nkeyCache = encodingType( basePath + pathSuffix)
-    ffp -> gfd: Populate shared flow acting as a facade with Cloud Firestore, using the key cache\ncall is executed using an ID token
-    gfd -> gfd: populate backend response in the Firestore db using key cache
+    ffp -> ffp: calculate the cache key\ncacheKey = encodingType( basePath + pathSuffix)
+    ffp -> gfd: Populate shared flow acting as a facade with Cloud Firestore, using the cache key\ncall is executed using an ID token
+    gfd -> gfd: populate backend response in the Firestore db using cache key
     gfd -> ffp: firestore populate status
-    ffp -> ffp: set context variables:\nflow.populate.content.cached = true \nflow.populate.status.code = 200 \nflow.populate.keycache = <keycache> \nflow.populate.documentid = <firestore documentId> \nflow.populate.collectionid = <firestore collectionId>
+    ffp -> ffp: set context variables:\nflow.populate.content = true \nflow.populate.status.code = 200 \nflow.populate.cachekey = <cacheKey> \nflow.populate.extcache.documentid = <firestore documentId> \nflow.populate.extcache.collectionid = <firestore collectionId>
     ffp -> fdp: shared flow response
 
 end

references/firestore-facade/diagram/sequence-firestore-kms-facade.txt

@@ -0,0 +1,60 @@
+@startuml
+
+title "Firestore Facade in Apigee X/hybrid"
+
+actor User as u
+entity "Client App" as b
+entity "Cloud\nFirestore database" as gfd
+entity "Cloud\nKMS" as kms
+box "Apigee API Platform" #LightBlue
+entity "API Proxy\nfirestore-data-proxy-v1" as fdp
+entity "SharedFlow\nsf-firestore-facade-lookup-v1" as ffl
+entity "SharedFlow\nsf-firestore-facade-populate-v1" as ffp
+end box
+participant "Backend" as backend
+
+u -> b: User interaction
+b -> b: App activity
+b -> fdp: Access the firestore facade api
+
+note over gfd,ffp: "Apigee API proxy and shared flows acting as a facade in front of Cloud Firestore db" 
+
+fdp -> ffl: Lookup data from the Cloud Firestore db based on base path, path suffix and encoding type (base64 only)
+ffl -> ffl: calculate the cache key\ncacheKey = encodingType( basePath + pathSuffix)
+ffl -> gfd: Lookup shared flow acting as a facade with Cloud Firestore, using the cache key\ncall is executed using an ID token
+gfd -> gfd: lookup in the Cloud Firestore db using cache key
+
+opt Data retrieved from Cloud Firestore
+    gfd -> ffl: (encrypted) data is retrieved from Cloud Firestore (lookup status)
+    ffl -> ffl: extract the encrypted dek from the response (encDek)
+    ffl -> kms: call cloud kms api to decrypt the encrypted dek
+    kms -> ffl: return the decrypted dek
+    ffl -> ffl: decrypt the encrypted content using the dek (decrypted)
+    ffl -> ffl: set context variables:\nflow.lookup.hit = true \nflow.lookup.content = "<json content retrieved from cache>" \nflow.lookup.status.code = 200
+    ffl -> fdp: shared flow response
+end
+
+opt Data is NOT retrieved from Cloud Firestore
+    gfd -> ffl: data is not retrieved from Cloud Firestore (lookp status)
+    ffl -> ffl: set context variables:\nflow.lookup.hit = false \nflow.lookup.content = "none" \nflow.lookup.status.code > 399
+    ffl -> fdp: shared flow response
+    fdp -> backend: request is forwarded to the backend API
+    backend -> fdp: backend response
+    fdp -> fdp: generate a random encryption key
+    fdp -> fdp: encrypt the content of the response
+    fdp -> kms: call cloud kms api to encrypt the encrypted dek
+    kms -> fdp: return the encrypted dek
+    fdp -> fdp: prepare the content to be cached = encrypted content + encrypted dek (envelope encryption pattern)
+    fdp -> ffp: Populate data from the Cloud Firestore db based on base path, path suffix and encoding type (base64 only)
+    ffp -> ffp: calculate the cache key\ncacheKey = encodingType( basePath + pathSuffix)
+    ffp -> gfd: Populate shared flow acting as a facade with Cloud Firestore, using the cache key\ncall is executed using an ID token
+    gfd -> gfd: populate backend response in the Firestore db using cache key
+    gfd -> ffp: firestore populate status
+    ffp -> ffp: set context variables:\nflow.populate.content = true \nflow.populate.status.code = 200 \nflow.populate.cachekey = <cacheKey> \nflow.populate.extcache.documentid = <firestore documentId> \nflow.populate.extcache.collectionid = <firestore collectionId>
+    ffp -> fdp: shared flow response
+
+end
+
+fdp -> b: JSON response is sent back to the app (200 OK)
+
+@enduml

references/firestore-facade/firestore-data-proxy-v1/apiproxy/policies/AM-SetResponse.xml

@@ -14,7 +14,7 @@
 <AssignMessage name="AM-SetResponse">
     <AssignVariable>
         <Name>message.content</Name>
-        <Ref>flow.lookup.content.cached</Ref>
+        <Ref>flow.lookup.content</Ref>
     </AssignVariable>
     <AssignVariable>
         <Name>message.header.content-type</Name>

references/firestore-facade/firestore-data-proxy-v1/apiproxy/proxies/default.xml

@@ -18,13 +18,13 @@
                 <Name>FC-LookupExternalCache</Name>
             </Step>
             <Step>
-                <Condition>flow.lookup.iscontent.cached = "true"</Condition>
+                <Condition>flow.lookup.hit = "true"</Condition>
                 <Name>AM-SetResponse</Name>
             </Step>
         </Request>
         <Response>
             <Step>
-                <Condition>(flow.lookup.iscontent.cached = "false") and (flow.lookup.status.code = 404)</Condition>
+                <Condition>(flow.lookup.hit = "false") and (flow.lookup.status.code > 399)</Condition>
                 <Name>FC-PopulateExternalCache</Name>
             </Step>
         </Response>
@@ -38,7 +38,7 @@
         <BasePath>/v1/firestore</BasePath>
     </HTTPProxyConnection>
     <RouteRule name="noroute">
-        <Condition>flow.lookup.iscontent.cached = "true"</Condition>
+        <Condition>flow.lookup.hit = "true"</Condition>
     </RouteRule>
     <RouteRule name="default">
         <TargetEndpoint>default</TargetEndpoint>

references/firestore-facade/pipeline.sh

@@ -33,7 +33,6 @@ envsubst < "$SCRIPTPATH"/templates/AM-SetFirestoreMock.template.xml > "$SCRIPTPA
 
 echo "[INFO] Deploying Google Firestore reference to Google API (For X/hybrid)"
 
-#token() { echo -n "$(gcloud config config-helper --force-auth-refresh --format json | jq -r '.credential.access_token')"; }
 TOKEN=$(gcloud auth print-access-token)
 
 SA_EMAIL="apigee-firestore-sa@$APIGEE_X_ORG.iam.gserviceaccount.com"

references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/policies/JS-SetFirebaseCacheVariables.xml → references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/policies/JS-SetFirestoreCacheVariables.xml

@@ -11,6 +11,6 @@
  See the License for the specific language governing permissions and
  limitations under the License.
 -->
-<Javascript name="JS-SetFirebaseCacheVariables">
-    <ResourceURL>jsc://JS-SetFirebaseCacheVariables.js</ResourceURL>
+<Javascript name="JS-SetFirestoreCacheVariables">
+    <ResourceURL>jsc://JS-SetFirestoreCacheVariables.js</ResourceURL>
 </Javascript>

references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/policies/JS-SetFirebaseMockedCacheVariables.xml → references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/policies/JS-SetFirestoreMockedCacheVariables.xml

@@ -11,6 +11,6 @@
  See the License for the specific language governing permissions and
  limitations under the License.
 -->
-<Javascript name="JS-SetFirebaseMockedCacheVariables">
-    <ResourceURL>jsc://JS-SetFirebaseMockedCacheVariables.js</ResourceURL>
+<Javascript name="JS-SetFirestoreMockedCacheVariables">
+    <ResourceURL>jsc://JS-SetFirestoreMockedCacheVariables.js</ResourceURL>
 </Javascript>

references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/resources/jsc/JS-SetFirebaseCacheVariables.js → references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/resources/jsc/JS-SetFirestoreCacheVariables.js

@@ -13,16 +13,16 @@
 
 // has a content been retrieved from firestore cache or not ?
 var isDataRetrievedFromCache = (context.getVariable('servicecallout.SC-Lookup-FirestoreCache.failed').toString().toLowerCase() === 'true');
-// set variable *** flow.lookup.iscontent.cached ***
-context.setVariable('flow.lookup.iscontent.cached',!(isDataRetrievedFromCache));
+// set variable *** flow.lookup.hit ***
+context.setVariable('flow.lookup.hit',!(isDataRetrievedFromCache));
 
-var content = 'none';
+var content = null;
 if ( !(isDataRetrievedFromCache) ) {
     // get content as a string as this is what we want (a JSON stringified content!)
     content = JSON.parse(context.getVariable('firestoreCacheResponse.content')).fields.data.stringValue;
 }
-// set variable *** flow.lookup.content.cached ***
-context.setVariable('flow.lookup.content.cached',content);
+// set variable *** flow.lookup.content ***
+context.setVariable('flow.lookup.content',content);
 
 // set variable *** flow.lookup.status.code ***
 context.setVariable('flow.lookup.status.code',context.getVariable("firestoreCacheResponse.status.code"));

references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/resources/jsc/JS-SetFirebaseMockedCacheVariables.js → references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/resources/jsc/JS-SetFirestoreMockedCacheVariables.js

@@ -11,8 +11,8 @@
   limitations under the License.
 */
 
-// set variable *** flow.lookup.iscontent.cached ***
-context.setVariable('flow.lookup.iscontent.cached','true');
+// set variable *** flow.lookup.hit ***
+context.setVariable('flow.lookup.hit','true');
 
 // mock content
 var content = {
@@ -21,8 +21,8 @@ var content = {
     code: "FIRESTORE-MOCK001"
 }
 
-// set variable *** flow.lookup.content.cached ***
-context.setVariable('flow.lookup.content.cached',JSON.stringify(content));
+// set variable *** flow.lookup.content ***
+context.setVariable('flow.lookup.content',JSON.stringify(content));
 
 // set variable *** flow.lookup.status.code ***
 context.setVariable('flow.lookup.status.code',200);

references/firestore-facade/sf-firestore-facade-lookup-v1/sharedflowbundle/sharedflows/default.xml

@@ -20,15 +20,15 @@
         <Name>AM-Base64EncodePathInfo</Name>
     </Step>
     <Step>
-        <Condition>flow.firestore.demo.enabled != "true"</Condition>
+        <Condition>flow.firestore.mock.enabled != "true"</Condition>
         <Name>SC-Lookup-FirestoreCache</Name>
     </Step>
     <Step>
-        <Condition>flow.firestore.demo.enabled != "true"</Condition>
-        <Name>JS-SetFirebaseCacheVariables</Name>
+        <Condition>flow.firestore.mock.enabled != "true"</Condition>
+        <Name>JS-SetFirestoreCacheVariables</Name>
     </Step>
     <Step>
-        <Condition>flow.firestore.demo.enabled = "true"</Condition>
-        <Name>JS-SetFirebaseMockedCacheVariables</Name>
+        <Condition>flow.firestore.mock.enabled = "true"</Condition>
+        <Name>JS-SetFirestoreMockedCacheVariables</Name>
     </Step>
 </SharedFlow>

references/firestore-facade/sf-firestore-facade-populate-v1/sharedflowbundle/policies/JS-PopulateFirebase.xml → references/firestore-facade/sf-firestore-facade-populate-v1/sharedflowbundle/policies/JS-PopulateFirestore.xml

@@ -11,6 +11,6 @@
  See the License for the specific language governing permissions and
  limitations under the License.
 -->
-<Javascript continueOnError="false" enabled="true" timeLimit="200" name="JS-PopulateFirebase">
-    <ResourceURL>jsc://JS-PopulateFirebase.js</ResourceURL>
+<Javascript continueOnError="false" enabled="true" timeLimit="200" name="JS-PopulateFirestore">
+    <ResourceURL>jsc://JS-PopulateFirestore.js</ResourceURL>
 </Javascript>

references/firestore-facade/sf-firestore-facade-populate-v1/sharedflowbundle/policies/SC-PopulateCache.xml

@@ -17,9 +17,6 @@
             <Headers>
                 <Header name="Accept">application/json</Header>
             </Headers>
-            <QueryParams>
-                <QueryParam name="documentId">{flow.pathSuffix}</QueryParam>
-            </QueryParams>
             <Verb>POST</Verb>
             <Payload contentType="application/json">{"fields": {"data": {"stringValue": "{flow.jsonContentAsString}"}}}</Payload>
         </Set>

references/firestore-facade/sf-firestore-facade-populate-v1/sharedflowbundle/resources/jsc/JS-PopulateFirebase.js → references/firestore-facade/sf-firestore-facade-populate-v1/sharedflowbundle/resources/jsc/JS-PopulateFirestore.js

@@ -13,25 +13,25 @@
 
 // has a content been stored in firestore cache or not ?
 var isDataStoredInCache = (context.getVariable('servicecallout.SC-PopulateCache.failed').toString().toLowerCase() === 'true');
-// set variable *** flow.populate.iscontent.cached ***
-context.setVariable('flow.populate.iscontent.cached',!(isDataStoredInCache));
+// set variable *** flow.populate.success ***
+context.setVariable('flow.populate.success',!(isDataStoredInCache));
 
-var content = 'none';
+var content = null;
 if ( !(isDataStoredInCache) ) {
     // get content as a string
     content = JSON.parse(context.getVariable('firestoreCacheResponse.content')).fields.data.stringValue;
 }
-// set variable *** flow.populate.content.cached ***
-context.setVariable('flow.populate.content.cached',content);
+// set variable *** flow.populate.content ***
+context.setVariable('flow.populate.content',content);
 
 // set variable *** flow.populate.status.code ***
 context.setVariable('flow.populate.status.code',context.getVariable("firestoreCacheResponse.status.code"));
 
-// set variable *** flow.populate.keycache ***
-context.setVariable('flow.populate.keycache',context.getVariable("flow.basePath")+'/'+context.getVariable("flow.pathSuffix"));
+// set variable *** flow.populate.cachekey ***
+context.setVariable('flow.populate.cachekey',context.getVariable("flow.basePath")+'/'+context.getVariable("flow.pathSuffix"));
 
-// set variable *** flow.populate.documentid ***
-context.setVariable('flow.populate.documentid',context.getVariable("flow.pathSuffix"));
+// set variable *** flow.populate.extcache.documentid ***
+context.setVariable('flow.populate.extcache.documentid',context.getVariable("flow.pathSuffix"));
 
-// set variable *** flow.populate.collectionid ***
-context.setVariable('flow.populate.collectionid',context.getVariable("flow.basePath"));
+// set variable *** flow.populate.extcache.collectionid ***
+context.setVariable('flow.populate.extcache.collectionid',context.getVariable("flow.basePath"));

references/firestore-facade/sf-firestore-facade-populate-v1/sharedflowbundle/sharedflows/default.xml

@@ -19,6 +19,6 @@
         <Name>SC-PopulateCache</Name>
     </Step>
     <Step>
-        <Name>JS-PopulateFirebase</Name>
+        <Name>JS-PopulateFirestore</Name>
     </Step>
 </SharedFlow>

references/firestore-facade/templates/AM-SetFirestoreMock.template.xml

@@ -13,7 +13,7 @@
 -->
 <AssignMessage name="AM-SetFirestoreMock">
     <AssignVariable>
-        <Name>flow.firestore.demo.enabled</Name>
+        <Name>flow.firestore.mock.enabled</Name>
         <Value>$IS_FIRESTORE_MOCK_ENABLED</Value>
     </AssignVariable>
     <IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>