Dapr Data Management

Welcome to the Part 2 of the Dapr Series.

The advantage of Microservices is that each Microservice can choose it's own type of Storage. Through this repo, we will check our services use multiple Data Storing technologies. Also in the process look at Dapr State Management, Secrets Configuration etc.

The main branch, is the code base from Part 1 of the Dapr Series which covers majority of Dapr Building Blocks, and daprDataManagement branch is the refactored code base covering Dapr State Management and other concepts.

This version of the code uses Dapr 1.7

Pre-Requisites to Run the Application

• VS Code
  • with Dapr Extension
  • and with Azure Cache Extension
• .NET Core 3.1 SDK
• Docker installed (e.g. Docker Desktop for Windows)
• Dapr CLI installed
• Access to an Azure subscription

Architecture Overview

In our Part 1 we used ShoppingBasket as one of our API service endpoint, to add items, remove or update our line items in the shopping basket. Also our EventCatalog service used to get Events and Categories from a relational SQL Database.

In this repo, we would be replacing the Shopping Basket service with an Azure Redis Cache for our State Management so that there is no need to persist the data in the database, but need fast access, and also may need data to be expired or be deleted.

Also for our Event Catalog service we would be replacing relational SQL Database with Azure Cosmos DB database.

The Cosmos DB Endpoint, Key, DatabaseName and Azure Redis Cache Key would be retrieved from the Dapr secrets store.

Running the app locally

Once VS Code with Dapr Extension has been installed, we can leverage it to scaffold the configuration for us, instead of manually configuring launch.json.

A tasks.json file also gets prepared by the Dapr extension task.

Follow this link to know more about configuring launch.json and tasks.json

In VS Code go to Run and Debug, and Run All Projects at the same time or Individual project.

All the projects which have build successfully and running, can be viewed in the Call Stack window.

Once the application and side car is running, navigate to address http://localhost:5000 in your preferred browser, to access the application.

We can also apply breakpoint to debug the code. Check this link for more info.

The Darp extension added also provides information about the applications running and the corresponding components loaded for that application.

Dapr Building Blocks Covered

1. State Management

*We would be using Azure Redis Cache for our State Management, but if Azure Subscription is not available, one can use Redis Cache container spun up by Dapr.

Refer this link to know more.

We have an Interface IShoppingBasketService which specifies all capabilities of our Shopping Basket i.e. Add, Update, Get or Remove items from the basket.

Follow this link to check more metadata fields for a Redis Component.

To set up Azure Redis follow this link and make sure to select "Azure" tab.

Once the code has been run via VS Code, we can test the Redis Cache values with the help of Azure Cache Extension.

Initial Run	After Item added to Shopping Basket

2. Secrets Management

Refer this link to know more about about dapr Secrets Management.

The Dapr .Net SDK features a .Net Configuration Provider. It loads specified secrets into underlying .Net Configuration API.

Our application can then reference secrets from the IConfiguration dictionary that is registered in ASP.NET Core dependency injection.

The secrets configuration provider is available from the Dapr.Extensions.Configuration Nuget package. The provider can be registered in the Program.cs of an ASP.NET Web API application.

We use the secrets configuration provider to load our Cosmos DB Endpoint, Key and DatabaseName from secrets.json file, like below:

.ConfigureAppConfiguration(config => 
{
   var daprClient = new DaprClientBuilder().Build();                  
   var secretDescriptors = new List<DaprSecretDescriptor>
   {
       new DaprSecretDescriptor("CosmosDb:Endpoint"),
       new DaprSecretDescriptor("CosmosDb:Key"),
       new DaprSecretDescriptor("CosmosDb:DatabaseName")
  };
  config.AddDaprSecretStore("secretstore", secretDescriptors, daprClient);
})

Now secrets are loaded into the application configuration. You can retrieve them by calling the indexer of the IConfiguration instance in Startup.cs ConfigureServices

services.AddDbContext<EventCatalogCosmosDbContext>(options =>
   options.UseCosmos(Configuration["CosmosDb:Endpoint"],
   Configuration["CosmosDb:Key"],
   Configuration["CosmosDb:DatabaseName"])
);

Troubleshooting notes

• If not able to load Dapr projects when running from VS Code, check if Docker Engine is running, so that it can load all components.
• If using Azure Service Bus as a Pub Sub Message broker make sure to enter primary connection string value for "servicebus" key in secrets.json
• If using Cosmos DB make sure to enter Endpoint and Key in secrets.json file "CosmosDb" section.
• If using Azure Redis Cache make sure to enter Key in secrets.json file "redis" section.
• If mail binding is not working, make sure maildevimage is running. Refer this link for more info.
• For any more service issues, we can check Zipkin trace logs.