Camunda and .NET Core – friends or foes?

Our teams, at Altkom Software & Consulting, are using the Camunda BPM platform successfully in many projects for over 5 years. We designed, built, and deployed solutions based on Camunda in the financial and insurance industries allowing our customers to digitize their business and deliver business value faster, directly to their customers. Five years ago Camunda was a novelty, now it is well known and established platforms, one of the leaders in the workflow automation area, but most of the projects we observed were using Camunda only together with Java. But Camunda is an excellent platform with REST-based APIs, so why not try to take advantage of it in projects based on other platforms, like .NET Core for example.

In this article, we are going to present the design of a sample solution based on Camunda and implemented in .NET Core and walkthrough typical tasks involved when using Camunda, like: starting a process instance, getting a list of users’ tasks, completing a task, handling a service task.
Full source code for this article can be found at our ASC LAB GitHub repository.

Camunda BPM Platform – What is it?

Camunda BPM platform is a lightweight, open-source, platform for Business Process Management. It supports several standard notations for defining business processes, among which BPMN is the most popular. It consists of the following main components:

There is the Modeler, a graphical tool for defining workflows. The next element is the heart of the whole platform – Engine. It manages executions of business processes based on definitions. Engine functionality is available through Java API and REST API.
There are also client applications: Admin allowing users to manage security and settings, Cockpit that allows users to monitor engine activity and view information about each business process managed by the engine and Tasklist, which provides a simple user interface for managing users’ tasks in business processes.

You can find out more in the introductory section of official Camunda docs. Also if you want to have a quick introduction to Camunda, there is an excellent series of video tutorials available here.

Our Project

Our project aims to support a fictional Hire Heroes company in their business of allowing customers to hire superheroes with a given set of superpowers. The diagram below presents the business process of hiring a superhero using BPMN notation.

The process is started by a customer using Customer Portal. The customer sends an order with information about what superpower is needed and for which period. The first step in the process is an offer preparation by a salesman. Note that tasks for humans are depicted as boxes with a little user icon. Salesman uses Sales Portal to view the list of their tasks. When they see a task for offer preparation they must choose one of the available superheroes or reject an order due to a lack of free superheroes. If the offer is created then the next stop is a task for a customer to review and accept or reject and offer. When the user rejects the offer, then the process finishes. If the offer is accepted then we have a so-called service task to create an invoice. Service tasks are the ones with little cogs icon and they are tasks to be carried out not by humans but by computer systems. In our case, our project will subscribe to the process engine to get a notification when there is a need to generate an invoice. Once the invoice is generated and sent to the client, the process will wait for payment confirmation. In our sample solution, this step is performed manually by the salesman in the Sales Portal by a salesman. When payment is confirmed in the application, a message is sent to the process engine to complete the process. If payment is not confirmed within 5 days the process engine will execute the Cancel Order task, so the whole order and related hero assignment are canceled.

To support this process we designed a solution with the following components

Solution components:

Customer Portal is a single-page Angular application. It allows customers to log in, view their orders and tasks, review offers, and accept or reject it, view notifications, and monitor orders status. It communicates via REST HTTP with SSO and HeroesForHire microservices.

Sales Portal is also a single-page Angular application. It allows salesmen to log in, view their tasks based on the information from Camunda processes and HeroesForHire service, create offers, notify customers that no heroes are available for their orders, view invoices, and confirm payments. It communicates via REST HTTP with SSO and HeroesForHire microservices.

SSO is a simple authentication service that is used for the login process. It has a hard-coded list of users and passwords. When the user is successfully authenticated it returns JWT token with given user roles and in case of a customer additional information that identifies the user’s company.

HeroesForHire is our main application service. It provides an API that both portals use to manage users’ tasks, manage orders, offers, invoices, notifications, and heroes assignments.
It is a typical .NET Core Web API application written in C#. It uses Entity Framework Core for data access. Its internal implementation follows CQS principles and Domain-Driven Design tactical patterns. All API functions are exposed from Web API controllers, while their implementations are nicely encapsulated in command/query handlers with a little help of the excellent MediatR library. For interactions with Camunda two additional external libraries are used: Camunda REST API Client and Camunda Worker – we will discuss them in more detail later. Both of these libraries wrap Camunda REST APIs with C# APIs and their help greatly simplifies the way we can interact with Camunda from .NET Core app.

Camunda Engine is an instance of Camunda BPM platform running in a Docker container, with its REST API on the top and Cockpit, Admin, and Tasklist applications.

There are also two databases involved: PostgreSQL database used by our main service HeroesForHire and H2 in-memory database used by Camunda Engine.

Starting Camunda

Our first task is to start the Camunda BPM Platform. There are many ways to do this, but if we want to avoid dealing with Java, then using Docker is the best solution. We are going to use the official Camunda image available at the Camunda GitHub repository. There are many options you can customize but for demonstration purposes, we can safely leave all the default settings.
As a part of our project, we prepared a docker-compose file that allows us to run Camunda and also Postgresql database, which we will use in our microservices. You can find it here.

version: "2.4"
services:
   camunda:
       image: camunda/camunda-bpm-platform:latest
       ports:
           - 8080:8080
  
   dbef:
       container_name: camunda-dotnetpoc-db1
       image: postgres:12.0-alpine
       volumes:
           - pg-data-camundadotnetpocdata1:/var/lib/postgresql/data
           - ./DbScripts/dbschema:/docker-entrypoint-initdb.d
       ports:
           - 5435:5432
       environment:
           POSTGRES_DB: camunda_poc_db
           POSTGRES_USER: lab_user
           POSTGRES_PASSWORD: lab_pass
  
volumes:
   pg-data-camundadotnetpocdata1:

Now we can use docker-compose to run our infrastructure. Once we run:

docker-compose -f infrastructure.yml up

we can open the following URL in a browser: http://localhost:8080/camunda/ and login as user demo with password demo.

You can now explore Camunda Cockpit, Admin, and Tasklist application. You can read more about it in official Camunda docs.
Camunda REST API is available at http://localhost:8080/engine-rest, we will need that information later to set up communication between our .NET Core microservice and Camunda engine.

Deploying our process

Now it is time to start building our microservice and connect it to Camunda instance. The first thing we need to do is to deploy process definition to Camunda.
We put hire-heroes.bpmn file with the process definition as part of our .NET project.

We need to use Camunda REST API to deploy it and we need to do it as part of the application startup. For the first task, we could use HttpClient or a library like Refit but there is already a library that wraps Camunda REST API in .NET API.

We need to add Camunda.Api.Client dependency to our project.

Install-Package Camunda.Api.Client -Version 2.5.0

The solution for the second part of our task is to create a hosted service and register it in our Startup.cs. Our hosted service is pretty simple, you can find its code here. The most important step is a call to the BpmnService DeployProcessDefinition method. This method is presented below

public async Task DeployProcessDefinition()
{
   var bpmnResourceStream = this.GetType()
       .Assembly
       .GetManifestResourceStream("HeroesForHire.Bpmn.hire-heroes.bpmn");
  
   try
   {
       await camunda.Deployments.Create(
           "HireHeroes Deployment",
           true,
           true,
           null,
           null,
           new ResourceDataContent(bpmnResourceStream, "hire-heroes.bpmn"));
   }
   catch (Exception e)
   {
       throw new ApplicationException("Failed to deploy process definition", e);
   }
}

It loads our bpmn file definition from the assembly resource stream and calls the Camunda REST API method to deploy it. One it is done you can go to the Camunda Cockpit, select Processes from the top menu, Hire Hero should appear in the list, once you click on it you should see process definition and deployment information.

Once we have our process definition deployed we can start implementing our first functionality – starting a workflow instance.
The important piece of information here is the definition key. Its value is Process_Hire_Hero. We will need this value to start instances of this process or to find instances or tasks related to it.

Starting a process instance

In our process, a workflow instance is started when a customer submits an order. The entry point for this functionality is OrderController class and its PlaceOrder method. It delegated the whole processing to PlaceOrder.Handler class.

Let’s see how order’s submission is handled in our service:

public async Task Handle(Command request, CancellationToken cancellationToken)
{
   var newOrder = new Order
       (
       db.Customers.First(c=>c.Code==request.CustomerCode),
       db.Superpowers.First(s=>s.Code==request.SuperpowerCode),
       DateRange.Between(request.OrderFrom,request.OrderTo)
       );

   db.Orders.Add(newOrder);
   await db.SaveChangesAsync(cancellationToken);

   var processInstanceId = await bpmnService.StartProcessFor(newOrder);
   newOrder.AssociateWithProcessInstance(processInstanceId);
   await db.SaveChangesAsync(cancellationToken);
  
   return Unit.Value;
}

We create an instance of Order domain object, save it into the database, then we use previously mentioned BpmnService class to start an instance of Camunda workflow, and finally, we save an identifier of this process instance in our domain object. We will need this in order to correlate data in our domain objects with tasks in the Camunda process.

Let’s see how the process instance is started using Camunda API

public async Task StartProcessFor(Order order)
{
   var processParams = new StartProcessInstance()
       .SetVariable("orderId", VariableValue.FromObject(order.Id.Value.ToString()))
       .SetVariable("orderStatus", VariableValue.FromObject(order.Status.ToString()))
       .SetVariable("customerCode", VariableValue.FromObject(order.Customer.Code));

   processParams.BusinessKey = order.Id.Value.ToString();
  
   var processStartResult = await
       camunda.ProcessDefinitions.ByKey("Process_Hire_Hero").StartProcessInstance(processParams);

   return processStartResult.Id;
}

Here we create a set of process variables, these will be sent to Camunda and saved there as a state of the newly created process instance. Finally, we call StartProcessInstance method passing these variables, and passing the process definition key Process_Hire_Hero.

If we run the Customer portal app (instructions on how to run our sample project can be found in the README file), log in as Tim.Client with password pass, go to New Order option and submit a new order:

And then refresh your process list in Camunda, you should see that now, there is 1 instance of our process running. You can get to the details and see where the process is at the moment

As you can see the process is stopped at a user task called Prepare Offer. Now it’s time to learn how we can get a list of tasks for our users, how to assign it to users, and how to complete it.

Working with users’ tasks

Next step in our process in offer creation. Now it’s time to log into Sales Portal as Jim.Salesman with password pass. Once logged in the user can see a list of tasks assigned to her/him or to his/her group.

The entry point of this functionality is SalesmanController and its method MyTasks. It passes control to GetSalesmanTasks.Handler class.

public async Task<ICollection> Handle(Query request, CancellationToken cancellationToken)
{
   var tasks = await bpmnService.GetTasksForCandidateGroup("Sales", request.SalesmanLogin);
   var processIds = tasks.Select(t => t.ProcessInstanceId).ToList();

   var orders = await db.Orders
       .Include(o=>o.Customer)
       .Include(o=>o.Superpower)
       .Where(o => processIds.Contains(o.ProcessInstanceId))
       .ToListAsync(cancellationToken: cancellationToken);
   var processIdToOrderMap = orders.ToDictionary(o => o.ProcessInstanceId, o => o);

   return (from task in tasks
       let relatedOrder = processIdToOrderMap.ContainsKey(task.ProcessInstanceId) ? processIdToOrderMap[task.ProcessInstanceId] : null
       select TaskDto.FromEntity(task, relatedOrder))
       .ToList();
}

We call BpmnService to get tasks where the candidate group is “Sales” or task is assigned to currently logged salesman. Candidate group is a property set on user tasks in the process definition – for example “Prepare Offer” tasks candidate group property is set to “Sales”.

Then we load Order domain objects from our service database that have associated BPM process instance id equal to one of the ids of process instances, from tasks we get in the first step. Finally, we combine data from domain objects and from Camunda tasks to display a list of tasks:

Here is the code that uses Camunda API to get a list of tasks assigned to a given user or to a given candidate group.

public async Task<List> GetTasksForCandidateGroup(string group, string user)
{
   var groupTaskQuery = new TaskQuery
   {
       ProcessDefinitionKeys = { "Process_Hire_Hero" },
       CandidateGroup = group
   };
   var groupTasks = await camunda.UserTasks.Query(groupTaskQuery).List();

   if (user != null)
   {
       var userTaskQuery = new TaskQuery
       {
           ProcessDefinitionKeys = { "Process_Hire_Hero" },
           Assignee = user
       };
       var userTasks = await camunda.UserTasks.Query(userTaskQuery).List();

       groupTasks.AddRange(userTasks);   
   }
  
   return groupTasks;
}

Once the user is presented with a list of tasks, he or she must claim the task before starting to work with it.

Claiming a task is pretty simple. It requires just one call to Camunda REST API. Below you can find code from BpmnService class that assigns a task to a given user.

public async Task ClaimTask(string taskId, string user)
{
   await camunda.UserTasks[taskId].Claim(user);
   var task = await camunda.UserTasks[taskId].Get();
   return task;
}

Once we have a task assigned, we can complete it. In our case, a user opens an appropriate form that will allow him or her to create an offer, or when no heroes are available, inform the customer about such a situation.

Once the user chooses an available hero that is capable of providing the required superpower he or she submits a new offer. Let’s see how it is handled in our microservice. The processing is started in OrderController CreateOffer method and then goes to CreateOffer.Handler class.

public async Task Handle(Command request, CancellationToken cancellationToken)
{
   using var tx = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled);
  
   var order = await db.Orders.FirstAsync(o => o.Id == request.OrderId, cancellationToken);
   var candidateHero = await db.Heroes.FirstAsync(h => h.Id == request.SelectedHero, cancellationToken);

   var availableHeroes = await db.FindHeroForOrder(order);
   var heroAvailable = availableHeroes.Any(h => h.Id == candidateHero.Id);
   if (!heroAvailable)
   {
       throw new ApplicationException(($"Hero {candidateHero.Name} not available!"));
   }
  
   order.CreateOfferWithHero(candidateHero);
   await db.SaveChangesAsync(cancellationToken);
   await bpmnService.CompleteTask(request.TaskId, order);
   tx.Complete();
  
   return Unit.Value;
}

We load the Order domain object, we load candidate hero and ensure that he or she is available for a mission, we create an instance of Offer domain object and connect it to Order, save it all in the database, and finally, we use BpmnService to complete the task.
Completing a task via Camunda API is done in the following way

public async Task CompleteTask(string taskId, Order order)
{
   var task = await camunda.UserTasks[taskId].Get();
   var completeTask = new CompleteTask()
       .SetVariable("orderStatus", VariableValue.FromObject(order.Status.ToString()));
   await camunda.UserTasks[taskId].Complete(completeTask);
   return task;
}

Here we update the process variable “orderStatus” and send it together with a request to complete the task with a given id. We can now log into the Camunda Cockpit and ensure that the process has moved to the next step, which is an offer’s review by the customer.

Offer acceptance or rejection is handled in a similar fashion to offer creation, you can find related code in AcceptOffer and RejectOffer classes.

Working with external service tasks

Once the customer accepts an offer, it is time to create an invoice for our hero’s services.
This is done using Camunda’s external service tasks. With this approach, Camunda does not actively call external service, but it creates a task in its internal database and external service query for new tasks to perform. Once an external service gets a new task, it locks it by the appropriate calling Camunda API, so no other service would take the same task. Finally, when external service performs the task, it informs Camunda by calling another API method. You can read more about this mechanism in Camunda documentation.
We could create another HostedService for this purpose and implement the whole process there, but it seems a bit complex. There are official client SDKs for external tasks for Java and Node provided by Camunda, but none for .NET Core. Fortunately, there is a NuGet package that solves our problem. Let’s add it to our solution.

Install-Package Camunda.Worker -Version 0.9.0

With its help, it is extremely easy to create handlers for external tasks. All we have to do is create a class derived from ExternalTaskHandler, attribute it with [HandlerTopics] and initialize the whole thing somewhere in the Startup class code.
In our solution, all stuff related to Camunda is initialized in BpmnInstaller class. Let’s see how service task client is initialized

services.AddCamundaWorker(options =>
{
   options.BaseUri = new Uri(camundaRestApiUri);
   options.WorkerCount = 1;
})
.AddHandler()
.AddHandler();

Here we specify the url for Camunda REST API, count of worker threads, and register our handlers.

Now we can have a look at how the actual handler for invoice creation is implemented.

[HandlerTopics("Topic_CreateInvoice", LockDuration = 10_000)]
public class CreateInvoiceTaskHandler : ExternalTaskHandler
{
   private readonly IMediator bus;

   public CreateInvoiceTaskHandler(IMediator bus)
   {
       this.bus = bus;
   }

   public override async Task Process(ExternalTask externalTask)
   {
       var invoiceId = await bus.Send(new CreateInvoice.Command
       {
           OrderId = new OrderId(Guid.Parse(externalTask.Variables["orderId"].AsString()))
       });

       return new CompleteResult
       {
           Variables = new Dictionary<string, Variable>
           {
               ["invoiceId"] = new Variable(invoiceId.Value.ToString(), VariableType.String)
           }
       };
   }
}

The first important thing here is linking between our handler and external task defined in the Camunda process. This is done by specifying a topic. In our case the topic is Topic_CreateInvoice. If you open process definition in a modeler and select “Create Invoice” task then open the properties tab, you can find it there.

Handler implementation is pretty simple, it is similar to the approach we have in controllers – we just create a command and send it using MediatR. The only interesting thing here is that we get order id for which invoice should be created from the process instance variables, and when our command is handled, we send a newly created invoice id together with confirmation, that the task was successfully handled. The actual handler does nothing special, it just loads order and creates an invoice based on our hero daily rate.

public async Task Handle(Command request, CancellationToken cancellationToken)
{
   using var tx = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled);
   var order = await db.Orders.FirstAsync(o => o.Id == request.OrderId, cancellationToken);
   var invoice = Invoice.ForOrder(order);
   db.Invoices.Add(invoice);
   await db.SaveChangesAsync(cancellationToken);
   tx.Complete();
   return invoice.Id;
}

Once the invoice is created and Camunda is notified about it, the process instance waits for confirmation of payment. If no payment is confirmed for 5 days the whole order is canceled.

Sending messages to Camunda engine

Our process waits in Camunda, in order to push it forward once the payment information is received we need to send a correlation message to our process instance. From the process definition perspective, it looks like this:

Let’s see how to implement this part in our microservice. In our sample, the payment confirmation step is performed in the Sales Portal.
The user selects an invoice for which payment was received and mark it as paid.

The entry point for this functionality is OrderController class and its MarkInvoicePaid method. It delegates the actual processing to MarkOrderPaid.Handler class.

public async Task Handle(Command request, CancellationToken cancellationToken)
{
   using var tx = new TransactionScope(TransactionScopeAsyncFlowOption.Enabled);
   var invoice = await db.Invoices.FirstAsync(i => i.Id == request.InvoiceId, cancellationToken);
   invoice.MarkPaid();
   await db.SaveChangesAsync(cancellationToken);
   await bpmnService.SendMessageInvoicePaid(invoice.Order);
   tx.Complete();
   return Unit.Value;
}

Here we load an invoice, mark it as paid, save changes to the database, and send a correlation message to Camunda. Below is the code that sends the actual message.

public async Task SendMessageInvoicePaid(Order order)
{
   await camunda.Messages.DeliverMessage(new CorrelationMessage
   {
       BusinessKey = order.Id.Value.ToString(),
       MessageName = "Message_InvoicePaid"
   });
}

Once Camunda receives our message the process moves on. In our case, this was the last step and the process instance is finished. Our job is finally done, we handled the whole business process for managing orders for superheroes services from the start, when the client submits an order, through offer creation and acceptance, invoice creation, and payment confirmation. Let’s sum up our little trip with Camunda and .NET Core.

Summary

Although there is no official support for building .NET Core applications with Camunda provided by its authors, it is fairly easy to find open source solutions or build your own.
Camunda provides a great, well-documented REST API, that makes it easy to use with any platform that provides HTTP client functionality.
We went through all the major tasks that are typically encountered when developing solutions with Camunda: we started the process instance from .NET Core app, we managed users’ tasks from .NET Core app, we handled service tasks and send correlation messages from our .NET Core app.

I hope this little example encourages you to experiment with Camunda and .NET Core and explore the great potential of Camunda BPM Platform on your own.

Wojciech Suwała
Head Architect