Camunda - Our experience

How have we used it?

Camunda is mostly used for orchestrating microservices and managing tasks meant for actual persons. This is exactly what we have used it for also.

In one of our projects we have many small microservices, which all do their own specific tasks and know nothing about each other. This is where Camunda comes in, for example, let’s say we have services for users, orders and invoices. When we want to generate a new invoice, we do not make invoice service call other services to get required data, instead we let Camunda do that. So whenever we get to the invoice generation step in our business process flow, Camunda calls user service to get user data, order service to get order data and finally invoice service with user and order data to create the invoice.

In order to make Camunda do all that, we need to mark the invoice generation task as a service task. Service tasks can be either external or internal. External means that Camunda offers a work item to external workers (some other service, which does not have to be written in Java) who will be polling Camunda for tasks. We have only used internal service tasks as we like to keep the implementations in one place instead of having to search over different repos to find the correct worker. To use an internal service task, we have to create a Java class, make it implement the correct interface and specify that class on the service task in the flow diagram.

There is also a possibility to call other services using REST API directly from the workflow instead of having Java class do it, but we have not used that functionality for the sake of consistency and always use Java delegates to implement service task logic.

In addition to service tasks, one of the most used task type is user task, which, as the name suggests, is a task that requires user to do something. When coming back to the same example we used to illustrate microservices and service tasks, then a user task would be a step in business flow where the order has to be consolidated and prepared for shipping. We have a separate back-office application running for employees who have to handle tasks like this. That application listens to a RabbitMQ messaging queue for new tasks and if a task is created, it is displayed to the users. Those tasks can be assigned and unassigned, postponed, completed, queried etc via Workflow Engine’s REST API. This makes it easy to manage the tasks so that the users do not have to keep track who is doing which task and so on. They just claim the task to themselves, do whatever is necessary to complete the task and finally complete it. If the task is completed, the Workflow Engine will continue with the flow and move on to the next task. It is possible to define which users/groups can handle which tasks, for example customers deal only with tasks for placing an order, while employees handle all other tasks.

There are also multiple events and gateways, such as timers, messages and conditional gateways to make the process behave as required, for example send an email to the customer to continue with the order or to decide if a task is required or not. A simple flow diagram to describe the same order flow would look something like this:

Why do we like using it?

Camunda helps to get business processes quickly in place without actually having to code anything. This could lead to discovering problems or bottlenecks in the very early stages, which in turn leads to saving time and money as nothing has been implemented yet. This is a good way for business people to describe how they see their application should be used. Once the initial flow is agreed and implementation has started, BPMNs will start to act as documentation, which is always up to date and non-technical people can have overview of the current flow at any point.

Adding new tasks and functionalities to the flow is rather easy, all you have to do is make sure that the flow is correct from the business point of view and from there it is just a matter of drawing correct elements on the flow and writing the implementation in case of service tasks.

Camunda has many different built-in features, which saves a lot of time for developers as they do not have to develop these functionalities themselves. Such features include timers and messages to control the process, auto retry to run failed service tasks again, Java API for easy communication between Workflow Engine and the application, human task management to build applications for users etc. One feature that we have found really useful is REST API. We mostly use it to make other services communicate with Camunda, but it also comes in handy when debugging, for example when a process instance seems not to go through the flow as expected. The REST API and it's documentation is very thorough, we have almost always found the endpoints and functionality that we need.

Camunda also has a user interface for their Workflow Engine (Cockpit), where it is easy to see what version of the flow is deployed and in which step a process instance currently is. It has almost the same functionality available as REST API to manage tasks, processes, deployments etc. If one of the process instances would have an incident, then the Cockpit gives an initial overview of what happened and usually it is enough to fix the problem, reducing time spent digging through logs trying to find the correct entry.

Last but not least, Camunda has an enterprise version, which comes with more advanced Cockpit, allowing users to do more things using UI instead of Rest API. Paid version has tools for reporting and analyzing the flow to optimize and improve it. We have not used the enterprise version as the community version meets all our current requirements.

What do we not like about Camunda?

One pretty big problem is that whenever flow is updated and deployed, all processes running in the previously deployed flow will still be there and not in the latest flow. To give you an example, let’s say that if the order was successfully paid for, we want to save the payment details for future usages and add a service task for it between “Pay for the order” and “Generate invoice” tasks so the flow looks like this:

Now we have two different deployments (flows) and three possible states of order that we have to think about:

1. Order that does not have a process instance yet. That is the simplest case and does not have any issues. Whenever a new process is started in Camunda, it will always be in the latest flow.
2. Order that is in the old flow, but has not completed the “Pay for the order” task yet. If it would continue the flow, it would not save the payment method as the flow, where it currently is, does not have that task. While it is technically possible to migrate the application from one deployment to another via REST API, in our experience it is not always required, because it is not worth the trouble as the migration is not as simple as one would think and we often accept the fact that if the same user creates another order, then the payment details have to be inserted again. And since the new order is created in the new flow, then that time the system would save the payment details for the future.
3. Order is in the old flow and already past the “Pay for the order” task. Let’s say that the order is ready for shipping and the process is in the “Ship the order” task. In this case there is no way to migrate the process to the new flow as it has to be moved to the “Save payment details” task and from there it automatically goes to “Consolidate order” task, which is actually already completed for that order, so the migration would create a mess where system says one thing, but the reality is something else.

For those reasons and because we know for sure that previous deployments will become unused after some specific period of time, we have decided not to migrate applications unless there is no other way. The good thing is that this affects only the flows, if changes were done in the Java delegates, then those changes will apply to all process instances running that delegate as Camunda does not version the code as it does with the diagrams.

Another downside is that code that seems unused is not always unused. Let’s imagine the opposite situation where we used to save payment details and decided not to anymore, so we remove the “Save payment details” service task, which in turn, makes the delegate for that task obsolete, so we remove that as well. And now we have a problem, because once again we have to think about the old deployments. Remember the good thing from the last paragraph about Camunda not versioning code? Not so good anymore as old deployments still require that delegate. We solved this by marking the delegate deprecated and every once in a while go over old deployments and delete the ones that have no active process instances and then delete deprecated code which really is unused now.

To conclude the bad things, the root of all evil is that developers have to always keep in mind that there are older deployments and think about how the new changes affect them. It is quite challenging and easy to forget in the beginning, but will come automatically later on.

Would we use it again in some other project?

The answer is the same as for any other tool. If it makes sense to use it, use it. Definitely there is no point to use it on a giant monolithic application which has no human tasks and there is no strict flow required for the system to work. The project we used Camunda for, had microservices and strict flow with user tasks and if our next project has them too, then we will strongly consider using Camunda to help us achieve great results.

---

Solutional is an agile software development company which has a team of professional engineers who are able to solve all software problems from beginning to the end without any middlemen.

Contact us at info@solutional.ee in case you have any new or existing projects needing help with successful execution.

---