Background processing in .NET Core: Hangfire
In the past few weeks I've been using Quartz.NET and Azure Functions for some projects, so given the fact that I also used Hangfire in the past, I thought I'd make a short comparison between these 3 based on my experience. In the next 3 articles you'll find a brief description of what they do, basic setup, pros and cons, and a conclusion. The idea is to give you a few "war stories" that really put these to the test, not just a "hello world" type of article.
An easy way to perform background processing in .NET and .NET Core applications. No Windows Service or separate process required.
Backed by persistent storage. Open and free for commercial use.
I used Hangfire almost 2 years ago for a small .NET Core project. The task was very simple, I had a .NET Core API distributed on multiple machines, and every 2 minutes I had to execute a background job. That job would sync some items in a database, so it was easier for me if it wasn't executed concurrently on every machine. This was easily done with Hangfire, but you'll soon see it came at a cost.
Hangfire could store the jobs in memory, but it could also use a database. In my project I already used MySQL so I configured Hangfire to use the same database. Another cool feature that Hangfire has is the built-in dashboard that they have. You just need a few lines of code in your Startup.cs file to enable it and also add authorization if you want. Instead of making queries in the database you have this nice dashboard, so I really liked this feature.
Hangfire dashboard example taken from their official website
Configuration for Hangfire
Once you do this, you just need to configure your job. For this you'll need a method to pass along to Hangfire and a schedule where you define when or how often that method should be executed. Hangfire is really flexible here, allowing you to create recurring jobs, fire-and-forget jobs, continuations, batches, etc. My app used recurring jobs.
Configuring a job to be executed with Hangfire
In the piece of code above you can see that I first remove that job if it exists, and then I AddOrUpdate my job using a unique ID. I do this because sometimes the job would get stuck and the easiest way to fix it would be to push a new ID. In this way the old job with the old ID would get discarded, and I would have a new one registered for execution.
Why would the job get stuck? I blame this on the way it synchronized jobs. In order to make sure that a job would not run concurrently, it would create a database lock. I'm not a DBA, but we had one in our team, and he was constantly pissed about this. I don't know many details but all the databases that we had were on the same Galera cluster, and Hangfire was causing significant performance issues for all of our apps, not just my API. If you don't believe me, here's a screenshot of a dashboard that our DBA sent me after removing Hangfire, you can tell he wasn't insane:
I honestly tried to search for a fix, and I found many people in the same situation. Eventually I landed on the official Hangfire forum where someone asked the same question and their reply was to switch to Azure Functions or something like this. This is when I decided to stop looking for a solution and remove Hangfire from our app. I would've tried a Lambda function (we were already on AWS), but the DevOps team preferred a Kubernetes cron job. I just had to create a new .NET Core console app which was triggered by the cron job that was executing my piece of code. The result was much better in terms of performance and although it wasn't perfect, we kept this solution.
As I said in the beginning of this post, I used Hangfire ~2 years ago, so maybe they fixed this or they're still suggesting going serverless. Either way, I enjoyed how easy it was to setup and considering that I was just learning .NET Core on a project with a strict deadline, I really appreciated this!