DYI: Scheduling Cron jobs in the multi-server system
https://medium.com/@mateuszpiatkowski/scheduling-cron-jobs-in-the-multi-server-system-afd77afa351b
A while ago I had a pleasure to work on a migration of a big monolithic system to a more fail-safe architecture. One of the important parts of the system was the server-side jobs that were scheduled with Cron. In total there were around 100 commands that were running with a different frequencies. The problem was that now all the system, as well as the Cron jobs, were supposed to run on two load-balanced servers. We had to find a solution not to duplicate the server-side tasks using the new architecture. In the following article I will present the approach we took. I am not saying it is the best one, but it is one of the options that exist and for us it meets all the requirements. Also that is something we haven’t found documented anywhere so I believe it is interesting to share.
Once I encountered the problem described above I started Googling around and found a few out-of-the-box solutions: