Writing your own cron service

In this post, I will be sharing my thoughts on how to implement task scheduling utilities like the ubiquitous cron or the popular Quartz library.

Approach 1:

  • We can have a main thread which will monitor the crontab file for any job additions.
  • As soon as there is a new job added, we can fire a new thread dedicated to that job.
  • This is a very simple approach.
  • However, it suffers from performance issues as a machine can only contain so many threads at a time. Moreover, thread creation and deletion is expensive.

Approach 2:

  • We can have a finite thread-pool which will execute all the tasks by picking them up from a PriorityBlockingQueue (thread-safe heap) prioritized on job.nextFireTime().
    Meaning that the top element of this heap will be always be the one that will fire the soonest.
  • We will have one thread which will be running in an infinite loop and submitting new jobs to the thread pool after consuming them from the queue.
  • Lets call it Thread1:
  • There will be one more thread Thread2 which will be monitoring the crontab file for any new job additions and will push them to the queue.

     

  • However, there is a problem with this:
    • Imagine that Thread1 is sleeping and will wake up after an hour.
    • Meanwhile a new task arrives which is supposed to run every minute.
    • This new task will not be able to start executing until an hour later.
  • To solve this problem, we can make Thread2 wakeup Thread1 from its sleep forcefully whenever the new task has to run sooner than the front task in the queue.

In the next blog post, I will be trying to answer these questions:

  • How to recover from system crashes?
  • How to handle all the missed recurring jobs?
  • How to make the scheduling service more resilient by making it distributed?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s