Cron Jobs on Load Balanced Multi Instance Elastic Beanstalk

By January 23, 2017 November 20th, 2018 Ideas For Your Business

Cron jobs on multi-instance Elastic Beanstalk can be challenging. Our goal is often to have the jobs handled by NodeJS and also guaranteed to run on only one given instance at a time. Sometimes a separate web worker instance makes sense for this, but a lot of times we don’t want a separate server and codebase.

You can specify a cron job to run on Elastic Beanstalk using .ebextensions and a “leader_only” flag, but this is not reliable in a situation where your original leader instance gets terminated.

There are many task queue alternatives that can manage jobs, but these usually require a central data store like redis.

One simple and self-contained strategy is to have the instance itself check if it is the “leader” and if so to run the NodeJS “cron” code. This can be accomplished by using the aws-sdk and adding some permissions to the role that the elastic beanstalk environment is using.

The steps in the code below are:

  1. Get the current instance ID
  2. Get the environment ID
  3. Get the other instances in the environment
  4. Determine if the current instance is “master” based on list order.

Node code:

 

IAM Policy for the EB role.[code]]czo1MDg6XCINCnsNClwiVmVyc2lvblwiOiBcIjIwMTItMTAtMTdcIiwNClwiU3RhdGVtZW50XCI6IFsNCnsNClwiU2lkXCI6IFwiU3RtdDE0MTUyMTJ7WyYqJl19ODMzMDAwXCIsDQpcIkVmZmVjdFwiOiBcIkFsbG93XCIsDQpcIkFjdGlvblwiOiBbDQpcInMzOipcIiwNClwiczM6R2V0KlwiLA0KXCJzMzpMaXN0KlwiLA17WyYqJl19ClwiczM6TGlzdEFsbE15QnVja2V0c1wiLA0KXCJzMzpQdXRPYmplY3RcIiwNClwic3RzOkFzc3VtZVJvbGVcIiwNClwiZWMyOkRlc2NyaWJlVGF7WyYqJl19Z3NcIiwNClwiZWxhc3RpY2JlYW5zdGFsazpEZXNjcmliZUVudmlyb25tZW50UmVzb3VyY2VzXCIsDQpcImF1dG9zY2FsaW5nOkRlc2NyaWJ7WyYqJl19ZUF1dG9TY2FsaW5nR3JvdXBzXCIsDQpcImNsb3VkZm9ybWF0aW9uOkxpc3RTdGFja1Jlc291cmNlc1wiLA0KXCJjb2RlZGVwbG95OipcIg0Ke1smKiZdfV0sDQpcIlJlc291cmNlXCI6IFsNClwiKlwiLA0KXCJhcm46YXdzOnMzOjo6KlwiLA0KXCJhcm46YXdzOnMzOjo6ZGVwbG95bWVudGJ1Y2tldC8qe1smKiZdfVwiLA0KXCJhcm46YXdzOnMzOjo6YXdzLWNvZGVkZXBsb3ktdXMtZWFzdC0xLypcIg0KXQ0KfQ0KXQ0KfQ0KXCI7e1smKiZdfQ==[[/code]

 

This code is based on some work found online including https://gist.github.com/ippeiukai/37e812e49f04ea0f26d84d380d050304

Related Posts

Tony Gutierrez

Tony Gutierrez

Architect - With a career spanning cities such as Boston, Edinburgh, and Madrid, Tony is bringing his many years of web systems development to our mobile projects. Tony’s hobbies include metal detecting and home renovation.