Automate ECS Fargate Scheduled Tasks

Following my previous post on using scheduled tasks on unsupportd regions on AWS, this one will show you how to automate the scheduling of all the tasks using cron expressions.

The post will require you to have Ruby installed and the use of the gem elastic_whenever.

For getting started you can install the gem elastic_whenever, you can either install it locally or on you CI server, you just need to your AWS credentials set.

Then you can create a file where you’ll place all the scheduled tasks (e.g mine is called schedule.rb), and example can be seen below.

every :hour do
  command "bundle exec rails rake_task_that_runs_hourly"
end

every :day, at: "19:00" do
  command "bundle exec rails rake_task_running_daily"
end

every "30 07,14 * * *" do
  command "bundle exec rails a_cron_expressioned_rake"
end

The command that you set for running the task will use the containerOverrides option on ECS, and thus will override the original task command. There are several more options that you can use for configure the scheduled tasks, the ones in the exampel below are related to a rails application.

To schedule all tasks just run elastic_whenever from the terminal (you may need to pass some extra configuration options, like cluster name, any vpc, subnet ids for the scheduling to go through).

After the first run, and when you need to reschedule the tasks, the gem will look for existing rules with a specific identifier, and wipe them all before re-creating them again.

Also note that the gem will need to receive a task definition, so if the tasks you’re scheduling are computationally heavy bear in mind that the selected task definition has enough resources for it.

After you run the command you can check on CloudWatch and see all the rules created there that will start your containers with the specific command.

Yet another disclaimer, be careful to not have tons of tasks that you need to schedule as AWS will only allow 100 cloud watch rules per account.