Is there any AWS tool does the same job as Heroku Scheduler?

Question

I've built rails app to fetch/scrape data from different resources by running rake tasks. However, I deployed the app to AWS via using Elastic Beanstalk but I am trying to figure out the way how I can automate and schedule the rake tasks on AWS. Any ideas what’s the best tool to do that? I thinking about Data pipeline but many people saying it’s used only for internal tasks, such as database backup and logs. What's your recommendation ?

Answer 1

It is possible to use Amazon data pipeline in this way, however I would not recommend it for several reasons:

• ADP is hard to work with, and it gets even harder when you're trying to use custom scripts
  • poor error reporting
  • confusing or incorrect documentation
• ADP will spin up an instance for each run, and because of the way AWS billing works you'll be charged for the full hour no matter how long the instance is up
• it's much easier and cheaper to just have a small instance with cron running on it than to use ADP unless you are going to use the built-in ETL modules