AWS Glue - CRON Scheduled Trigger

1.5k views Asked by At

I have been trying to find a Cron expression to use in my AWS Glue Job. I tried many examples and I'm still not sure if this is possible or NOT, since this is the first time I'm adding time schedule to a Glue Job, in a Cloud Formation template.

We are trying to schedule the Glue Job to run in the 5th or 6th business day of each month at 10:15AM.

I tried some ideas, but none gave consistent results and many cron validators show them as incorrect. Tried:

  • 0 0 ? * 3#3 *
  • 0 0 3-7 * ? *

Examples of what I'm trying to achieve:

5th Business day of the Month:

  • If the Job ran in January 2023 it should run on January 6th.
  • If the Job ran in December 2022 it should run on December 7th.
  • If the Job is schedule for ran in July 2023 it should run on July 7th.

6th Business day of the Month:

  • If the Job ran in January 2023 it should run on January 9th.
  • If the Job ran in December 2022 it should run on December 8th.
  • If the Job is schedule for ran in July 2023 it should run on July 10th.

Remember that is to be used in the Cloud Formation AWS Glue Template.

2

There are 2 answers

0
Rishabh Sahrawat On

In AWS (atleast), you can not set a cron expression to match "business/working" days. For cron expression there are no "business" days but only calender days.

As you mentioned, you can set a Lambda function to check if the day is a working day and then trigger the Gluen job.

0
Murray On

Cron doesn't support this.

The simplest solution would be to use the cron schedule to cover the possible days e.g 15 10 5-10 * 1-5 and then have Python code at the start of the Glue job that continues on or exits based on your specific business day logic.