In a collect stage of my BigQuery model, I have a job that collects every day a customer table in a full refresh mode.
Daily, I update my table a_customers based on the last file that I have on Google Cloud Storage.
Now I want to create a transformation query on dbt that would aggrgate data by month.
- For the previous months, it should calculate data based on the data collected for the last day of the previous month.
- For the current month, it should update the value of aggregation based on the data recieved every day until the month ends.
What's the best day to do that on dbt ?
Thanks in advance !