How to add a validation in azure data factory pipeline to check file size?

Question

I have multiple data sources I want to add a validation in azure data factory before loading into tables it should check for file size so that it is not empty. So if the file size is more than 10 kb or if it is not empty loading should start and if it is empty then loading should not start. I checked validation activity in Azure Data Factory but it is not showing size for multiple files in a folder. Any suggestions appreciated basically if I can add any python notebook for this validation will also do.

Answer 1

Use GetMetadata under General Activities, then send the result to an If Condition.

You will then need to get the file size from the Dataset.@item().name is the name of the file you want to get the size of.

If you are working with a directory do the following:

Then check the file size of each file.

This is what the ForEach settings looks like. Then you can use @item().name inside the ForEach to get at the file.

The data source will need to have the parameter FileName.

Answer 2

Following GIF shows step by step process on how to achieve the above requirement in ADF.