How to specify wild card paths while reading a file in U-SQL

79 views Asked by At

I am using below code to read a file from Azure data lake store (ADLS Gen1)using U-SQL

DECLARE @InputFile string = 2021/2021-Mar/*/{*}.json";    
@json =
EXTRACT
col1 int?,
col2 string    
 FROM @InputFile
USING new Microsoft.Analytics.Samples.Formats.Json.JsonExtractor("data[*]");

The 2021 folder has data for different months and I want to read data from all months from March-October (except Jan and Feb). I was looking for a regular expression for the same. I have used the below one, but didn't work

2021//{2021-Mar,2021-Apr,2021-May,2021-Jun,2021-Jul,2021-Aug,2021-Sep,2021-Oct/}/*/{*}.json"
1

There are 1 answers

1
Utkarsh Pal On

A double quote is missing from the DECLARE statement in your code. Also, you need to remove the extra * before {*} and it should work. Find the example below:

DECLARE @InputFile string = "2021/2021-Mar/{*}.json";