Unable to write the file while using windowing for streaming data use to ":" in Windows

54 views Asked by At

I am encountering an issue when running my Java ApacheBeam code. The program throws an error when trying to create a file with a colon in the name on my Windows system, stating that the file name is invalid.

I understood that the error was due to my windowingTimestamp where it contains : in its timestame as a file name.

I stuck in converting my timestamp value, i dont want to have a basic default file name.

Actual file Name: output2020-12-12T15:00:10.000Z-2020-12-12T15:00:15.000Z-pane-0-last-00000-of-00001

Expected file Name: output2020-12-12T15.00.10.000Z-2020-12-12T15.00.15.000Z-pane-0-last-00000-of-00001

output.apply(ParDo.of(new DoFn<KV<String,Long>,String> (){

     @ProcessElement
    public void processElement(ProcessContext c, BoundedWindow window){

        /*In Terminal Results
        System.out.println(String.format(
                "%s: %s %s", window.maxTimestamp(), c.element(). getKey(),c.element().getValue()));

         */
         c.output(String.format("%s %s %s",window.maxTimestamp(), Objects.requireNonNull(c.element()).getKey(), Objects.requireNonNull(c.element()).getValue()));

    }
})).apply(TextIO.write().to("src/main/resources/Sink/FixedWindow/output").withWindowedWrites().withNumShards(1));
0

There are 0 answers