I have one ADF environment where I developed and tested Pipelines for 2 separate Projects. Branch is same as well for the ADF pipelines. Now I am a stage of setting up CI/CD pipelines for both projects separately.
How should I proceed as I am doing this first time? I just realized that the "adf_publish" branch will be having code for both Projects in one. This is the first time I have to create CI/CD for ADF and I am on a time crunch.
It seems that you have two folders in master branch: one contains pipeline fully developed and tested and other one for in progress pipelines. However, as you correctly pointed out, adf_publish branch will contain the json code for the pipelines in both the folders.
I would recommend you segregate the in-progress pipelines and keep it in a local branch and do not merge until it's tested .
Download the support files of all pipelines under test folder and then delete them from master branch.
Create a new local branch and go to the github repo and upload the pipeline.json files. After fully tested, merge it with the master branch.
Create CICD pipeline for migrating the code in master branch to next environment: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery