In the realm of Azure DevOps, infrastructure management via YAML files has become increasingly prominent. While this shift offers immense power, it can lead to lengthy and often complex azure-pipelines.yaml files. But what if there was a more structured, readable approach?

Today, we’ll unravel the art of breaking down a monolithic azure-pipelines.yaml into distinct files, each representing its own stage.

Why Break Down Your YAML?

Readability: Smaller, function-specific files are easier to read, understand, and maintain, just like your code.
Collaboration: Different teams can own different stages without stepping on each other's toes.
Error Management: Isolating stages helps in pinpointing issues quickly.

Step-by-step Guide to Splitting Your YAML

1. Understand Your Existing Structure

First, take stock of your current azure-pipelines.yaml. How is it structured? What stages exist, and what does each stage entail? I'll use the yaml from our sample project as an example:

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'
  location: 'westeurope'
  configFileName: 'prod.json'
  resourceGroupName: 'azure-devops-yaml-pipeline'
  stagingAppUrl: 'https://bogdan-todo-app-staging.azurewebsites.net/health'


stages:
- stage: build
  jobs:
    - job: BuildSolution
      steps:
      - task: DotNetCoreCLI@2
        displayName: 'Build .NET solution'
        inputs:
          command: 'build'
      - task: DotNetCoreCLI@2
        displayName: 'Create publish artifact'
        inputs:
          command: 'publish'
          publishWebProjects: false
          arguments: '-o $(build.artifactStagingDirectory)'
          zipAfterPublish: false
      - task: PublishPipelineArtifact@1
        displayName: 'Publish artifact'
        inputs:
          targetPath: $(build.artifactStagingDirectory)
          artifact: 'drop'
          publishLocation: 'pipeline'

- stage: test
  jobs:
    - job: RunUnitTests
      steps:
      - task: DotNetCoreCLI@2
        displayName: 'Run unit tests'
        inputs:
          command: 'test'
          projects: '**/*[Tt]est*/*.csproj'

- stage: update_infrastructure
  displayName: 'Deploy Bicep Infrastructure'
  jobs:
    - job: UpdateAzureResources
      steps:
        - task: AzureCLI@2
          displayName: 'Deploy Bicep Infrastructure'
          inputs:
            azureSubscription: 'AzureConnection'
            scriptType: 'pscore'
            scriptLocation: 'scriptPath'
            scriptPath: './infrastructure/deploy.ps1'                
            arguments: '$(resourceGroupName) $(location) $(configFileName)'

- stage: deploy_app
  displayName: 'Deploy To App Service'
  jobs:
    - job: deploy
      steps:
        - task: DownloadPipelineArtifact@2
          displayName: 'Download pipeline artifact'
          inputs:
            buildType: 'current'
            artifactName: 'drop'
            targetPath: '$(Pipeline.Workspace)/drop'
        - task: AzureWebApp@1
          displayName: 'Deploy to staging slot'
          inputs:
            azureSubscription: 'AzureConnection'
            appType: 'webAppLinux'
            appName: 'bogdan-todo-app'
            package: '$(Pipeline.Workspace)/drop'
            deployToSlotOrASE: true
            slotName: 'staging'
            resourceGroupName: $(resourceGroupName)
            runtimeStack: 'DOTNETCORE|7.0'
            startUpCommand: 'dotnet ToDoApp.Server.dll'      
        - task: AzureCLI@2
          retryCountOnTaskFailure: 3
          displayName: 'Check API health before swapping slots'
          inputs:
            azureSubscription: 'AzureConnection'
            scriptType: 'pscore'
            scriptLocation: 'scriptPath'
            scriptPath: './infrastructure/healthcheck.ps1'                
            arguments: '$(stagingAppUrl)'   
        - task: AzureAppServiceManage@0
          displayName: 'Swap slot'
          inputs:
            azureSubscription: 'AzureConnection'
            Action: 'Swap Slots'
            WebAppName: 'bogdan-todo-app'
            ResourceGroupName: $(resourceGroupName)
            SourceSlot: 'staging'

2. Create Distinct YAMLs for Each Stage

In the yaml file above we have 4 stages, so we can create these yaml files:

build.yaml - for building the .NET solution
test.yaml - for running the tests
prod.yaml - for deploying the production infrastructure using Bicep and then deploying the app

Let's create them one by one, and we'll start with the build.yaml file:

stages:
- stage: Build
  jobs:
    - job: BuildSolution
      steps:
      - task: DotNetCoreCLI@2
        displayName: 'Build .NET solution'
        inputs:
          command: 'build'
      - task: DotNetCoreCLI@2
        displayName: 'Create publish artifact'
        inputs:
          command: 'publish'
          publishWebProjects: false
          arguments: '-o $(build.artifactStagingDirectory)'
          zipAfterPublish: false
      - task: PublishPipelineArtifact@1
        displayName: 'Publish artifact'
        inputs:
          targetPath: $(build.artifactStagingDirectory)
          artifact: 'drop'
          publishLocation: 'pipeline'

In the build.yaml file we build the project and then we publish it as an artifact that will be used in the later stages.

The test.yaml file looks like this:

stages:        
- stage: Test
  dependsOn: Build
  jobs:
    - job: RunUnitTests
      steps:
      - task: DotNetCoreCLI@2
        displayName: 'Run unit tests'
        inputs:
          command: 'test'
          projects: '**/*[Tt]est*/*.csproj'

In this stage we are just running the unit tests. At the beginning of the file, you'll observe the use of the dependsOn keyword. This instructs the pipeline to wait for the completion of the Build stage before proceeding.

The last one is production.yaml:

stages:
- stage: Production
  displayName: 'Deploy To Production Env'
  dependsOn: Test
  variables:
    location: 'westeurope'
    configFileName: 'prod.json'
    resourceGroupName: 'azure-devops-yaml-pipeline'
    stagingAppUrl: 'https://bogdan-todo-app-staging.azurewebsites.net/health'
  jobs:
    - job: UpdateAzureResources
      steps:
        - task: AzureCLI@2
          displayName: 'Deploy Bicep Infrastructure'
          inputs:
            azureSubscription: 'AzureConnection'
            scriptType: 'pscore'
            scriptLocation: 'scriptPath'
            scriptPath: './infrastructure/deploy.ps1'                
            arguments: '$(resourceGroupName) $(location) $(configFileName)'
    - job: Deploy
      dependsOn: UpdateAzureResources
      steps:
        - task: DownloadPipelineArtifact@2
          displayName: 'Download pipeline artifact'
          inputs:
            buildType: 'current'
            artifactName: 'drop'
            targetPath: '$(Pipeline.Workspace)/drop'
        - task: AzureWebApp@1
          displayName: 'Deploy to staging slot'
          inputs:
            azureSubscription: 'AzureConnection'
            appType: 'webAppLinux'
            appName: 'bogdan-todo-app'
            package: '$(Pipeline.Workspace)/drop'
            deployToSlotOrASE: true
            slotName: 'staging'
            resourceGroupName: $(resourceGroupName)
            runtimeStack: 'DOTNETCORE|7.0'
            startUpCommand: 'dotnet ToDoApp.Server.dll'      
        - task: AzureCLI@2
          retryCountOnTaskFailure: 3
          displayName: 'Check API health before swapping slots'
          inputs:
            azureSubscription: 'AzureConnection'
            scriptType: 'pscore'
            scriptLocation: 'scriptPath'
            scriptPath: './infrastructure/healthcheck.ps1'                
            arguments: '$(stagingAppUrl)'   
        - task: AzureAppServiceManage@0
          displayName: 'Swap slot'
          inputs:
            azureSubscription: 'AzureConnection'
            Action: 'Swap Slots'
            WebAppName: 'bogdan-todo-app'
            ResourceGroupName: $(resourceGroupName)
            SourceSlot: 'staging'

In this file, we have combined two previous stages: infrastructure deployment and deployment to the app service. The Production stage depends on the Test stage, so if your tests pass then we won't deploy our code to the production environment.

We have also relocated some variables from the azure-pipelines.yaml to this file, making them accessible to all the jobs within the Production stage.

One last thing I should mention is that jobs run in parallel, so I added the dependsOn clause to the Deploy job, this means that we will only deploy after the infrastructure deployment is finished successfully.

3. Reference these files within the main YAML

I will now move these files to a folder called stages and reference them like this in the azure-pipelines.yaml:

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'


stages:
- template: stages/build.yaml
- template: stages/test.yaml
- template: stages/production.yaml

That's it for now! We've successfully split our YAML into individual stages making it cleaner and easier to read and navigate.

The code is public and you can find it here: https://dev.azure.com/bujdea/_git/AzureDevopsYamlPipeline

Conclusion

In conclusion, just as we emphasize writing clean, concise, and readable code, the same principles should be applied to our YAML files in Azure DevOps pipelines. Keeping our YAML configurations compact and straightforward not only ensures better maintainability but also reduces the potential for errors and facilitates collaboration. Let's treat our infrastructure definitions with the same care and attention to detail as our application code, striving for clarity and simplicity in every line.

What's next

In the next article, I will add a new environment to our sample application in 5 simple steps. Feel free to subscribe to the newsletter below or follow me on Twitter if you'd like to be notified as soon as possible!

Azure DevOps Best Practices: Breaking Down the Monolithic YAML

Comments (1)

Azure DevOps

Conditional deployment in Bicep

More from this blog

10 Rules for Writing Production-Ready Code with AI

The Copilot's Log #1: Vibe-Coding vs AI-Assisted Coding

From CLU to Semantic Kernel: Building a Secure, Intelligent Teams Bot

Reinventing the Clobba Teams Bot: How Semantic Kernel Changed the Game

From Hype to Reality: How Semantic Kernel Powers Smarter Apps

Why Break Down Your YAML?

Step-by-step Guide to Splitting Your YAML

1. Understand Your Existing Structure

2. Create Distinct YAMLs for Each Stage

3. Reference these files within the main YAML

Conclusion

What's next

Command Palette

Comments (1)

Azure DevOps

Conditional deployment in Bicep

More from this blog

Why Break Down Your YAML?

Step-by-step Guide to Splitting Your YAML

1. Understand Your Existing Structure

2. Create Distinct YAMLs for Each Stage

3. Reference these files within the main YAML

Conclusion

What's next