Create Multi Stage Pipelines with YAML in Azure DevOps

Until recently, Azure DevOps had offered separate build and release views for its users. This was a little different from pipeline features in other CI/CD tools like Jenkins, where if you build a pipeline, it is a single unified experience. With recent update, they have released unified experience for the Multi Stage Pipelines. Now one can use a unified YAML experience and configure Azure DevOps pipelines to do CI, CD, or CI and CD together. This not only allows to control the build configuration as part of the source code but releases as well. In this blog post, we are going to create and work with the same.

Enable Multi Stage Pipeline feature

Since this feature is under preview, as of writing of this blog post, one needs to go to preview features from his user profile, and enable the same by sliding radio bar to the right:

enable multi stage pipelines from preview features

Write a Multi Stage Pipeline

A bit about syntax

Azure DevOps pipelines consists of multiple stages. Each stage describes the part of the CI/CD process. They denote a particular milestone in the CI/CD prcoess for example building source code, run unit tests, etc.

Stages consists of one or more jobs, which are units of works assignable to a build/release agent. Jobs consists of linear series of steps. Each step can be a simple task such as echo or a complex script or some other task referring to 3rd party like manual intervention etc.

To know more, one can read about the Azure DevOps YAML syntax here.

Hello-World YAML Pipeline

Consider below simple hello-world pipeline for demonstration of multi stage pipelines:


stages:
– stage: Build
jobs:
– job: Build
pool:
vmImage: 'Ubuntu-16.04'
continueOnError: true
steps:
– script: echo "hello to my first Build"
– stage: dev_deploy
jobs:
– deployment: dev_deploy
pool:
vmImage: 'Ubuntu-16.04'
environment: 'dev-hello'
strategy:
runOnce:
deploy:
steps:
– script: echo "hello, dev world !!!"
– stage: qa_deploy
jobs:
– deployment: qa_deploy
pool:
vmImage: 'Ubuntu-16.04'
environment: 'qa-hello'
strategy:
runOnce:
deploy:
steps:
– script: echo "hello, qa world !!!"

There are couple of interesting features like deployment and strategy with what was the usual YAML.

Environment Keyword

Environments are a very new feature and they represent the group of resources targeted by a pipeline, for example, Kubernetes clusters, Azure Web Apps, virtual machines, and databases. Environments are useful to group resources, for example, you can group dev resources for your application under an environment named “deployment”, group qa resources for your application under an enviroment named “staging” or “qa” and so on. You are free to name environments according to your choice.

Also, we can view deployments made on a environment using Azure Pipelines:

viewing deployments made on a environment

More capabilities will be added to environments over time, and we’ll cover those under separate blog posts.

Strategy

This impacts the deployment strategy, which defines how your application is rolled out across the cluster. The default strategy is runOnce, but in the future you’ll be able to easily indicate other strategies, such as canary or blue-green. Again, we’ll cover those under separate blog posts.

Create Multi Stage Pipeline

To create a pipeline, go to Azure Pipelines and select ‘new pipeline’:

create a new pipeline from azure pipelines

After this, select one of the option to let it know where the Source code resides:

configure the location of the source code repository

A small YAML icon next to the possible indicates that Azure DevOps will analyze your code and recommend a YAML template that makes sense for you and gets you up and running quickly. Select appropriate option to proceed.

After this, review and edit your pipeline as necessary and then click run to deploy the pipeline into action:

review, edit the pipeline yaml and then run

Run the Azure Pipeline and Viewing Details

Once your pipeline is created, click run and then we can view the same in action:

viewing azure pipeline runs

You can click on the pipeline run instance to view more details about it:

viewing azure pipeline run instance details

Real-World Dotnet Core Multi Stage Pipeline to Deploy on Azure Web App

Since we are now familiar with all the concepts, let’s create a real world dotnet core multi stage pipeline to deploy on azure web app by using below code:


stages:
– stage: Build_Source_Code # Build Source Code for Dotnet Core Web App
jobs:
– job: Build
pool: 'Hosted VS2017'
variables:
buildConfiguration: 'Release'
continueOnError: false
steps:
– task: DotNetCoreCLI@2
inputs:
command: build
arguments: '–configuration $(buildConfiguration)'
– task: DotNetCoreCLI@2
inputs:
command: publish
arguments: '–configuration $(buildConfiguration) –output $(Build.ArtifactStagingDirectory)'
modifyOutputPath: true
zipAfterPublish: true
– task: PublishBuildArtifacts@1
inputs:
path: $(Build.ArtifactStagingDirectory)
artifact: drop
– stage: Run_Unit_tests # Run Unit tests in the Source code
jobs:
– job: Tests
pool: 'Hosted VS2017'
variables:
buildConfiguration: 'Release'
continueOnError: true
steps:
– task: DotNetCoreCLI@2
inputs:
command: test
projects: dotnetcore-tests
arguments: '–configuration $(buildConfiguration) –logger trx'
– task: PublishTestResults@2
condition: succeededOrFailed()
inputs:
testRunner: VSTest
testResultsFiles: '**/*.trx'
– stage: Deploy_In_Dev # Deploy artifacts to the dev environment
jobs:
– deployment: azure_web_app_dev
pool: 'Hosted VS2017'
variables:
WebAppName: 'multistage-yaml-dev'
environment: 'dev-environment'
strategy:
runOnce:
deploy:
steps:
– task: AzureRMWebAppDeployment@4
displayName: Azure App Service Deploy
inputs:
WebAppKind: webApp
ConnectedServiceName: 'pay-as-you-go'
WebAppName: $(WebAppName)
Package: $(System.WorkFolder)/**/*.zip
– stage: Deploy_In_QA # Deploy artifacts to the qa environment
jobs:
– deployment: azure_web_app_qa
pool: 'Hosted VS2017'
variables:
WebAppName: 'multistage-yaml-qa'
environment: 'qa-environment'
strategy:
runOnce:
deploy:
steps:
– task: AzureRMWebAppDeployment@4
displayName: Azure App Service Deploy
inputs:
WebAppKind: webApp
ConnectedServiceName: 'pay-as-you-go'
WebAppName: $(WebAppName)
Package: $(System.WorkFolder)/**/*.zip
– stage: Deploy_In_Prod # Deploy artifacts to the production environment
jobs:
– deployment: azure_web_app_prod
pool: 'Hosted VS2017'
variables:
WebAppName: 'multistage-yaml'
environment: 'prod-environment'
strategy:
runOnce:
deploy:
steps:
– task: AzureRMWebAppDeployment@4
displayName: Azure App Service Deploy
inputs:
WebAppKind: webApp
ConnectedServiceName: 'pay-as-you-go'
WebAppName: $(WebAppName)
Package: $(System.WorkFolder)/**/*.zip

In above code, we have created 5 stages: Build Source Code, Run Unit Tests, Deploy in Dev, Deploy in QA and Deploy in Production environment. Since building source code consists of smaller subtasks.

If everything goes well, you would be able to see the pipeline processing smoothly:

viewing azure pipeline run instance

We can also go to Azure and verify we are able to access web app after deployment:

verify the azure web app is working

Summary and Notes

One can now choose to write either CI or CD or both of them using the Azure DevOps pipelines in YAML. This allows the configuration of both build and release as part of the source code.

If you are viewing this post on mobile, the source code might not be visible due to feature restrictions set by AMP. In such case, open this blog post in full browser.

The source code used in this blog post can be found here at GitHub and is available under blog/8496 and master branches.

7 thoughts on “Create Multi Stage Pipelines with YAML in Azure DevOps

  1. hi
    great article and definitely helpful for building multistage pipelines
    my question is around multiple pipelines for different environments. does one method have any advantage over the other (multistage vs multiple release pipelines?

    Like

    • Hi Sam, Multi stage pipelines are common way of configuring developments as they provide single view of artifacts moving across various environments say from dev to qa to uat to prod.

      Like

  2. Hello Mohit,

    Do we know how do we run 2 stages in parallel in multi-stage pipeline. Lets say if I want to run dev and QA pipeline in parallel? is it possible?

    Thanks,
    Ranjit

    Like

Leave a comment