Until recently, Azure DevOps had offered separate build and release views for its users. This was a little different from pipeline features in other CI/CD tools like Jenkins, where if you build a pipeline, it is a single unified experience. With recent update, they have released unified experience for the Multi Stage Pipelines. Now one can use a unified YAML experience and configure Azure DevOps pipelines to do CI, CD, or CI and CD together. This not only allows to control the build configuration as part of the source code but releases as well. In this blog post, we are going to create and work with the same.
Enable Multi Stage Pipeline feature
Since this feature is under preview, as of writing of this blog post, one needs to go to preview features from his user profile, and enable the same by sliding radio bar to the right:
Write a Multi Stage Pipeline
A bit about syntax
Azure DevOps pipelines consists of multiple stages. Each stage describes the part of the CI/CD process. They denote a particular milestone in the CI/CD prcoess for example building source code, run unit tests, etc.
Stages consists of one or more jobs, which are units of works assignable to a build/release agent. Jobs consists of linear series of steps. Each step can be a simple task such as echo or a complex script or some other task referring to 3rd party like manual intervention etc.
To know more, one can read about the Azure DevOps YAML syntax here.
Hello-World YAML Pipeline
Consider below simple hello-world pipeline for demonstration of multi stage pipelines:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stages: | |
– stage: Build | |
jobs: | |
– job: Build | |
pool: | |
vmImage: 'Ubuntu-16.04' | |
continueOnError: true | |
steps: | |
– script: echo "hello to my first Build" | |
– stage: dev_deploy | |
jobs: | |
– deployment: dev_deploy | |
pool: | |
vmImage: 'Ubuntu-16.04' | |
environment: 'dev-hello' | |
strategy: | |
runOnce: | |
deploy: | |
steps: | |
– script: echo "hello, dev world !!!" | |
– stage: qa_deploy | |
jobs: | |
– deployment: qa_deploy | |
pool: | |
vmImage: 'Ubuntu-16.04' | |
environment: 'qa-hello' | |
strategy: | |
runOnce: | |
deploy: | |
steps: | |
– script: echo "hello, qa world !!!" |
There are couple of interesting features like deployment and strategy with what was the usual YAML.
Environment Keyword
Environments are a very new feature and they represent the group of resources targeted by a pipeline, for example, Kubernetes clusters, Azure Web Apps, virtual machines, and databases. Environments are useful to group resources, for example, you can group dev resources for your application under an environment named “deployment”, group qa resources for your application under an enviroment named “staging” or “qa” and so on. You are free to name environments according to your choice.
Also, we can view deployments made on a environment using Azure Pipelines:
More capabilities will be added to environments over time, and we’ll cover those under separate blog posts.
Strategy
This impacts the deployment strategy, which defines how your application is rolled out across the cluster. The default strategy is runOnce, but in the future you’ll be able to easily indicate other strategies, such as canary or blue-green. Again, we’ll cover those under separate blog posts.
Create Multi Stage Pipeline
To create a pipeline, go to Azure Pipelines and select ‘new pipeline’:
After this, select one of the option to let it know where the Source code resides:
A small YAML icon next to the possible indicates that Azure DevOps will analyze your code and recommend a YAML template that makes sense for you and gets you up and running quickly. Select appropriate option to proceed.
After this, review and edit your pipeline as necessary and then click run to deploy the pipeline into action:
Run the Azure Pipeline and Viewing Details
Once your pipeline is created, click run and then we can view the same in action:
You can click on the pipeline run instance to view more details about it:
Real-World Dotnet Core Multi Stage Pipeline to Deploy on Azure Web App
Since we are now familiar with all the concepts, let’s create a real world dotnet core multi stage pipeline to deploy on azure web app by using below code:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stages: | |
– stage: Build_Source_Code # Build Source Code for Dotnet Core Web App | |
jobs: | |
– job: Build | |
pool: 'Hosted VS2017' | |
variables: | |
buildConfiguration: 'Release' | |
continueOnError: false | |
steps: | |
– task: DotNetCoreCLI@2 | |
inputs: | |
command: build | |
arguments: '–configuration $(buildConfiguration)' | |
– task: DotNetCoreCLI@2 | |
inputs: | |
command: publish | |
arguments: '–configuration $(buildConfiguration) –output $(Build.ArtifactStagingDirectory)' | |
modifyOutputPath: true | |
zipAfterPublish: true | |
– task: PublishBuildArtifacts@1 | |
inputs: | |
path: $(Build.ArtifactStagingDirectory) | |
artifact: drop | |
– stage: Run_Unit_tests # Run Unit tests in the Source code | |
jobs: | |
– job: Tests | |
pool: 'Hosted VS2017' | |
variables: | |
buildConfiguration: 'Release' | |
continueOnError: true | |
steps: | |
– task: DotNetCoreCLI@2 | |
inputs: | |
command: test | |
projects: dotnetcore-tests | |
arguments: '–configuration $(buildConfiguration) –logger trx' | |
– task: PublishTestResults@2 | |
condition: succeededOrFailed() | |
inputs: | |
testRunner: VSTest | |
testResultsFiles: '**/*.trx' | |
– stage: Deploy_In_Dev # Deploy artifacts to the dev environment | |
jobs: | |
– deployment: azure_web_app_dev | |
pool: 'Hosted VS2017' | |
variables: | |
WebAppName: 'multistage-yaml-dev' | |
environment: 'dev-environment' | |
strategy: | |
runOnce: | |
deploy: | |
steps: | |
– task: AzureRMWebAppDeployment@4 | |
displayName: Azure App Service Deploy | |
inputs: | |
WebAppKind: webApp | |
ConnectedServiceName: 'pay-as-you-go' | |
WebAppName: $(WebAppName) | |
Package: $(System.WorkFolder)/**/*.zip | |
– stage: Deploy_In_QA # Deploy artifacts to the qa environment | |
jobs: | |
– deployment: azure_web_app_qa | |
pool: 'Hosted VS2017' | |
variables: | |
WebAppName: 'multistage-yaml-qa' | |
environment: 'qa-environment' | |
strategy: | |
runOnce: | |
deploy: | |
steps: | |
– task: AzureRMWebAppDeployment@4 | |
displayName: Azure App Service Deploy | |
inputs: | |
WebAppKind: webApp | |
ConnectedServiceName: 'pay-as-you-go' | |
WebAppName: $(WebAppName) | |
Package: $(System.WorkFolder)/**/*.zip | |
– stage: Deploy_In_Prod # Deploy artifacts to the production environment | |
jobs: | |
– deployment: azure_web_app_prod | |
pool: 'Hosted VS2017' | |
variables: | |
WebAppName: 'multistage-yaml' | |
environment: 'prod-environment' | |
strategy: | |
runOnce: | |
deploy: | |
steps: | |
– task: AzureRMWebAppDeployment@4 | |
displayName: Azure App Service Deploy | |
inputs: | |
WebAppKind: webApp | |
ConnectedServiceName: 'pay-as-you-go' | |
WebAppName: $(WebAppName) | |
Package: $(System.WorkFolder)/**/*.zip |
In above code, we have created 5 stages: Build Source Code, Run Unit Tests, Deploy in Dev, Deploy in QA and Deploy in Production environment. Since building source code consists of smaller subtasks.
If everything goes well, you would be able to see the pipeline processing smoothly:
We can also go to Azure and verify we are able to access web app after deployment:
Summary and Notes
One can now choose to write either CI or CD or both of them using the Azure DevOps pipelines in YAML. This allows the configuration of both build and release as part of the source code.
If you are viewing this post on mobile, the source code might not be visible due to feature restrictions set by AMP. In such case, open this blog post in full browser.
The source code used in this blog post can be found here at GitHub and is available under blog/8496 and master branches.
[…] we discussed in one of our earlier posts, the YAML pipeline can consist of both CI and CD tasks or can contain them individually. So […]
LikeLike
[…] it was not possible to do it for the YAML based pipelines up until now. As we discussed in our previous blog post on how to write multi stage pipelines, we also discussed about the environments. […]
LikeLike
hi
great article and definitely helpful for building multistage pipelines
my question is around multiple pipelines for different environments. does one method have any advantage over the other (multistage vs multiple release pipelines?
LikeLike
Hi Sam, Multi stage pipelines are common way of configuring developments as they provide single view of artifacts moving across various environments say from dev to qa to uat to prod.
LikeLike
Hello Mohit,
Do we know how do we run 2 stages in parallel in multi-stage pipeline. Lets say if I want to run dev and QA pipeline in parallel? is it possible?
Thanks,
Ranjit
LikeLike
Use property dependsOn in QA stage and set it to empty array to remove the implicit dependency on previous stage (Dev)
LikeLike
[…] I made some changes to the template following the available documentation on multi-stage pipelines. […]
LikeLike