Add a private build agent to VSTS and configure for capabilities

VSTS or Visual Studio Team Services is a great DevOps tool from Microsoft and it comes as a Software-as-a-Service. While configuring build pipelines, once can choose the option to build and deploy using a hosted agent. A hosted agent is provided by Microsoft and is pre-configured with all major build tools installed like Visual Studio, Java, Ant, etc. So for many organizations, this is the simplest way to build and deploy.

However, the hosted agent might not suffice for your needs. For example, you may want to use your own testing engine or compilation engine, or you are working on a not so common programming language, etc. In such cases, you will need to setup your private build server. Private agents give you more control to install dependent software needed for your builds and deployments. In this blog post, we’ll learn how to add a build server to VSTS and also configure it for build capabilities.

Provision a new server

First, we need to configure a new server. It may have OSX, Windows, or Linux. It may also be Linux docker container. Also, it can be either on-premise and any cloud service provider. When its on-premise, it can either connect to internet using proxy or direct connection. So lots of choices over here. You can choose a server most suitable to your environment.

Provision for network configuration for server

The agent communicates with VSTS or TFS to determine which job it needs to run, and to report the logs and job status. This communication is always initiated by the agent. All the messages from the agent to VSTS or TFS happen over HTTP or HTTPS, depending on how you configure the agent. So you need to make sure that you are able to connect over port 80/443 to VSTS are allowed. If you are having a cloud virtual machine, this should already be setup for you.

Configure for authentication

To register an agent, you need to be a member of the administrator role in the agent pool. Your agent can authenticate to VSTS using Personal Access token only. It won’t work with user’s credentials, etc. For this, first login into the VSTS account https://{account_name}.visualstudio.com. Now go to the user profile from right top and then select security:

access security from the profile
access security from the profile

Now, we can create a personal access token:

add-personal-access-token
add-personal-access-token

Select a token expiry date. For the scope select Agent Pools (read, manage) and make sure all the other boxes are cleared. If its a deployment group agent, for the scope select Deployment group (read, manage) and make sure all the other boxes are cleared. Copy the token generated as VSTS will not save it for later. We’ll need this token when configuring the agent.

Download and configure the agent

Let’s first login into the private agent server by using appropriate credentials. Once its done, sign in to the VSTS account https://{account_name}.visualstudio.com. Then go to account settings -> agent pools section and create a new pool:

Create new agent pool
Create new agent pool

Provide a name for the new pool. Once its created, click download agent. This will download file on the build machine. Now go ahead and extract the files into a directory called C:\agent.

You can also choose to use below PowerShell commands for same:

PS C:\> mkdir agent ; cd agent
PS C:\agent> Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$HOME\Downloads\vsts-agent-win7-x64-2.123.0.zip", "$PWD")

These commands may vary per OS. For example, you can use below commands on ubuntu server:

~/$ mkdir myagent && cd myagent
~/myagent$ tar zxvf ~/Downloads/vsts-agent-ubuntu.14.04-x64-2.123.0.tar.gz

Once its extracted, run the configure.cmd batch file. It will ask for certain details such as VSTS server location, PAT etc. Also, for the most part, we’ll accept defaults and click yes. Below is a quick sample from my agent:

Configure the agent properties
Configure the agent properties

Do make sure that account you are running it with has read-write access to C:\agent directory.

Install various build tools on server

In many cases after you deploy an agent you’ll need to install software or utilities. Generally you should install on your agents whatever software and tools you use on your dev machine. The general tools include such as Visual Studio, Java, Ant, Maven, npm, etc. Do make sure to restart vsts agent service once you are done with this.

Configure capabilities for build agent

Every agent has a set of capabilities that indicate what it can do. Capabilities are name-value pairs that are either automatically discovered by the agent software, in which case they are called system capabilities, or those that you define, in which case they are called user capabilities.

The agent software automatically determines various system capabilities such as the name of the machine, type of operating system, and versions of certain software installed on the machine. Also, environment variables defined in the machine automatically appear in the list of system capabilities.

Few build tasks may require you to advertise the capabilities so that an appropriate agent can be selected from list of pools. So, even though your server may have java installed, if you have not advertised it, your build may fail. We can configure this by navigating to the Agent pools hub and selecting the Capabilities tab for the desired agent.

Below is a quick sample:

Configure the user capabilities for agent
Configure the user capabilities for agent

Once all of above is done, our private agent is ready to be used. Specify the same from next time you queue the build.

18 thoughts on “Add a private build agent to VSTS and configure for capabilities

  1. I can’t get this to work – I get an error when running the config.cmd script when I paste in the PAT:

    “`
    Enter personal access token > ****************************************************
    TF400813: Resource not available for anonymous access. Client authentication required.
    “`

    What setting could be preventing this?

    Like

  2. Hi Mohit, we are recently using VSTS. I created private 1 agent pool and underneath Installed 4 agents. But when I queue build always its picking only default 1 agents and the rest 3 are idle all the time. Do i need to change any setting

    Like

    • Hi AJ, thanks for stopping by. Check these things in order:
      1. All 4 agents are healthy and online in VSTS agent pool
      2. There are enough jobs for each server. The agents will be selected on the basis of availability. If first agent is always idle, it will never go to other agent.
      3. You have purchased additional pipelines. Free accounts get one pipeline at a time which translates to one build/release job.

      Like

  3. […] However sometime ago, Microsoft started providing docker image for the agents which makes it really easy to run them as containers. If you are running agents as docker containers, you can provision them on the fly to perform the build or the release job and then remove them. This way you can utilize your resources more efficiently. In this blog post, we are going to discuss steps for the same. If you want to use agents installed in traditional way, you are covered here. […]

    Like

  4. Are you able to set this up with the default Hosted Pool ?
    I am getting the following error when using PAT: Cannot modify agents for pool Hosted because it is hosted and managed by the system. Agents may only be registered with private pools.

    Like

  5. Hi Mohit,

    thanks for putting this up. I can make a private agent and run commands on it, but i’m having some issues with installing tools. I have a python CI pipeline that runs fine on the microsoft hosted agents but not so good on self hosted ones. The problems are:
    1. I installed python to the correct _work/_tool/Python/x.x.x/x64 path meaning the use python task passes, but when i run the python script path and print(sys.version) i get the os default version.

    2. when i try to pip install something i get the error saying command not found.

    couldn’t find any documentation on installing tools on private agents do any help you could give would be appreciated.

    Like

    • That would most likely be an issue with system path. Correct the system path to reflect newer binaries path and restart agent service to get it reflected on the server side as well.

      Like

  6. Hi Mohit,
    thank you for your Instructions. There is something for the all the lazy ones here: called “Private Build Agents for Azure DevOps Lite” in the Azure Marketplace.

    It deploys an Azure VM with a Build Agent and automatically connects to your Azure DevOps. The offer is free, you just need to pay the VM. Enjoy 🙂

    Like

Leave a comment