Configure Azure Automation hybrid runbook workers and run hybrid runbooks

Azure automation is Microsoft cloud based workflow engine that can be used to run workflows in Azure. It started out its life as engine to operate against azure resources. However with time it gained the capability to run against on-premises resources as well by introduction of new feature called hybrid runbook workers.

You can think of hybrid runbook workers as one or more servers (think high availability) in your on-premise datacenter that can act on behalf of runbooks located in azure cloud. It has the capability to execute a runbook (known as hybrid runbook) which can be as simple as PowerShell script or as complex as a PowerShell workflow can be. So you can use it to to orchestrate complex, repetitive, or time-consuming tasks for your on-premise servers.

Below diagram from Microsoft’s documentation can help in visualizing the architecture:

Each server designated as hybrid runbook worker requires Microsoft Management Agent to be installed on it. This agent connects to Microsoft Operations Management suite located on cloud and the azure automation runbook environment. Operation management suite is only used to install and maintain the management agent and to monitor the functionality of the worker. The delivery of the runbooks and the instructions to run them are performed by the azure automation.

Note that you do not need to open any ports in your firewall for this to work.

In this post we’ll discuss steps on this new feature and walk through the setup experience from scratch to getting our first local runbook execution from an Azure subscription in the cloud.

If you do not already have a azure automation account, login into azure and create one azure automation account.

Now we’ll need to add operational insights to our azure subscription in the form of a workspace. Good thing is that workspace starts as free. You need to open this url and create an account for you. This is a separate portal for Microsoft Operations Management Suite (OMS). Once you have created an account, you’ll need to create a new workspace:

This will open up a new form. Provide basic details such as workspace name, region location and email id as below:

Inside the workspace, we add the azure automation component. This will appears under the “Solutions Gallery” tile in the OMS page:

From the list of solutions, find azure automation analytics and add it:

After it’s added, you’ll see a new tile on the home screen of the OMS portal. Open it and use the Configure button at the bottom of the page to ensure the correct Automation Account is selected. You can also select to create new azure automation account.

Once its done, we’ll need to download and install microsoft monitoring agent now. This is same as SCOM agent so you cannot install it on a server which is having a agent already installed. Go to workspace -> settings -> connected sources -> windows servers:

As indicated in the above url, you’ll find the workspace id and primary key at same place. This will be required to uniquely identify the workspace and connecting to it by agent and needed at installation time.

When installation completes, you will find a new Control Panel entry if you need to update the settings:

At this point we have configured our agent to point to cloud workspace. You can check the properties of the agent if its connected fine or not:

You should now be able to find a new folder named ‘AzureAutomationFiles’ under the installation directory, by default C:\Program Files\Microsoft Monitoring Agent\Agent. Once downloaded, the folder should contain a new PowerShell module, ‘HybridRegistration’. We use this to register the Hybrid Runbook Worker with the automation account, letting it know the new agent is alive and ready to receive runbook jobs.

To do this, you’ll need to run below commands on the hybrid runbook server:

Import-Module HybridRegistration

$settings = @{
 Name = "OnPremSPServer"
 EndPoint = "https://sea-agentservice-prod-1.azure-automation.net/accounts/13e39260-efeb-40cc-938b-32fb659c77c3"
 Token = "Z36GoNfoomADGtUZmEY0+OknBnfMVgJYZF0zUU6YQ9wUi7oFPQFQYOWAMVDt/a01y3Blp197bgWNtPfbJgRCeQ=="
}

Add-HybridRunbookWorker @settings

You can find the value of endpoint and token in the azure automation account:

You should be able to see a Hybrid Runbook Worker entry appear from the list you see when clicking the Hybrid Worker Groups tile in your Automation Account view:

Runbook output and messages are sent to Azure Automation from hybrid workers just like runbook jobs run in the cloud. So unless outbound HTTPS is blocked in your environment, you don’t need to run around network admins asking to open ports for this. So now we have configured a fancy new hybrid runbook worker for running hybrid runbooks.

For our demo purposes, let’s choose a simple task of copying files between two shared locations in on-premise network. Since the task is completely localized to on-premises network, a hybrid runbook worker is perfectly suitable to do it.

So, we’ll create a share on the local server and put some files in there:

We’ll also need to create a runbook with following code:

workflow Copy-Files
{
    Param
    (
        [parameter(Mandatory=$true)]
        [String] $SourcePath,

        [parameter(Mandatory=$true)]
        [String] $DestinationPath
    )
 
    Write-Output "Executing runbook on hybrid runbook worker: $env:ComputerName"
    Write-Output "Source: $SourcePath"
    Write-Output "Destination: $DestinationPath"
    $result = InlineScript
    {
        try
        {
            Copy-Item -Path "$using:SourcePath" -Destination "$using:DestinationPath"
        }
        catch
        {
            $errorMessage = $error[0].Exception.Message
        }
  
        if($errorMessage -eq $null)
        {
            return "Successfully copied files"
        }
        else
        {
            return "Failed: Encountered error(s) while copying files. Error message=[$errorMessage]"
        }
    }
 
    Write-Output $result
    Write-Output "Execution finished"
}

After publishing, we’ll then start runbook. This will prompt for $SourcePath and $DestinationPath values and select hybrid worker group:

Clicking OK, the job is immediately queued. After few minutes, it should start running and get completed.

On the hybrid server end, if we check event logs, we can see event log indicating to receive notification to create a new sandbox:

This would be followed by log to start sandbox creation:

It will run the workflow in an isolated process:

After a short time, the runbook finishes and reports its status back to Azure, where we should see the output produced by the script. At this point, if we check the destination share folder, we can see the files copied.