Configure Azure Automation hybrid runbook workers and run hybrid runbooks

Azure automation is Microsoft cloud based workflow engine that can be used to run workflows in Azure. It started out its life as engine to operate against azure resources. However with time it gained the capability to run against on-premises resources as well by introduction of new feature called hybrid runbook workers.

You can think of hybrid runbook workers as one or more servers (think high availability) in your on-premise datacenter that can act on behalf of runbooks located in azure cloud. It has the capability to execute a runbook (known as hybrid runbook) which can be as simple as PowerShell script or as complex as a PowerShell workflow can be. So you can use it to to orchestrate complex, repetitive, or time-consuming tasks for your on-premise servers.

Below diagram from Microsoft’s documentation can help in visualizing the architecture:

Azure automation hybrid runbook worker overview
Azure automation hybrid runbook worker overview

Each server designated as hybrid runbook worker requires Microsoft Management Agent to be installed on it. This agent connects to Microsoft Operations Management suite located on cloud and the azure automation runbook environment. Operation management suite is only used to install and maintain the management agent and to monitor the functionality of the worker. The delivery of the runbooks and the instructions to run them are performed by the azure automation.

Hybrid runbook worker components
Hybrid runbook worker components

Note that you do not need to open any ports in your firewall for this to work.

In this post we’ll discuss steps on this new feature and walk through the setup experience from scratch to getting our first local runbook execution from an Azure subscription in the cloud.

If you do not already have a azure automation account, login into azure and create one azure automation account.

Now we’ll need to add operational insights to our azure subscription in the form of a workspace. Good thing is that workspace starts as free. You need to open this url and create an account for you. This is a separate portal for Microsoft Operations Management Suite (OMS). Once you have created an account, you’ll need to create a new workspace:

create-a-new-workspace-in-the-oms-suite
Create a new workspace for the OMS suite

This will open up a new form. Provide basic details such as workspace name, region location and email id as below:

Create a new workspace for the OMS suite - 2
Create a new workspace for the OMS suite – 2

Inside the workspace, we add the azure automation component. This will appears under the “Solutions Gallery” tile in the OMS page:

location-of-solution-gallery
Location of solution gallery icon

From the list of solutions, find azure automation analytics and add it:

azure-automation-analytics
Add automation pack named azure automation analytics

After it’s added, you’ll see a new tile on the home screen of the OMS portal. Open it and use the Configure button at the bottom of the page to ensure the correct Automation Account is selected. You can also select to create new azure automation account.

Once its done, we’ll need to download and install microsoft monitoring agent now. This is same as SCOM agent so you cannot install it on a server which is having a agent already installed. Go to workspace -> settings -> connected sources -> windows servers:

Download windows agent for OMS suite
Download windows agent for OMS suite

As indicated in the above url, you’ll find the workspace id and primary key at same place. This will be required to uniquely identify the workspace and connecting to it by agent and needed at installation time.

When installation completes, you will find a new Control Panel entry if you need to update the settings:

Microsoft management suite location in control panel
Microsoft management suite location in control panel

At this point we have configured our agent to point to cloud workspace. You can check the properties of the agent if its connected fine or not:

Microsoft monitoring agent properties
Microsoft monitoring agent properties

You should now be able to find a new folder named ‘AzureAutomationFiles’ under the installation directory, by default C:\Program Files\Microsoft Monitoring Agent\Agent. Once downloaded, the folder should contain a new PowerShell module, ‘HybridRegistration’. We use this to register the Hybrid Runbook Worker with the automation account, letting it know the new agent is alive and ready to receive runbook jobs.

To do this, you’ll need to run below commands on the hybrid runbook server:

Import-Module HybridRegistration

$settings = @{
 Name = "OnPremSPServer"
 EndPoint = "https://sea-agentservice-prod-1.azure-automation.net/accounts/13e39260-efeb-40cc-938b-32fb659c77c3"
 Token = "Z36GoNfoomADGtUZmEY0+OknBnfMVgJYZF0zUU6YQ9wUi7oFPQFQYOWAMVDt/a01y3Blp197bgWNtPfbJgRCeQ=="
}

Add-HybridRunbookWorker @settings

You can find the value of endpoint and token in the azure automation account:

Endpoint address and primary access key for azure automation account
Endpoint address and primary access key for azure automation account

You should be able to see a Hybrid Runbook Worker entry appear from the list you see when clicking the Hybrid Worker Groups tile in your Automation Account view:

Access hybrid worker groups in the automation account
Access hybrid worker groups in the automation account

Runbook output and messages are sent to Azure Automation from hybrid workers just like runbook jobs run in the cloud. So unless outbound HTTPS is blocked in your environment, you don’t need to run around network admins asking to open ports for this. So now we have configured a fancy new hybrid runbook worker for running hybrid runbooks.

For our demo purposes, let’s choose a simple task of copying files between two shared locations in on-premise network. Since the task is completely localized to on-premises network, a hybrid runbook worker is perfectly suitable to do it.

So, we’ll create a share on the local server and put some files in there:

Source files in file share on local server
Source files in file share on local server

We’ll also need to create a runbook with following code:

workflow Copy-Files
{
    Param
    (
        [parameter(Mandatory=$true)]
        [String] $SourcePath,

        [parameter(Mandatory=$true)]
        [String] $DestinationPath
    )
 
    Write-Output "Executing runbook on hybrid runbook worker: $env:ComputerName"
    Write-Output "Source: $SourcePath"
    Write-Output "Destination: $DestinationPath"
    $result = InlineScript
    {
        try
        {
            Copy-Item -Path "$using:SourcePath" -Destination "$using:DestinationPath"
        }
        catch
        {
            $errorMessage = $error[0].Exception.Message
        }
  
        if($errorMessage -eq $null)
        {
            return "Successfully copied files"
        }
        else
        {
            return "Failed: Encountered error(s) while copying files. Error message=[$errorMessage]"
        }
    }
 
    Write-Output $result
    Write-Output "Execution finished"
}

After publishing, we’ll then start runbook. This will prompt for $SourcePath and $DestinationPath values and select hybrid worker group:

Parameters for running hybrid runbook
Parameters for running hybrid runbook

Clicking OK, the job is immediately queued. After few minutes, it should start running and get completed.

On the hybrid server end, if we check event logs, we can see event log indicating to receive notification to create a new sandbox:

Event notification received to create a new sandbox for running workflow
Event notification received to create a new sandbox for running workflow

This would be followed by log to start sandbox creation:

Event log indicating to start sandbox creation
Event log indicating to start sandbox creation

It will run the workflow in an isolated process:

runbook-is-executed-within-new-orchestrator-sandbox-process
Runbook is executed within new orchestrator sandbox process

After a short time, the runbook finishes and reports its status back to Azure, where we should see the output produced by the script. At this point, if we check the destination share folder, we can see the files copied.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s