In our previous blog post, we discussed how we can leverage Azure Policy to bring Azure Resources to into compliance with the Organizational configuration standards. We also created a custom Azure Policy and observed its effects on compliance. There are however close to 100 Builtin Azure Policy which are applied at any time under the default initiative. To this, add the number of policies that would be created and applied to manage resources for different departments and to enforce different standards. You’ll be quickly looking at hundreds of Azure Policies and assignments and exclusions and need some help to manage them.
This also means that drilling down into Azure Policy Compliance tab and identifying non-compliant issues and taking resolution actions can become overwhelming quickly. In large organizations, it is also quite common that the action needed to bring resource into compliance lies with different teams or groups. It would be definitely great if you can get notification of when an Azure Resources becomes non-compliant and you can be able to quickly alert the respective group or team to take appropriate action on it. We can integrate Azure Log Analytics with Azure Policies to generate alerts for helping with the Azure Compliance.
Create Custom Azure Policy
First, we need to create an custom Azure Policy and apply the same for Azure Resources. Since tagging azure resources is a very common and useful operation, let’s create an Azure Policy to check if the Azure Resource Groups are properly tagged or not. Since there is no builtin policy for monitoring resource groups, we need to define our own policy for it. However, we’ll import one of the custom definitions from this GitHub location and apply the same at subscription level using below code:
You can create as many policies and assignments, as you need. Lets wait for the policy to get triggered and generate compliance data for the resource groups:
In our case, this policy has identified that two of the resource groups are non-compliant with it:
Create Log Analytics Workspace
To create log analytics workspace, go to All Services -> Search and Select ‘Log Analytics Workspace’ and create new. Provide the details to create log analytics workspace and select create:
Connecting the Activity Log to Log Analytics
For this, go to Log Analytics workspace -> Azure Activity log. There we can see the existing status of the connection with workspace. Here, select the subscription for which you want to configure the connection:
This will open a new blade in which you can click ‘connect’:
It will take few minutes for the connection to be established. Once its configured, you can see the status as well:
This will now start sending data from Azure Activity Logs to the Logs Analytics Workspace.
Identifying the required Log Analytics query
We now have to write a Log Analytics query that will return the resource groups that are missing tags. It may take some time to find the exact query for the logs you are looking for. We’ll use below query to help with our case:
AzureActivity | where OperationName contains "policy" and Properties contains "audit-resourceGroup-tags" and Properties contains "True"
In above query, we have mentioned that we are looking for events which contain “audit-resourceGroup-tags” as defined in the policy definition ID. You can check this from the policy definitions blade.
We can also try to run the query and confirm status. You can ignore the ActivityStatus = Succeeded. This is simply telling us that the policy compliance check was successful, not that the resource was compliant.
Configure Log Analytics alerts
Next, we need to configure alerts so that we can get notifications whenever a non-compliance event is logged. Within Log Analytics, select Alerts -> New Alert Rule:
First, we need to select the resource:
Once its set, click on add condition will allow us to configure signal logic:
Do note that after a policy is applied it’s typically re-evaluated every 24 hours. If a non-compliant policy is still found it will log that same event again. Therefore there’s little benefit in setting the period to more than 24 hours. It’s also possible to reduce the frequency but this will mean you’re getting multiple emails throughout the day reporting the same non-compliant policies.
Now, we’ll need to configure notification details from action group. These are your standard Azure Alerts configuration. You can either select an existing one or create a new one:
In this blog post’s case, we are keeping it simple by creating a new action group to just send myself an email. However, action groups are hugely flexible and can also call webhooks, functions, logic apps etc. So it opens up a lot of possibilities that you can do once you start getting notifications.
Verifying Email Alerts and Checking Details
Once you have configured the Azure Log Analytics successfully, you should start receiving notifications in the inbox for the recipients. One of the samples of the alerts is as below:
Within the Insight section of the email, there will be details of the Azure Resource which is non-compliant with the Azure Policy:
Since now you have the alert, you can choose it to forward to appropriate teams or groups to take corrective actions. You can also choose to directly send the alerts to the respective teams. Do note that the in the alerts rules you can configure webhooks, functions, logic apps etc. So it is very flexible and will go a very long way in enforcing compliance standards for your organization.
Summary and Notes
In a large Organization, it can quickly become overwhelming to monitor hundreds of Azure Policies, monitor non-compliant resources and take appropriate actions in a timely manner. We can integrate Azure Policies with Azure Log Analytics to get alerts when a resource becomes non-compliant and take appropriate actions.
This post borrows lot of information from blog written by Andrew Kelleher at this link. A lot of credit goes to him for the same.
Source code hosted on this blog can be found under master and blog/8699 branch on GitHub here.
For Further Reading
Azure Monitor Logs – https://docs.microsoft.com/en-us/azure/governance/policy/how-to/get-compliance-data
Azure Log Queries – https://docs.microsoft.com/en-us/azure/azure-monitor/log-query/log-query-overview
Andrew Kelleher (Originating Post) – https://medium.com/azure-architects/using-log-analytics-alerts-for-non-compliant-azure-policies-8d99f74089d9
