Logs - Catch all em diagnostics

This one is actually quite neat and I find it quite valuable. So configuring diagnostics for each single resource into tables in log analytics manually is not always as straightforward. By leveraging Monitor we can see our diagnostic settings, or lack thereof perhaps? If you have a lot of resources good luck scrolling down the load more option in the GUI. That was sarcasm yes!

First let's review what diagnostics accomplish for us. You might want to use Graph API that permits sending some of the data dependent upon what limitations the API permission that is provisioned can GET. In my opinion that only provides up to a certain extent. Event Hub I find more flexible as now we are streaming raw data events. But that's why we need to understand diagnostics. Because depending on what we do with the logs we have to decide where to put them.

So I like to use the illustration below, even some name changes occured, it does showcases it in a simplistic view.

So these event sources (diagnostics) can be ingested in log analytics tables, where it's interactive for up to 2 years, then we can send it to be archived using a storage account. That would be a Microsoft native solution. The other scenario we use an Event Hub to stream the data to external sources.

There is a limit of 5 diagnostic settings per resource. So that's the basic logic that will bring us to the next section.

Management groups are very neat where able to provide a hierarchical setup where the tenant root group will have these logical management groups where able to have multiple subscriptions bundled together. That could be based on environment prod/dev/uat just to throw one idea out there. Management groups also gives us the option to apply policies to the them so it can be managed top-down.

So when the consideration of deployment at scale of diagnostics, Azure Policy would be a good candidate for the job.

https://github.com/JimGBritt/AzurePolicy/tree/master/AzureMonitor/Scripts

So we will use the script in the above repository to generate the custom policy definitions. After we use the ARM template to provision the custom settings.

Start by running the "Create-AzDiagPolicy.ps1" script to pull all resource types in place:

Now that the custom objects are created we use the ARM template as follows:

Then we can see it in the portal and assign that definition. Now this is a policy initiative meaning it's a group of policies found underneath an initiative.

When using remediation with a managed identity assignment that applies automation for any future resources (for resource types pulled) to apply the setting. For existing resources just run a remediation job to catch them up to speed.