10 AWS and Azure Features MSPs Should Know — and Exploit

CloudWatch is a log collection, monitoring and analysis service that’s indispensable for keeping abreast of an AWS environment — think of it as a combination syslog server and Splunk-lite. While CloudWatch has the alerting and dashboard features MSPs would expected of any monitoring tool, its real power comes in its automation potential. For example, you gain the ability to trigger responses to events, such as auto-scaling EC2 instances, invoking a Lambda (serverless) function or posting to an Amazon SNS (Simple Notification Service) event queue, or “topic,” in AWS-speak.

CloudTrail is another monitoring and auditing tool. It logs changes to an AWS account by recording all AWS API calls, similar to how a Linux tool like Tripwire logs any change to system files. On AWS (and other cloud services), every operation is performed using APIs for each service, which makes CloudTrail a means to granularly track access to things like the AWS Management Console, CLI usage or any changes to a deployed service. It logs so-called “Trails,” definitions of the events to be tracked, to an S3 bucket in JSON format that can be monitored with CloudWatch or filtered in real time for troubleshooting.

Config is another auditing service, but this time focused on verifying that operating AWS systems match predefined configuration templates. Config continually watches for configuration changes and, like CloudWatch, can post notifications to Amazon SNS when it detects events. Config can also auto-discover resources in an AWS account, record their configurations and use those as baselines against which future changes are logged. Config can use compliance rules built by AWS or custom rules written as a Lambda function (which supports Node.js, Python or Java). 

AWS Systems Manager is an automation tool that centralizes data and management of multiple services and resources. MSPs can create a hierarchical structure for similar resources, like EC2 instances, S3 buckets, RDS databases or EFS shares, allowing them to be grouped by type, application, customer or whatever organizational unit makes sense for customers. Systems Manager console integrates with AWS Resource Groups and can create or automate actions across a group. Automation tasks are encoded as JSON documents that can be executed through the AWS Management Console, CLI or various service SDKs and be manually started or automatically triggered in response to CloudWatch events.

Trusted Advisor is a resource-optimization tool that checks an AWS environment for compliance with best practices in five areas: cost management, performance, security, fault tolerance and service limits. For example, it will examine the usage rates of EC2 on-demand instances to identify those that are candidates for reserved instances (steady-state usage), downsizing (underutilization) or load balancing. Likewise, it scans for security vulnerabilities, such as instances with unrestricted access or open ports, overly permissive S3 bucket permissions or root/administrator accounts with lax IAM passwords and no multifactor authentication. Note that access to all Trusted Advisor features requires a business or enterprise support plan.

Automation is an Azure service that facilitates the programmatic execution of repetitive tasks. It can be used to automate: processes, using PowerShell or Python runbooks; configuration management, to collect inventory, track changes or compare running infrastructure with defined configurations; and update systems. It supports role-based access controls (RBAC) to allow segmenting administrative responsibilities and source code control systems (GitHub and Visual Studio Team Services) for collaborative development of runbooks.

Advisor is a resource-optimization service, similar to AWS Trusted Advisor, that checks a customer’s environment for compliance with best practices in the same four areas: cost, performance, security and reliability/high availability. For example, it’s best to protect applications against the failure of a compute instance by running it across two or more VMs in an availability set, so Advisor will flag instances that aren’t part of a set. Likewise, it will identify VMs not configured to back up their data and run an environment through Azure Security Center to identify security weaknesses and adherence to any custom security policies.

Monitor collects service metrics (real-time records of system variables) and logs (telemetry records and events) across a range of Azure services and customer applications. It can also collect data from Application Insights, Azure’s APM service for web applications. Monitor writes metrics into a data store that feeds the Metrics Explorer UI in the Azure Portal and sends log data to Log Analytics, formerly a standalone service that is being integrated into Monitor, for further analysis; it has a dashboard to summarize data or perform ad-hoc analysis, valuable to MSPs to answer customer queries. Monitor can also trigger alerts or actions, like auto-scaling an instance group, when data exceeds defined limits.

Azure Migrate scans on-premises VMware workloads to assess their suitability for migration to Azure VM instances. Migrate can auto-discover VMs and map dependencies of multitier applications. It also provides sizing recommendations based on each VMs runtime performance and estimates the monthly costs. Its output can feed Azure Site Recovery and Azure Database Migration services to automate the migration process. Although Migrate doesn’t yet work with Hyper-V VMs, running these through the Azure Site Recovery Deployment Planner will produce a similar set of recommendations.

Azure Resource Manager (ARM) Templates are a mechanism to use the ARM portal’s APIs to automate resource configuration and deployment using a superset of JSON code. Templates add expressions and functions to the basic JSON syntax and support hierarchical nesting that allows templates to call (link to) other templates. The ARM portal has a GUI for authoring templates, but MSPs with dev chops can also write them in Visual Studio or your text editor of choice — preferably one with syntax checking.