Optimize Application Insights
Application Insights is a powerful tool for monitoring your applications, but making the most of it requires careful consideration of various factors. This page provides a comprehensive set of recommendations based on the Azure Well-Architected Framework to help you optimize your Application Insights implementation across five key pillars: Reliability, Security, Cost Optimization, Operational Excellence, and Performance Efficiency.
By implementing these recommendations, you can ensure your application monitoring solution is robust, secure, cost-effective, and easy to manage.
Talk to an Azure expert! Email us or schedule a 30-minute consultation and let's optimize your Azure environment together!
Stay ahead with actionable insights for Azure optimization. Subscribe to updates and unlock the full potential of Azure!
Cost Optimization Recommendations
Limit Unplanned Charges with Daily Caps
Set daily caps in both Application Insights and Log Analytics to prevent unexpected charges.
While setting daily caps can help prevent overspending, they can also lead to data loss. Try to avoid reaching the daily cap to ensure continuous data collection.
Tune Data Collection
Use sampling at the Application Insights level to reduce data volume and storage costs.
Leverage log levels to tune and reduce the volume of trace log telemetry being collected. Decrease the sample rate for less critical flows and increase it for high-criticality flows.
Remove Unneeded Components
Regularly review and remove or optimize legacy, unused, or underutilized components of your monitoring solution.
Use telemetry filters to exclude nonessential telemetry from being collected and stored.
Edit ApplicationInsights.config to disable unnecessary data collection modules (e.g., performance counters or dependency data).
Optimize Environment Costs
Optimize log levels and metrics collected based on the environment (e.g., production vs. development).
Production environments may require more detailed logging than development environments.
Optimize Code Costs
Ensure you are using the latest Application Insights SDKs.
Older versions may collect more data by default, leading to higher costs.
Optimize Availability Test Locations
Adjust the number of locations for standard availability tests based on environment needs.
Production environments may require testing from more locations than pre-production environments. Reducing unnecessary test locations can reduce costs.
Co-locate Resources for Cost Efficiency
Deploying Application Insights in the same region as the Log Analytics workspace minimizes latency and associated data transfer costs.
Limit Client-Side Reporting
Limit the number of AJAX calls reported in each page view or disable AJAX reporting if it is not essential.
Disabling AJAX calls will also disable JavaScript correlation. This helps control data volume from client-side applications.
Utilize Pre-aggregated Metrics
Use pre-aggregated metrics to handle high-volume telemetry data efficiently and limit the use of custom metrics, which can be more costly.
Monitor the costs associated with enabling alerting on custom metric dimensions.
Consider Autoinstrumentation
Using autoinstrumentation saves software engineering time and optimizes costs by eliminating manual SDK updates and reducing code maintenance overhead.
Limit Unwanted Trace Logging
Take steps to limit unwanted trace logging, which can contribute significantly to data volume and storage costs.
- Remove health checks: Filter out noisy traces to optimize cost
- Azure Kubernetes Service (AKS): Adjust control plane and data plane logs to reduce volume.
- Azure Functions: Adapt log levels and scope to optimize log volume.
Reliability Recommendations
Instance Per Workload Per Environment
Use a separate Application Insights resource for each workload and environment (e.g., development, staging, production).
This practice prevents telemetry from different application versions from being mixed and ensures that a misconfiguration in one environment doesn't affect logging in another.
Co-locate with Log Analytics Workspace
Deploy your Application Insights resource in the same Azure region as the underlying Log Analytics workspace.
Application Insights relies on Log Analytics for proper functioning. Having these resources in different regions increases the chance of a regional failure impacting your monitoring.
Implement a Resilient Workspace Design
Follow best practices for Azure Monitor Logs to ensure a resilient Log Analytics workspace design.
This ensures continuous and robust monitoring by minimizing potential disruptions to your logging infrastructure.
Create a Comprehensive Resiliency Plan
Define how your workload should behave if Application Insights becomes unreachable.
- Conduct a failure mode analysis to identify potential points of failure.
- Determine how your workload should handle the unavailability of Application Insights.
- Document the expected behavior and test your resiliency plan thoroughly.
Workspace Resiliency and Recovery
Define targets for your data collection components to ensure data recovery in case of failures.
- Assess the criticality of your collected data and determine if it requires recovery.
- Review service limits for Application Insights and Log Analytics to understand restrictions.
- Consider using a dedicated cluster for workspace resilience if the continuous availability of telemetry within the retention period is business-critical.
- Use diagnostic settings to export platform logs and metrics for backup and recovery.
Scalable Data Ingestion Strategy
Implement a timely and reliable scaling strategy to plan for data ingestion growth.
Monitor and adjust limits on sampling and data ingestion as your application traffic grows to prevent data loss due to exceeding capacity. This ensures that your monitoring solution scales effectively with increasing data volumes.
Recovery With Infrastructure as Code
Adopt infrastructure as code using tools like Bicep templates to create or re-create your Application Insights setup.
This includes alerts, dashboards, and queries. This approach helps to quickly restore all critical components, minimizing downtime and maintaining service reliability in case of failures.
Security Recommendations
Azure Monitor Security Baseline
Review and implement the Azure Monitor security baseline.
This provides guidance on security best practices to help secure your cloud solutions on Azure, including your Application Insights setup.
Keep Instrumentation Up to Date
Regularly update your Application Insights SDKs and Azure Monitor OpenTelemetry Distro.
Updating at least once a year ensures that you have the latest security patches and features, minimizing potential vulnerabilities.
Define a Personal Data Handling Strategy
Define a strategy for handling personal data in Application Insights to comply with relevant regulations (e.g., GDPR).
Regularly verify that the collection and handling of sensitive data, including IP addresses and personal data, comply with all applicable regulations.
Create Intentional Segmentation
Determine the number of Application Insights resources you need to segment your monitoring effectively.
This allows for better isolation and control over access to sensitive data.
Azure Monitor Customer-Managed Key
Use customer-managed keys to encrypt the data and saved queries in your Log Analytics workspaces.
This provides you with more control over access and encryption than using Microsoft-managed keys.
Control Network Traffic
Consider using private connectivity options like Azure Private Link for accessing Azure services.
Private connectivity isolates your traffic from the public internet, enhancing the security of your monitoring data.
Secure Storage Systems
Review the Log Analytics service guide for guidance on securing the data collected by Application Insights.
Implement appropriate security measures to protect your monitoring data at rest.
Autoinstrumentation When Possible
Consider using autoinstrumentation if your business needs and hosting environment allow it.
Autoinstrumentation eliminates the need for manual SDK updates and code changes, reducing the risk of security vulnerabilities and ensuring consistent monitoring without manual intervention.
Managed Identities and Azure AD
Use managed identities and Azure Active Directory (Azure AD) for authentication and authorization.
This eliminates the need for managing credentials, as Azure handles their management, rotation, and protection.
Minimize or Obfuscate Personal Data
Stop collecting personal data or take steps to obfuscate, anonymize, or adjust the collected data.
By default, Application Insights does not store IP addresses. Keep this setting to prevent compliance issues. If you must collect personal data, ensure it is handled in accordance with all applicable regulations.
Instance Per Workload Per Environment
Similar to the reliability recommendation, this helps ensure data isolation and security by separating telemetry from different environments.
It makes it easier to apply environment-specific configurations and access controls.
Azure Private Link for Secure Access
Use Azure Private Link to access Azure services over a private endpoint.
This enhances the confidentiality of your workload's logs by keeping logging traffic on fully private networks. Be aware of the configuration and limitations of Azure Private Link. Avoid connecting your Azure Monitor Private Link Scope (AMPLS) to multiple virtual networks that share the same DNS to prevent "wrong private endpoint" errors.
Operational Excellence Recommendations
Integrate Team Specializations
Integrate the specializations of your application monitoring team members into your instrumentation and monitoring practices.
This ensures a comprehensive and well-aligned approach to monitoring.
Keep Instrumentation Up-to-Date
Regularly update your Application Insights SDKs and Azure Monitor OpenTelemetry Distro.
This ensures optimal performance and access to the latest features and bug fixes.
Formalize Planning Processes
Use work item integration to easily create work items in platforms like GitHub or Azure DevOps, embedding relevant Application Insights data within them.
This helps streamline your monitoring and development workflows.
Monitor Availability and Responsiveness
Configure Application Insights to monitor the availability and responsiveness of your web application.
Utilize built-in features like queries and dashboards tailored to your specific business needs. After deployment, set up recurring tests to continuously monitor availability and responsiveness.
Emergency Operations Practice
Use alerts and workbooks to effectively identify and respond to incidents.
Clearly define human responsibilities in case of emergencies (e.g., who reboots the application if it fails).
Define Safe Deployment Practices
Clearly define safe deployment practices for your workload.
Utilize Release Annotations as part of your failure mitigation strategies to keep track of deployments and other events, enabling faster incident diagnosis.
Consider Autoinstrumentation
Using autoinstrumentation, when possible, optimizes software engineering time by reducing the need for manual SDK updates and code maintenance.
OpenTelemetry Distro
If manual instrumentation is required, adopt the Azure Monitor OpenTelemetry Distro to avoid future migration challenges from the Application Insights SDKs (Classic API).
This approach aligns your instrumentation strategy with the evolving OpenTelemetry standard.
Use Connection Strings
Use connection strings for telemetry ingestion to improve reliability and remove dependencies on global ingestion endpoints.
Performance Recommendations
Define Performance Targets
Define clear performance targets for your application monitoring solution to establish a baseline for measurement and optimization.
Conduct Capacity Planning
Understand your application's usage patterns, the volume of incoming data, and review ingestion and sample rates to plan for capacity effectively.
Choose the Right Region
Deploy your Application Insights resource in the same region as your application and its Log Analytics workspace.
This minimizes latency and potential performance bottlenecks.
Evaluate Resource Needs
Carefully evaluate how many Application Insights resources you need.
While monitoring multiple applications or components with a single resource can provide a holistic view, it may impact the performance of features like Application Map and Usage.
Optimize Code and Infrastructure
Regularly evaluate and optimize your custom Application Insights code.
Reduce complexity, improve performance, and ensure the code is up-to-date with the latest SDKs.
Understand Usage Patterns
Monitor ingestion and sample rates to understand your data usage patterns.
Adjust them accordingly to optimize performance. Consider reducing the amount of custom metrics being collected.
Continuously Optimize Performance
Utilize built-in features like Smart Detection, queries, and dashboards to proactively identify components that might be experiencing performance issues.
Instance Per Workload Per Environment
This practice prevents mixing telemetry from different application versions, which can improve the performance of analysis and troubleshooting.
Set Appropriate Profiling Settings
When using profiling, ensure the profiling frequency and duration are set appropriately to avoid adding excessive overhead to the running process.