Azure App Service - logging and monitoring made easy
Updated: Aug 12, 2019
Azure Application Insights is one of the services that makes Azure an amazing development, monitoring and management environment. Comparable capabilities are much more difficult to achieve in terms of effort and cost in on-premises deployments.
Setting up Application Insights Integration
Microsoft provides a good documentation how integrate your application with Azure Application Insights
Note: Configuration of Java applications can be a little more involved therefore pre-configured java application is provided in GitHub.
Once integrated you can query your application logs in Application Insights using Log Analytics.
No longer you need to browse your application logs with grep, awk and create scheduled tasks to alert you of undesirable behavior. Instead you direct your application logs to Azure Log Analytics, use Log Analytics query language and raise to Azure Alerts for results that we want to be notified about.
And you can do it literally within minutes even if you never have done it before.
For example we can notify ourselves when application returns http error responses.
Go to your App Service Application Insights blade
Go to Log Analytics
Application Insights will provide sample queries to get you started quick
Your application and web server logs are represented in Log Analytics by "traces" and "requests" respectively as shown in examples below.
"traces" category allows to query your application logs. For example if you put information logs into your application code
You can query application logs easily
"requests" category is essentially you web server http logs.
For example lets find http error responses within time period.
You can see exception in ApplicationInsights UI or query them in format that you prefer to see in AI logs.
If you have multiple App Services sharing Application Insights instance then cloud rule name allow to differentiate what App Service the produced the exception.
You can display timestamps as UTC or Local Time
Monitor Java applications
Setting up monitoring for Java requires a little more configuration compared to .Net. For us the most useful was the following in web.config (the code is available in AzureAIJavaSpringBoot in GitHub)
You need to setup Application Insights to integrate with.
For example like below:
Setup Application Insights key in Azure Portal:
Configure AI key in application.properties
Configure logger with Application Insights. For example for Logback use logback-spring.xml
Then log messages in you code as usual
Assuming LogMessage is configured in App Service Application Settings
We now can query the output in Application Insights. For example
traces | where timestamp > ago(1h) | where message contains 'build' | sort by timestamp desc
Beyond Application Insights
Sometimes you may be getting 5xx errors raised before your code is reached... This can happen due to mis-configuration of resource starvation. In this case no records will appear in Application Insights.
To capture detailed traces of those errors you need to go to Monitoring->Diagnostic logs blade.
Please note at the moment (2019-May-20) there seems to be a bug in portal code that do not always render "Detailed Error Messaging" and "Failed Request tracing"settings. If this happens to you just refresh portal several times until the settings appear.
Please see screen shots below.
* Failed Request Tracing is missing:
* Failed Request Tracing Found
For example if you java application does not start and returns error like below
Check your recent std out or the logfile and most probably find the root cause of the error
Also following section can be helpful for the troubleshooting in web.config
The section below can affect your application performance and availability. Please make sure to remove this code after troubleshooting.
<remove path="*" />
<add provider="ASP" verbosity="Verbose" />
<add provider="ASPNET" areas="Infrastructure,Module,Page,AppServices" verbosity="Verbose" />
<add provider="ISAPI Extension" verbosity="Verbose" />
<add provider="WWW Server" areas="Authentication,Security,Filter,StaticFile,CGI,Compression,Cache,RequestNotifications,Module,FastCGI" verbosity="Verbose" />
<failureDefinitions statusCodes="200-999" />