The last two posts had me writing about how logging can be implemented in your Azure Functions and how you can reuse class libraries using a different logging library, like log4net. You probably already have some logging- and monitoring system in place, but if you’re starting to use Azure Functions (or any other Azure service for that matter), the best tooling to use is Application Insights, in my opinion. You don’t even have to use Azure services in order to use Application Insights. You can also integrate it with any other on-premise server or client application.

For those of you who aren’t yet familiar with Application Insights, you should check it out immediately! It’s an awesome tool in Azure which enables you to view logging, metrics, exceptions, performance and more of your applications. It’s also possible to create enormous dashboards, reports and alerts, so everything you need in order to monitor your applications. A real must-have for a professional devops team.

Integrate with Azure Functions

Integrating your Azure Functions (Function App) with Application Insights is pretty straightforward.
The easiest way is to integrate is by selecting Application Insights when creating a Function App. Just press `On`, the location you want it deployed and proceed with creating the Function App. This will make sure the newly created Application Insights instance will be used by your Function App.

image

If you have neglected to turn this feature on, or have decided you want to use Application Insights after the Function App has been created, no worries you can still turn on this feature. If you need to integrate it manually, you should navigate to the `Application Settings` of your Function App and add a new entry with the key name `APPINSIGHTS_INSTRUMENTATIONKEY` and the value has to be the Instrumentation Key which can be found on the overview page of your Application Insights instance.

Having done so, you will immediately see a new notification popping up on the `Monitor` page of your Azure Function stating you can now check out your monitoring over there!

clip_image001

That’s all you need to do in order to integrate a Function App with this awesome monitoring system. Your operationsdevops people will be grateful for adding this to your solution.

Why add it?

‘Easy to add’ isn’t a very compelling reason to add Application Insights to your Function App. But if you take a moment to check out all of the basic features, I think you will see the power of this tool.

All of the metrics!

Just taking a look at the Overview page is fascinating already. Over here you are able to see how many server instances are running at this time, the response times of your functions and how many requests are being handled at a specific timespan.

clip_image001[5]

Clicking on either one of these graphs will show you even more details!

I like the Live Stream option a lot, because it gives me the feeling I’m an uber-cool-operations-guy by showing me a lot of graphs and telemetry data which gives you a good first impression of the status of your application.

clip_image001[7]

I know it can be quite overwhelming at first. Just spend a couple of minutes clicking and investigating all of the options is enough for most developers to understand what’s going on and which information is useful for your job.

One of the other useful pages available is the Performance page, this is especially useful if you’re providing public HTTP triggered endpoints via your Azure Functions.

clip_image001[9]

You can do some filtering, selecting timespans and inspect a lot of other metrics which have to do with performance of your piece of code.

Logging & Exceptions

Even though all of those metrics are useful in a production environment, as a developer I don’t do a lot with all of this information (mostly because I don’t have permission to view this data in Production).

What I do find interesting though is the availability of logging and being able to query through it quite fast. As I already mentioned, all of the Azure Function logging is stored in Application Insights (if configured) and you are able to search through all of this logging in the Search page. This is the main reason why I’ve spent some time in configuring log4net with a special appender in order to see ALL of my logging in Application Insights.

Just head down to the Search page and you’ll see all of the logging which has occurred in a specific interval.

clip_image001[11]

Obviously, you can change these filters and interval if you want to investigate some stuff. Most useful, to me, is the filtering on severity level of my logging messages and of course, being able to track (unhandled) Exceptions which are a special type.

clip_image001[1]

While this search page is great for doing some quick research and investigation, there’s an even more awesome page you can check out to see if there have been any exceptions. At this time, this page is labeled Failures. On this page you are able to see when exceptions have occurred and which type of exceptions have been thrown.

clip_image001[3]

Clicking on a specific exception will provide you with some more details.

clip_image001[5]

Zooming in further on such an exception will provide you with a lot of information of the client, server and even the stacktrace of the exception.

clip_image001[7]

As you probably  know, having a stacktrace is crucial information to do some proper investigation on problems which have occurred in the past. Along with all of the other logging you can store within Application Insights it provides you with all of the tooling and analysis information to do proper monitoring and troubleshooting of your service(s).

One of the other features which I like very much is the ability to set Alerts when something fishy is going on, like an increase of exceptions or Errors/Criticals logged being stored. This way you don’t have to keep track of the dashboards all day, but get notified when there’s something important to look at. I’ve only scratched the surface of Application Insights so far. There are a lot of other things you can check out and configure in order to automate your DevOps workflow as much as possible.

I need more!

Even though I like Application Insights very much, I can image a ‘real’ operations persons will probably find it a bit too ‘light’ as it might not provide all of the information they want to see. Well, no worries! The team has you covered and has added a button labelled Analytics on the overview page.

image

Pressing this button will navigate you to a new environment which is meant for power users of this system. You can do some SQL-like statements over here, create charts, query just about everything and visualize it just the way you want.

image

I’m not very familiar with this piece of tooling (yet), but it sure looks amazing and I know I’ll put some time in this as it appears to be even more powerful and useful compared to the ‘default’ Application Insights. My guess this piece has been built on top of the Kusto platform, which is an great piece of technology I’d like to get my hands on! I’ll be sure to follow up on these pieces of tooling, but for now I’ll leave it to this and hope I’ve triggered you in using Application Insights!

Creating a solution with multiple small services is great of course. It provides you with a lot of flexibility and scalability.

There are however a couple of things you have to think about when designing and developing a solution with multiple services. One of the things you need to figure out is how to implement proper logging. For an actual production system you need to have this in place in order to monitor and debug the overall solution.

We, developers using Azure Functions, are already blessed with some logging mechanisms and tools provided out of the box! The out of the box stuff is pretty basic, but it gets the job done and will make your life much easier when the need arises to analyze a production issue.

Logging in your function

When first creating your Azure Function you will probably see a parameter being passed in your `Run` method called `log`.

public static class Function1
{
    [FunctionName("Function1")]
    public static async Task<HttpResponseMessage> Start(
        [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)]
        HttpRequestMessage req, 
        TraceWriter log)
    {
        log.Info("C# HTTP trigger function processed a request.");

        // Logic
    }
}

This `log`-parameter can be used to do some simple logging inside your Azure Function. When invoking this function you’ll be able to see the specified log entry in your console.

[10-4-2018 19:12:12] Function started (Id=a669fc9a-b15b-44b9-a863-47a303aca6b6)
[10-4-2018 19:12:13] Executing 'Function1' (Reason='This function was programmatically called via the host APIs.', Id=a669fc9a-b15b-44b9-a863-47a303aca6b6)
[10-4-2018 19:12:13] C# HTTP trigger function processed a request.

All of the other common logging levels, like `Error`, `Warning` and `Verbose`, are also implemented in the TraceWriter.

After having deployed the Azure Function to Azure you will also be able to see the log messages in the Portal.

2018-04-10T19:30:09.112 [Info] Function started (Id=71cead2b-3f90-4c04-8c82-42f6a885491a)
2018-04-10T19:30:09.127 [Info] C# HTTP trigger function processed a request.
2018-04-10T19:30:09.205 [Info] Function completed (Success, Id=71cead2b-3f90-4c04-8c82-42f6a885491a, Duration=87ms)

Checking out the log messages inside the log monitor is nice and all, but the best thing over here is the (automatic) integration with Application Insights! If you have activated this feature you will be able to see all logs in the Application Insights Search blade.

image

I totally recommend using Application Insights to monitor your application, along with all of the logging, as it’s an awesome piece of technology with endless possibilities for your DevOps teams. I’ll share more of my experience with Application Insights in a later post, because it’s too much to share in a single post.

Using the `TraceWriter` is a nice way to get you started with logging, but it’s probably not something you want to use in an actual application.

Introducing the `ILogger`

A different way to start logging is by using the `ILogger` from the `Microsoft.Extensions.Logging` namespace. You can just replace the earlier mentioned `TraceWriter` with the `ILogger`, change the logging methods and be done with it. The Azure Functions runtime will make sure some concrete implementation is being injected which will act similarly to the `TraceWriter`.
The main benefit will be the improved testability of your Azure Function as this parameter is much easier to mock.

One thing I did notice though is the `ILogger` doesn’t output any logging to the Azure Functions console application when running locally. There appears to be an (now open) issue on GitHub which mentions this bug. But don’t worry, you can see all the expected logging in the available tooling of the Azure Portal.

As mentioned, the `ILogger` is a better solution if you want to use Azure Functions in your actual project, because it offers better testability. However, the functionality is quite limited and you can’t extend much on the `ILogger` at this time. If you already have some kind of logging mechanism or framework elsewhere in your solution your probably don’t want to introduce this new logging implementation. My go-to logging framework is log4net and I'd like to use it for the logging inside my Azure Functions also. In order to do so you need to configure a thing or two when setting up log4net. It isn’t very complicated, but something to keep in mind when setting up the Azure Functions.
I’ll describe the necessary steps in a later post.

For now my main advice is just to use the `ILogger` instead of the `TraceWriter`. Both work very well, but `ILogger` offers some advantages compared to the `TraceWriter`.

In my previous post I’ve described how to use Application Insights and use it within your new web application. Most of us aren’t working in a greenfield project, so new solutions have to be integrated with the old.

The project I’m working on uses log4net for logging messages, exceptions, etc. In order for us to use Application Insights, we had to search for a solution to integrate both. After having done some research on the subject we discovered this wasn’t a big problem.

The software we are working on are Windows Services and Console Applications, so we should not add the Application Insights package for web applications. For these kind of applications it’s enough to just add the core package to your project(s).

image_thumb7

Once added, we are creating the TelemetryClient in the Main of the application.

private static void Main(string[] args)
{
	var telemetryClient = new TelemetryClient { InstrumentationKey = "[InstrumentationKey]" };

	/*Do our application logic*/

	telemetryClient.Flush();
}

You will notice we are setting the InstrumentationKey property explicitly. That’s because we don’t use an ApplicationInsights.config file, like in the web application example and this key has to be specified in order to start logging.

This final flush will make sure all pending messages will be pushed to the Application Insights portal right away.
Now you might not see your messages in the portal right away. We discovered this while testing out the libraries. The reason for this is probably due to some caching in the portal or all messages being handled by some queue mechanism on the backend (assumption on my behalf). So don’t be afraid, your messages will show up within a few seconds/minutes.

After having wired up Application Insights, we still had to add it to log4net. When browsing through the nuget packages we noticed the Log4NetAppender for Application Insights.

After having added this package to our solution, the only thing we had to do is creating a new log4net appender to the configuration.

<root>
    <level value="INFO" />
    <appender-ref ref="LogFileAppender" />
    <appender-ref ref="ColoredConsoleAppender" />
    <appender-ref ref="aiAppender" />
</root>
<appender name="aiAppender" type="Microsoft.ApplicationInsights.Log4NetAppender.ApplicationInsightsAppender, Microsoft.ApplicationInsights.Log4NetAppender">
    <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%message%newline" />
    </layout>
</appender>

This appender will make sure all log4net messages will be sent to the Application Insights portal.

As you can see, adding Application Insights to existing software is rather easy. I would really recommend using one of these modern, cloud based, logging solutions. It’s much easier to use this compared to ploughing through endless log files on the disks or creating your own solution.

Some time ago the Application Insights became available as a preview in the Azure portal. Application Insights helps you monitor the state of an application, server, clients, etc. As said, it’s still in preview, but it’s rather stable and very easy to use and implement in your applications.
The documentation is still being worked on, but with all the getting started guides on the Microsoft site you can kick start your project with it in a couple of minutes.

The main reason for me to dig into Application Insights is because we still had to implement a proper logging solution for our applications which are migrating to the Azure Cloud.  As it happened, Application Insights just became available at the time and because of the tight Azure integration we were really eager to check it out (not saying you can’t use it with non-Azure software of course).

If you want to start using Application Insights, the first thing you will have to do is creating a new Application Insights application. You will be asked what type of application you want to use.

image

It’s wise to select the proper application type over here, because a lot of settings and measuring graphs will be set for you depending on the choice made over here. For this example I’ll choose the ASP.NET web application. After having waited for a few minutes the initial (empty) dashboard will be shown.

image

As you can see, the dashboard is filled with (empty) graphs useful for a web application. In order to see some data over here, you need to push something to this portal. To do this, you’ll need the instrumentation key of this specific Application Insights instance. This instrumentation key can be found within the properties of the created portal.

image

Now let’s head to Visual Studio and create a new MVC Web Application and add the Microsoft.ApplicationInsights.Web nuget package to the project.

image

This package will add the ApplicationInsights.config file to the web project in which you have to specify the instrumentation key.

<InstrumentationKey>2397f37c-669d-41a6-9466-f4fef226fe41</InstrumentationKey>

The web application is now ready to log data to the Application Insights portal.
You might also want to track some custom events, exceptions, metrics, etc. into the Application Insights portal. To do this, just create a new TelemetryClient to your code and start pushing data. An example is shown below.

public class HomeController : Controller
{
	public ActionResult Index()
	{
		var tc = new TelemetryClient();
		tc.TrackPageView("Index");

		return View();
	}

	public ActionResult About()
	{
		ViewBag.Message = "Your application description page.";
		var tc = new TelemetryClient();
		tc.TrackTrace("This is a test tracing message.");
		tc.TrackEvent("Another event in the About page.");
		tc.TrackException(new Exception("My own exception"));
		tc.TrackMetric("Page", 1);
		tc.TrackPageView("About");

		return View();
	}

	public ActionResult Contact()
	{
		ViewBag.Message = "Your contact page.";
			
		var tc = new TelemetryClient();
		tc.TrackPageView("Contact");

		return View();
	}
}

Running the website an navigating between the different pages (Home, About and Contact) will result in the web portal pushing data to Application Insights. Navigating back to the selected Application Insights portal will show the results in the dashboard.

image

You can click on any of the graphs over here and see more details about the data. I’ve clicked on the Server Requests graph to zoom a bit deeper into the requests.

image

As you can see, a lot of data is send to the portal, response time, response code, size. The custom messages, like the exceptions, are also pushed towards the Application Insights portal. You can see these when zooming in on the Failures graph.

image

As you can see in the details of an exception, a lot of extra data is added which will make it easier to analyze the exceptions.

This post talked about adding the Application Insights libraries to an ASP.NET Web Application via a nuget package, but that’s not the only place you can use this tooling. It’s also possible to add the Application Insights API for Javascript Applications to your application. This way you are able to push events to the portal on the front end. Awesome, right?

Of course there are plenty of other logging solutions available, but Application Insights is definitely a great contender in this space in my opinion.