07: Logging with App Insight

Before we start, we need to verify if Azure Function App is connected to an instance of App Insight or not.

To check this, go to Application Insights in Azure Function and verify if it is connected to an App Insight Resource. If not, we either need to create a new App Insight resource and attach it to Function App or use an existing App Insight app.

Get App Insight Connection String

1. Click on App Insight attached to Azure Function
2. Copy Instrumentation Key and Connection String from overview tab

Code Changes

Notes:

1. An AI need a unique role_name instance to separate events from different app. This can be done using a callback function. (function name: callback_add_role_name)
2. Azure has custom logger that can emits logs to AI. This is available in opencensus.ext.azure.log_exporter module.
3. We need to enable traces for logging and if there is need to trace HTTP calls, then we also need to enable it for requests. Please check here for all available integration.

1. Create a folder appinsight in src.

2. Create a python file logger.py in src/appinsight

3. In src/appinsight/logger.py, we will create custom logger for AzureLogHandler

   import inspect
   import logging
   from functools import wraps
   from logging import Logger
   
   from opencensus.ext.azure.log_exporter import AzureLogHandler
   from opencensus.trace import config_integration
   from opencensus.trace import execution_context
   from opencensus.trace.tracer import Tracer
   
   config_integration.trace_integrations(['logging', 'requests'])
   
   AI_KEY = "xxxxxx-ae15-44c2-xxxx-xxxxxxxxx"
   AI_CONNECTION_STRING = "InstrumentationKey=xxxxxx-ae15-44c2-xxxx-xxxxxxxxx;IngestionEndpoint=https://westus-0.in.applicationinsights.azure.com/;LiveEndpoint=https://westus.livediagnostics.monitor.azure.com/"
   WEBSITE_NAME = "function-demo"
   
   
   class CustomDimensionsFilter(logging.Filter):
       """Add custom-dimensions in each log by using filters."""
   
       def __init__(self, custom_dimensions=None):
           """Initialize CustomDimensionsFilter."""
           super().__init__()
           self.custom_dimensions = custom_dimensions or {}
   
       def filter(self, record):
           """Add the default custom_dimensions into the current log record."""
           dim = {**self.custom_dimensions, **getattr(record, "custom_dimensions", {})}
           record.custom_dimensions = dim
           return True
   
   
   def callback_add_role_name(envelope):
       """Add role name for logger."""
       envelope.tags['ai.cloud.role'] = WEBSITE_NAME
       envelope.tags["ai.cloud.roleInstance"] = WEBSITE_NAME
   
   
   def get_logger(name: str, propagate: bool = True, custom_dimensions=None) -> Logger:
       if custom_dimensions is None:
           custom_dimensions = {}
   
       # Azure Log Handler
       azure_log_handler = AzureLogHandler(connection_string=AI_CONNECTION_STRING)
       azure_log_handler.add_telemetry_processor(callback_add_role_name)
       azure_log_handler.setLevel(logging.DEBUG)
   
       # Formatter
       formatter = logging.Formatter('%(asctime)s traceId=%(traceId)s api=%(name)s.%(funcName)s '
                                     '[%(levelname)-7s]: %(message)s')
       azure_log_handler.setFormatter(formatter)
       azure_log_handler.addFilter(CustomDimensionsFilter(custom_dimensions))
   
       # Logger
       logger = logging.getLogger(name)
       logger.setLevel(logging.DEBUG)
       logger.addHandler(azure_log_handler)
       logger.propagate = propagate
       if not logger.handlers:
           logger.addHandler(azure_log_handler)
       return logger
   

4. Now let’s replace logger with an AzureLogHandler in all place i.e.

replace

import logging
logger = logging.getLogger(__name__)

with

from src.appinsight.logger import get_logger
logger = get_logger(__name__)

5. Once updated as above, restart azure function locally from VSCode to test.
6. To Test, let us invoke API: http://localhost:7071/vault?secret=name with/without Authorization Header. Event Types = Request may not be logged for localhost. We can push code and test on Azure Function deployed on Cloud for this to test again.
7. Push the code.

Transaction Search

!!! info “It takes upto 3-5 minutes for Transaction and Traces to propagate in App Insights”

1. Open App Insight attached to Azure Function.
2. Under Investigate > Transaction Search
3. We are returning x-function-invocationId as response header from our code. We can use this to search Transaction related to a specific request.