Automatic Input Blob Binding in Azure Functions from Queue Trigger Message Data

Reading additional blob content when an Azure Function is triggered can be accomplished by using an input blob binding by defining a parameter in the function run method and decorating it with the [Blob] attribute.

For example, suppose you have a number of blobs that need converting in some way. You could initiate a process whereby the list of blob files that need processing are added to a storage queue. Each queue message contains the name of the blob that needs processing. This would allow the conversion function to scale out to convert multiple blobs in parallel.

The following code demonstrates one approach to do this. The code is triggered from a queue message that contains text representing the input bob filename that needs reading, converting, and then outputting to an output blob container.

using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;

namespace FunctionApp1
{
    public static class ConvertNameCase
    {
        [FunctionName("ConvertNameCase")]
        public static void Run([QueueTrigger("capitalize-names")]string inputBlobPath)
        {
            string originalName = ReadInputName(inputBlobPath);

            var capitalizedName = originalName.ToUpperInvariant();

            WriteOutputName(inputBlobPath, capitalizedName);
        }
        
        private static string ReadInputName(string blobPath)
        {
            CloudStorageAccount account = CloudStorageAccount.DevelopmentStorageAccount;
            CloudBlobClient blobClient = account.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference("names-in");

            var blobReference = container.GetBlockBlobReference(blobPath);

            string originalName = blobReference.DownloadText();

            return originalName;
        }

        private static void WriteOutputName(string blobPath, string capitalizedName)
        {
            CloudStorageAccount account = CloudStorageAccount.DevelopmentStorageAccount;
            CloudBlobClient blobClient = account.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference("names-out");

            CloudBlockBlob cloudBlockBlob = container.GetBlockBlobReference(blobPath);
            cloudBlockBlob.UploadText(capitalizedName);            
        }

    }
}

In the preceding code, there is a lot of blob access code (which could be refactored). This function could however be greatly simplified by the use of one of the built-in binding expression tokens. Binding expression tokens can be used in binding expressions and are specified inside a pair of curly braces {…}. The {queueTrigger} binding token will extract the content of the incoming queue message that triggered a function.

For example, the code could be refactored as follows:

using System.IO;
using Microsoft.Azure.WebJobs;

namespace FunctionApp1
{
    public static class ConvertNameCase
    {
        [FunctionName("ConvertNameCase")]
        public static void Run(
        [QueueTrigger("capitalize-names")]string inputBlobPath,
        [Blob("names-in/{queueTrigger}", FileAccess.Read)] string originalName,
        [Blob("names-out/{queueTrigger}")] out string capitalizedName)
        {
                capitalizedName = originalName.ToUpperInvariant();         
        }
}

In the preceding code, the two [Blob] binding paths make use of the {queueTrigger} token. When the function is triggered, the queue message contains the name of the file to be processed. In the two [Blob] binding expressions, the {queueTrigger} token part will automatically be replaced with the text contents of the incoming message. For example if the message contained the text “File1.txt” then the two blob bindings would be set to names-in/File1.txt and names-out/File1.txt respectively. This means the input blob nameBlob string will automatically be read when the function is triggered,

To learn more about creating precompiled Azure Functions in Visual Studio, check out my Writing and Testing Precompiled Azure Functions in Visual Studio 2017 Pluralsight course.

You can start watching with a Pluralsight free trial.

SHARE:

Dynamic Binding in Azure Functions with Imperative Runtime Bindings

When creating precompiled Azure Functions, bindings (such as a blob output bindings) can be declared in the function code, for example the following code defines a blob output binding:

[Blob("todo/{rand-guid}")]

This binding creates a new blob with a random (GUID) name. This style of binding is called declarative binding, the binding details are declared as part of the binding attribute.

In addition to declarative binding, Azure Functions also offers imperative binding. With this style of binding, the details of the binding can be chosen at runtime. These details could be derived from the incoming function trigger data or from an external place such as a configuration value or database item

To create imperative bindings, rather than using a specific binding attribute, a parameter of type IBinder is used. At runtime, a binding can be created (such as a blob binding, queue binding, etc.) using this IBinder. The Bind<T> method of the IBinder can be used with T representing an input/output type that is supported by the binding you intend to use.

The following code shows imperative binding in action. In this example blobs are created and the blob path is derived from the incoming JSON data, namely the category.

public static class CreateToDoItem
{
    [FunctionName("CreateToDoItem")]
    public static async Task<HttpResponseMessage> Run(
        [HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequestMessage req,
        IBinder binder,
        TraceWriter log)
    {
        ToDoItem item = await req.Content.ReadAsAsync<ToDoItem>();
        item.Id = Guid.NewGuid().ToString();

        BlobAttribute dynamicBlobBinding = new BlobAttribute(blobPath: $"todo/{item.Category}/{item.Id}");

        using (var writer = binder.Bind<TextWriter>(dynamicBlobBinding))
        {
            writer.Write(JsonConvert.SerializeObject(item));
        }

        return req.CreateResponse(HttpStatusCode.OK, "Added " + item.Description);
    }
}

If the following 2 POSTS are made:

{
    "Description" : "Lift weights",
    "Category" : "Gym"
}
{
    "Description" : "Feed the dog",
    "Category" : "Home"
}

Then 2 blobs will be output with the following paths - note the random filenames and imperatively-bound paths: Gym and Home :

http://127.0.0.1:10000/devstoreaccount1/todo/Gym/5dc4eb72-0ae6-42fc-9a8b-f4bf646dcd28

http://127.0.0.1:10000/devstoreaccount1/todo/Home/530373ef-02bc-4200-a4e7-948448ac081b

SHARE:

Create Precompiled Azure Functions With Azure Event Grid Triggers

Visual Studio can be used to create precompiled Azure Functions using standard C# classes and tools/techniques and then they can be published to Azure.

This article assumes you’ve created the resources (resource group, Event Grid Topic, etc.) from this previous article.

In Visual Studio 2017, create a new Azure Functions project.

Next update the pre-installed Microsoft.NET.Sdk.Functions NuGet package to the latest version.

To get access to the Azure Event Grid function trigger attribute, install the Microsoft.Azure.WebJobs.Extensions.EventGrid NuGet package (this package is currently in preview/beta).

Add a new class to the project with the following code:

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.EventGrid;
using Microsoft.Azure.WebJobs.Host;

namespace DCTDemos
{
    public static class Class1
    {
        [FunctionName("SendNewLeadWelcomeLetter")]
        public static void SendNewLeadWelcomeLetter([EventGridTrigger] EventGridEvent eventGridEvent, TraceWriter log)
        {
            log.Info($"EventGridEvent" +
                $"\n\tId:{eventGridEvent.Id}" +
                $"\n\tTopic:{eventGridEvent.Topic}" +
                $"\n\tSubject:{eventGridEvent.Subject}" +
                $"\n\tType:{eventGridEvent.EventType}" +
                $"\n\tData:{eventGridEvent.Data}");
        }
    }
}

Notice in the preceding code, the method name SendNewLeadWelcomeLetter is the same as specified in the function name attribute, this may be required due to a bug in the current preview/beta implementation – if these are different your function may not be executed when an event occurs.

Right-click on the function project and choose publish. Follow the wizard and create a new Function App and select your resource group where your Event Grid Topics is. Select West US 2 if you need to create any new Azure resources/storage account/etc..

Once deployed, head over to Azure Portal, open your new function app and select the newly deployed SendNewLeadWelcomeLetter function:

Adding an Azure Event Grid subcription for an Azure Function

At the top right select Add Event Grid subscription. And follow the wizard to create a new subscription - this will enable the new function to be triggered by an Event Grid Subscription. As part of the subscription we’ll limit the event type to new-sales-lead-created:

Adding an Azure Event Grid subcription for an Azure Function

Next go to the function app platform features tab and select Log Streaming. We can now use Postman to POST the following JSON to the Event Grid Topic we created earlier.

[
    {
        "id": "1236",
        "eventType": "new-sales-lead-created",
        "subject": "myapp/sales/leads",
        "eventTime": "2017-12-08T01:01:36+00:00",
        "data":{
            "firstName": "Amrit",
            "postalAddress": "xyz"
        }
    }
]

Head back to the streaming logs and you should see your precompiled Azure Function executing in response to the Event Grid event:

2017-12-08T06:38:25  Welcome, you are now connected to log-streaming service.

2017-12-08T06:38:49.841 Function started (Id=ec927bc1-fa15-4211-a7bd-8e593f5d4840)

2017-12-08T06:38:49.841 EventGridEvent
    Id:1234
    Topic:/subscriptions/797e1c4e-3fd4-4cd6-84b8-ef103cee8b6b/resourceGroups/DCTEGDemo/providers/Microsoft.EventGrid/topics/sales-leads
    Subject:myapp/sales/leads
    Type:new-sales-lead-created
    Data:{

  "firstName": "Amrit",

  "postalAddress": "xyz"

}

2017-12-08T06:38:49.841 Function completed (Success, Id=ec927bc1-fa15-4211-a7bd-8e593f5d4840, Duration=0ms)

 

To learn how to create precompiled Azure Functions in Visual Studio, check out my Writing and Testing Precompiled Azure Functions in Visual Studio 2017 Pluralsight course.

You can start watching with a Pluralsight free trial.

SHARE:

Getting Started with Azure Event Grid

In a previous article we got an introduction to Azure Event Grid, if you’re new to Event Grid you should check it out first to familiarise yourself with some basic concepts.

In this article we’ll create an Azure Event Grid topic and subscription and see it in action.

First off, if you want to create a free Azure Account you can do so, then log into the Azure portal.

Next go and create a new resource group, Azure Event Grid is currently in preview and only available in selected locations such as West US 2.

Creating a new resource group

Once the resource group is created, head down to the More services option and search for Event Grid.

Navigating to Event Grid Topics

There are topics provided by Azure services (such as blob storage ) and there is also the ability to create your own custom topics for custom applications/third parties/etc.

Click Event Grid Topics and this will take you to a list of all your topics. Click the +Add button to begin creation of a custom topic. Give the topic a name sales-leads and choose the resource group created earlier, once again choose West US 2.

Creating a new Azure Event Grid Topic

Click create, wait for the deployment to complete and hit refresh in the topics list to see your new topic:

Azure Event Grid topic added

Click on the newly added sales-leads topic, notice the overview showing publish metrics:

Event Grid Topic details

At the top right hover over the Topic Endpoint and click the button to copy this to the clipboard (we’ll use this later):

Getting Event Grid Topic endpoint

In this example the copied endpoint is: https://sales-leads.westus2-1.eventgrid.azure.net/api/events

We’ll also need an access key to be able to HTTP POST to this custom topic later, to do this click the Access keys option and copy Key 1 for later use:

Getting access key for Azure Event Grid topic

Click back on Overview and click the +Event Subscription button:

Creating a new Azure Event Grid Subscription

In this example we’ll create a subscription that will call an external (to Azure) service that will mail a conference brochure to all new sales leads. In this example we are simulating a temporary extension to the sales system for a limited period during the run-up to a sales conference. This is one use case for Azure Event Grid that allows extension of a core system without needing to modify it (assuming that events are being emitted).

To simulate this external service we’ll use RequestBin which you can learn more about in this article. Once you’ve created your request bin, take a note of the Bin URL.

Creating a RequestBin URL

Fill out the new event subscription details:

  • Name: send-upcoming-conference-brochure
  • Subscribe to all event types: Untick
  • Event Types: new-sales-lead-created
  • Subscriber endpoint: https://requestb.in/1bbopge1 (this is the RequestBin URL created above)

Event subscription details

Click Create.

To recap, there is now a custom topic called sales-leads that we can publish events to at its URL: https://sales-leads.westus2-1.eventgrid.azure.net/api/events. There is also an event subscription set up for this topic but that is limited to only those events published of type new-sales-lead-created. This event subscription uses the Azure Event Grid WebHooks event handler to HTTP push events to the RequestBin URL.

To see this in action, open Postman and select POST and paste the topic URL (https://sales-leads.westus2-1.eventgrid.azure.net/api/events). Add a header called aeg-sas-key and paste in the key that was copied earlier:

Basic Postman setup

The final thing to do is define the event data that we want to publish:

[
    {
        "id": "42",
        "eventType": "new-sales-lead-created",
        "subject": "myapp/sales/leads",
        "eventTime": "2017-12-07T01:01:36+00:00",
        "data":{
            "firstName": "Jason",
            "postalAddress": "xyz"
        }
    }
]

Event JSON data

And then click Send in Postman. You should get a 200 OK response.

Heading back to the RequestBin window and refreshing the page shows the subscription working and the event being pushed to RequestBin:

RequestBin receiving Azure Event Grid event

Because the event subscription is filtered on an event type of new-sales-lead-created, if we send a different event type from Postman (e.g.: "eventType": "new-sales-lead-rejected",), the subscription won’t activate nor push the event to RequestBin.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

Understanding Azure Event Grid

Azure Event Grid (currently in preview) is a managed publisher-subscriber service that pushes events to registered subscribers.

Azure Event Grid does not replace other services such as Azure Service Bus and it has a different focus. Whereas Azure Service Bus might be employed where you need very high reliability, message ordering etc, Azure Event Grid is more about emitting notifications of things that have happened.

Azure Event Grid uses a push model (with some retry logic built in) to push events to subscribers both inside and outside of Azure.

Messages and Events

One way to differentiate when Azure Event Grid may be more appropriate is to think of the publisher’s expectations. Firstly lets use a very general definition of a message as being a single piece of information that is produced somewhere and is (possibly) consumed somewhere. It this case we’re thinking about messages as individual “datagrams” as opposed to an ongoing/continuous stream of data.

If the sender of the message has an expectation when the message is sent we can think of this as a “message with intent”.

If the sender of the message has no expectation of what happens when the messages is sent we can think of this as a “message with no intent”.

For the sake of this article, we’ll call a “message with intent” a command and a “message with no intent” an event. Commands are messages  instructing the consumer to do something and that maybe return a result to the sender; events are messages  that represent a fact about something that has happened in the system.

Use Case

One use case for Azure Event Grids is to allow easier system extensibility beyond the core business functionality. For example in a sales system new sales leads are captured and stored in a database, this is the core system. This core system could also publish events with no expectation or knowledge about who may be responding to them. An example of an event could be when a new sales lead is added. When the new lead is entered into the core system, a “new-lead” event is published into Azure Event Grid. Now anyone who is interesting in knowing when a new lead has been added can subscribe to this type of event and do something with it.

Azure Event Grid Terminology

An Azure Event Grid event describes what happened in the system represented as JSON data. It contains the custom data specific to the type of event, in addition to information that is contained in all events such as the event source, event time, and a unique event identifier. The full Azure Event Grid event schema is available as part of the docs.

An event source is the place the event happened, for example the sales system, Azure Storage, etc. Event sources publish events. At present the docs list the following supported event sources (with more to be added in the future):

  • Resource group management operations
  • Azure subscriptions management operations
  • Event Hubs
  • Storage Blobs
  • Custom Topics (HTTP POST-able endpoints)

A topic is an arbitrary categorization of events. Once created, topics have endpoints that event sources publish event to. Events of different types can be sent to the same topic, for example a “sales-leads” topic that holds both “new-lead” and “converted-lead” events.

Event subscriptions wire up topics to event handlers. A topic can have 0, 1, or many subscriptions.

An Event Grid event handler is the place where a subscription sends an event to, for example sending the event using the HTTP webhook event handler. At present the docs list the following Azure event handlers (with more to be added in the future):

  • Azure Functions
  • Logic Apps
  • Azure Automation
  • WebHooks
  • Microsoft Flow

An event consumer (while not explicitly stated in the docs) can be thought of as the thing that the event handler pushes the event to. The event consumer receives the pushed event and uses it, for example an Azure Function responding to a “new-lead” event and sending out a conference invitation letter/email.

Costs

The cost of using Azure Event Grid is based on usage at the “operation” level with an “operation” being defined as “all ingress events, advanced match, delivery attempt, and management calls”. At the time of writing the preview cost is USD $0.30 per million operations with 100,000 free operations per month. You can find the latest  pricing here.

To learn more, check out the Azure Event Grid  docs.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

New Pluralsight Course: Writing and Testing Precompiled Azure Functions in Visual Studio 2017

Azure Functions have come a long way in a short time. With newer releases you can now create functions in Visual Studio using standard C# class files along with specific attributes to help define triggers, bindings, etc. This means that all the familiar powerful Visual Studio tools, workflows, NuGet packages, etc. can be used to develop Azure Functions. Visual Studio also provides publish support so you can upload your functions to the cloud once you are happy with them. Another feature that makes developing functions in Visual Studio easier is the local functions runtime that let’s you run and debug functions on your local development machine, without needing to publish to the cloud just to test them.

In my new Writing and Testing Precompiled Azure Functions in Visual Studio 2017 Pluralsight course you will learn how to:

  • Set up your local development environment
  • Develop and test Azure Functions locally
  • Publish functions to Azure
  • Create functions triggered from incoming HTTP requests
  • Trigger functions from Azure Storage queues and blobs
  • Trigger functions from Azure Service Bus and Azure Event Hubs
  • Trigger functions periodically on a timer
  • Unit test Azure Function business logic

Check out the full course outline for more details.

You can start watching with a Pluralsight free trial.

SHARE:

Using PostgreSQL Document Databases with Azure Functions and Marten

With the appearance of managed PostgreSQL databases on Azure, we can now harness the simplicity of Marten to create document databases that Azure Functions can utilize.

Marten is on open source library headed by Jeremy Miller and offers simple document database style persistence for .NET apps which means it can also be used from Azure Functions.

Creating a PostgreSQL Azure Server

Log in to the Azure Portal and create a new “Azure Database for PostgreSQL”:

Creating a PostgreSQL Azure Server

You can follow these detailed steps to create and setup the PostgreSQL instance. Be sure to follow the firewall instructions to be able to connect to the database from an external source.

Creating a PostgreSQL Azure Server

Connecting and Creating a Database Using pgAdmin

pgAdmin is a tool for working with PostgreSQL database. Once installed, a new connection can be added to the Azure database server (you’ll need to provide the server, username, and password).

Connecting and Creating a Database Using pgAdmin

Once connected, right-click the newly added Azure server instance and choose Create –> Database. In this example a “quotes” database was added:

Connecting and Creating a Database Using pgAdmin

Notice in the preceding screenshot there are currently no tables in the database.

Reading and Writing to an Azure PostgreSQL Database from an Azure Function

Now we have a database, we can access it from an Azure Function using Marten.

First create a new Azure Functions project in Visual Studio 2017, reference Marten, and add a new POCO class called Quote:

public class Quote
{
    public int Id { get; set; }
    public string Text { get; set; }
}

Next add a new HTTP-triggered function called QuotesPost that will allow new quotes to be added to the database:

using System.Net;
using System.Net.Http;
using System.Threading.Tasks;
using Marten;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;

namespace MartenAzureDocDbDemo
{
    public static class QuotesPost
    {
        [FunctionName("QuotesPost")]
        public static async Task<HttpResponseMessage> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "quotes")]HttpRequestMessage req, 
            TraceWriter log)
        {
            log.Info("C# HTTP trigger function processed a request.");

            Quote quote = await req.Content.ReadAsAsync<Quote>();

            using (var store = DocumentStore
                .For("host=dctquotesdemo.postgres.database.azure.com;database=quotes;password=3ncei*3!@)nco39zn;username=dctdemoadmin@dctquotesdemo"))
            {
                using (var session = store.LightweightSession())
                {
                    session.Store(quote);

                    session.SaveChanges();
                }
            }

            return req.CreateResponse(HttpStatusCode.OK, $"Added new quote with ID={quote.Id}");
        }
    }
}

Next add another new function called QuotesGet that will read quote data:

using System.Net;
using System.Net.Http;
using Marten;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;

namespace MartenAzureDocDbDemo
{
    public static class QuotesGet
    {
        [FunctionName("QuotesGet")]
        public static HttpResponseMessage Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "quotes/{id}")]HttpRequestMessage req, 
            int id, 
            TraceWriter log)
        {
            log.Info("C# HTTP trigger function processed a request.");

            using (var store = DocumentStore
                .For("host=dctquotesdemo.postgres.database.azure.com;database=quotes;password=3ncei*3!@)nco39zn;username=dctdemoadmin@dctquotesdemo"))
            {
                using (var session = store.QuerySession())
                {
                    Quote quote = session.Load<Quote>(id);
                    return req.CreateResponse(HttpStatusCode.OK, quote);
                }
            }                        
        }
    }
}

Testing the Azure Functions Locally

Hit F5 in Visual Studio to start the local functions runtime, and notice the info messages, e.g.

Http Function QuotesGet: http://localhost:7071/api/quotes/{id}
Http Function QuotesPost: http://localhost:7071/api/quotes

We can now use a tool like Postman to hit these endpoints.

We can POST to “http://localhost:7071/api/quotes” the JSON: { "Text" : "Be yourself; everyone else is already taken." } and get back the response “"Added new quote with ID=3001"”.

If we use pgAdmin, we can see the mt_doc_quote table has been created by Marten and the new quote added with the id of 3001.

Querying Azure PostgreSQL with pgAdmin

 

Doing a GET to “http://localhost:7071/api/quotes/3001” returns the quote data:

{
    "Id": 3001,
    "Text": "Be yourself; everyone else is already taken."
}

Pricing details are available here.

To learn more about Marten, check out the docs or my Pluralsight courses Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten.

To learn more about Azure Functions check out the docs, my other posts or my Pluralsight course Azure Function Triggers Quick Start .

You can start watching with a Pluralsight free trial.

SHARE:

Creating Precompiled Azure Functions with Visual Studio 2017

As the Azure Functions story continues to unfold, the latest facility is the ease of creation of precompiled functions. Visual Studio 2017 Update 3 (v15.3) brings the release of functionality to create function code in C# using all the familiar tools and abilities of Visual Studio development (you can also use the Azure Functions CLI).

Precompiled functions allow familiar techniques to be used such as separating shared business logic/entities into separate class libraries and creating unit tests. They also offer some cold start performance benefits.

To create your first precompiled Azure Function, first off install Visual Studio 2017 Update 3 (and enable the "Azure development tools" workload during installation) and once installed also ensure the Azure Functions and Web Jobs Tools Visual Studio extension is installed/updated.

Azure Functions and Web Jobs Tools Visual Studio 2017 extension

After you’ve created an Azure account (free trials may be available), open Visual Studio and create a new Azure Functions project (under the Cloud section):

Creating a new Azure Functions project in Visual Studio

This will create a new project with a .gitignore, a host.json, and a local.settings.json file.

To add a new function, right click the project, choose add –> new item. Then select Azure Function:

Adding a new function to and Azure Function app

The name of the .cs file can be anything, the actual name of the function in Azure is not tied to the class file name.

Next the type of function (trigger) can be selected, such as a function triggered by an HTTP request:

Choosing a function trigger type

Adding this will create the following code (note the name of the function has been changed in the [FunctionName] attribute):

namespace MirrorMirrorOnTheWall
{
    public static class Function1
    {
        [FunctionName("WhosTheFairestOfThemAll")]
        public static async Task<HttpResponseMessage> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]HttpRequestMessage req, 
            TraceWriter log)
        {
            log.Info("C# HTTP trigger function processed a request.");

            // parse query parameter
            string name = req.GetQueryNameValuePairs()
                .FirstOrDefault(q => string.Compare(q.Key, "name", true) == 0)
                .Value;

            // Get request body
            dynamic data = await req.Content.ReadAsAsync<object>();

            // Set name to query string or body data
            name = name ?? data?.name;

            return name == null
                ? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a name on the query string or in the request body")
                : req.CreateResponse(HttpStatusCode.OK, "Hello " + name);
        }
    }
}

We can simplify this code to:

namespace MirrorMirrorOnTheWall
{
    public static class Function1
    {
        [FunctionName("WhosTheFairestOfThemAll")]
        public static HttpResponseMessage Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]HttpRequestMessage req, 
            TraceWriter log)
        {
            log.Info("C# HTTP trigger function processed a request.");

            return req.CreateResponse(HttpStatusCode.OK, "You are.");
        }
    }
}

Hitting F5 in Visual Studio will launch the local Azure Functions environment and host the newly created function:

Local Azure Functions runtime environment

Note in the preceding screenshot, the “WhosTheFairestOfThemAll” function is loaded and is listening on “http://localhost:7071/api/WhosTheFairestOfThemAll”. If we hit that URL, we get back “You are.”.

Publishing to Azure

Right click the project in Visual Studio and choose publish, this will start the publish wizard:

Publish Wizard

Choose to create a new Function App and follow the prompts, you need to select your Azure Account at the top right and choose an App Name (in this case “mirrormirroronthewall”). You also need to choose existing items or create new ones for Resource Groups, etc.

App Service settings for Azure Function app

Click create and the deployment will start.

Once deployed, the function is now listening in the cloud at “https://mirrormirroronthewall.azurewebsites.net/api/WhosTheFairestOfThemAll”.

Because earlier we specified an Access Rights setting of Function, a key needs to be provided to be able to invoke the function. This key can be found in the Azure portal for the newly created function app:

Getting an Azure Function key

Now we can add the key as a querystring parameter to get: “https://mirrormirroronthewall.azurewebsites.net/api/WhosTheFairestOfThemAll?code=UYikfB4dWIHdh66Iv/vWMiCpbDgTaDKB/vFMYtRzDwEpFW48qfEKog==”. Hitting up this URL now returns the result “You are.” as it did in the local environment.

To learn how to create precompiled Azure Functions in Visual Studio, check out my Writing and Testing Precompiled Azure Functions in Visual Studio 2017 Pluralsight course.

You can start watching with a Pluralsight free trial.

SHARE:

Architecting Azure Functions: Function Timeouts and Work Fan-Out with Queues

When moving to Azure Functions or other FaaS offerings it’s possible to fall into the trap of “desktop development’ thinking, whereby a function is implemented as if it were a piece of desktop code. This may negate the benefits of Azure Functions and may even cause function failures because of timeouts. An Azure Function can execute for 5 minutes before being shut down by the runtime when running under a Consumption Plan. This limit can be configured to be longer in the host.json (currently to a mx of 10 minutes). You could also investigate something like Azure Batch.

Non Fan-Out Example

Azure functions flow

In this initial attempt, a blob-triggered function is created that receives a blob containing a data file. Each line has some processing performed on it (simulated in the following code) and then writes multiple output blobs, one for each processed line.

using System.Threading;
using System.Diagnostics;

public static void Run(TextReader myBlob, string name, Binder outputBinder, TraceWriter log)
{
    var executionTimer = Stopwatch.StartNew();

    log.Info($"C# Blob trigger function Processed blob\n Name:{name}");

    string dataLine;
    while ((dataLine = myBlob.ReadLine()) != null)
    {
        log.Info($"Processing line: {dataLine}");
        string processedDataLine = ProcessDataLine(dataLine);
        
        string path = $"batch-data-out/{Guid.NewGuid()}";
        using (var writer = outputBinder.Bind<TextWriter>(new BlobAttribute(path)))
        {
            log.Info($"Writing output line: {dataLine}");
            writer.Write(processedDataLine);
        }
    }

    executionTimer.Stop();

    log.Info($"Procesing time: {executionTimer.Elapsed}");
     
}

private static string ProcessDataLine(string dataLine)
{
    // Simulate expensive processing
    Thread.Sleep(1000);

    return dataLine;
}

Uploading a normal sized input data file may not result in any errors, but if a larger file is attempted then you may get a function timeout:

Microsoft.Azure.WebJobs.Host: Timeout value of 00:05:00 was exceeded by function: Functions.ProcessBatchDataFile.

Fan-Out Example

Embracing Azure Functions more, the following pattern can be used, whereby there is no processing in the initial function. Instead the function just divides up each line of the file and puts it on a storage queue. Another function is triggered from these queue messages and does the actual processing. This means that as the number of messages in the queue grows, multiple instances of the queue-triggered function will be created to handle the load.

Azure functions fan-out flow

public async static Task Run(TextReader myBlob, string name, IAsyncCollector<string> outputQueue, TraceWriter log)
{
    log.Info($"C# Blob trigger function Processed blob\n Name:{name}");

    string dataLine;
    while ((dataLine = myBlob.ReadLine()) != null)
    {
        log.Info($"Processing line: {dataLine}");
               
        await outputQueue.AddAsync(dataLine);
    }
}

And the queue-triggered function that does the actual work:

using System;
using System.Threading; 

public static void Run(string dataLine, out string outputBlob, TraceWriter log)
{
    log.Info($"Processing data line: {dataLine}");

    string processedDataLine = ProcessDataLine(dataLine);

    log.Info($"Writing processed line to blob: {processedDataLine}");
    outputBlob = processedDataLine;
}


private static string ProcessDataLine(string dataLine)
{
    // Simulate expensive processing
    Thread.Sleep(1000);

    return dataLine;
}

When architecting processing this way there are other limits which may also cause problems such as (but not limited to) queue scalability limits.

To learn more about Azure Functions, check out my Pluralsight courses: Azure Function Triggers Quick Start  and  Reducing C# Code Duplication in Azure Functions.

SHARE:

New Pluralsight Course: Reducing C# Code Duplication in Azure Functions

Azure Functions allow small discrete pieces of code to execute in response to an external stimulus such as a HTTP request, message queue message, new blob data, etc.

Just because functions are easy to create (even writing and testing code right in the Azure Portal) doesn’t mean good practices such as avoiding code duplication can be abandoned.

My new Pluralsight course Reducing C# Code Duplication in Azure Functions shows some ways to reduce or remove code duplication both in a single Function App and across apps.

SHARE: