NUnit 3 Quick Tips: Asserting On Collections

When the result you want to check is a collection, you can use NUnit to assert that it has the expected number of items or is empty, that all items are unique, that specific items do/not exist, and that items exist that satisfy some condition or predicate.

Asserting on the Number of Items in a Collection with NUnit Asserts

var names = new[] { "Sarah", "Amrit", "Amanda", "Sarah" };

Assert.That(names, Has.Exactly(4).Items); // pass
Assert.That(names, Is.Empty); // fail
Assert.That(names, Is.Not.Empty); // pass

Asserting That All Items in a Collection are Unique with NUnit Asserts

Assert.That(names, Is.Unique); // fail - 2 Sarah items exist

Asserting That An Item Does or Does Not Exist in a Collection with NUnit Asserts

Assert.That(names, Contains.Item("Sarah")); // pass

// Alternative syntax
Assert.That(names, Does.Contain("Sarah")); // pass
Assert.That(names, Does.Not.Contain("Arnold")); // pass

Asserting That An Item Appears a Specified Number Of Times in a Collection with NUnit Asserts

Assert.That(names, Has.Exactly(1).EqualTo("Sarah")); // fail
Assert.That(names, Has.Exactly(2).EqualTo("Sarah")); // pass
Assert.That(names, Has.Exactly(2).EqualTo("Sarah")
                      .And.Exactly(1).EqualTo("Amrit")); // pass

Asserting That All Items In a Collections Satisfy a Predicate/Condition with NUnit Asserts

Assert.That(names, Is.All.Not.Null); // pass
Assert.That(names, Is.All.Contains("a")); // fail lowercase a
Assert.That(names, Is.All.Contains("a").IgnoreCase); // pass
Assert.That(names, Is.All.Matches<string>(name => name.ToUpperInvariant().Contains("A"))); // pass
Assert.That(names, Is.All.Matches<string>(name => name.Length > 4)); // pass

Asserting That Only One Item In a Collection Satisfies a Predicate with NUnit Asserts

Assert.That(names, Has.Exactly(1).Matches<string>(name => name.Contains("mri"))); // pass
Assert.That(names, Has.Exactly(1).Matches<string>(name => name.Contains("ara"))); // fail (2 Sarah items exist)

To learn more about NUnit 3 check out my Introduction to .NET Testing with NUnit 3 Pluralsight course to learn everything you need to know to get started, including asserts, categories, data-driven tests, customizing NUnit, and reducing duplicate test code.

SHARE:

NUnit 3 Quick Tips: Asserting On Object Reference Equality

When asserting on equality using the EqualConstraint you may not always get the behaviour you want depending on what objects are being asserted on. This can be influenced by whether or not the objects are value or reference types and if the type implements or overrides methods such as IEquatable<T> or object.Equals overrides.

Asserting on Value Type Equality with NUnit

int a = 42;
int b = 42;

Assert.That(a, Is.EqualTo(b)); // pass - values are same, ints are structs with value semantics
Assert.That(a, Is.SameAs(b)); // fail - a and b do not point to the same object in memory

int c = a;

Assert.That(c, Is.EqualTo(a)); // pass - values are same

Asserting on Reference Type Equality with NUnit

By default, 2 instances of a reference type will not pass an equality assert:

class Person
{
    public string Name { get; set; }
}
Person p1 = new Person { Name = "Sarah" };
Person p2 = new Person { Name = "Sarah" };

Assert.That(p1, Is.EqualTo(p2)); // fail, Person is class with reference semantics

Asserting That Two References Point to the Same Object with NUnit

If you want to assert that 2 object references point to the same object you can use the SameAsConstraint:

Assert.That(p1, Is.SameAs(p2)); // fail, p1 and p2 point to different objects in memory Person p3 = p1; Assert.That(p3, Is.SameAs(p1)); // pass, p3 and p1 point to same object in memory Assert.That(p3, Is.Not.SameAs(p2)); // pass, p3 and p2 point to different objects in memory

Customizing Equality Asserts with NUnit

There are a number of ways to influence how NUnit performs equality assertions including implementing IEquatable<T>:

class Employee : IEquatable<Employee>
{
    public string Name { get; set; }

    public bool Equals(Employee other)
    {
        if (other is null)
        {
            return false;
        }

        return Name == other.Name;
    }
}
Employee e1 = new Employee { Name = "Sarah" };
Employee e2 = new Employee { Name = "Sarah" };

Assert.That(e1, Is.EqualTo(e2)); // pass - IEquatable<Employee>.Equals implementation is used

To learn more about NUnit 3 check out my Introduction to .NET Testing with NUnit 3 Pluralsight course to learn everything you need to know to get started, including asserts, categories, data-driven tests, customizing NUnit, and reducing duplicate test code.

SHARE:

NUnit 3 Quick Tips: Asserting Within Ranges

If you are asserting that a value is equal to something and you want to specify some tolerance you can do so.

Specifying a Range for Values with NUnit Asserts (e.g. int)

var i = 42;

Assert.That(i, Is.EqualTo(40)); // fail

Assert.That(i, Is.EqualTo(40).Within(2)); // pass

Assert.That(i, Is.EqualTo(40).Within(1)); // fail "Expected: 40 +/- 1"

Specifying a Range as a Percentage with NUnit Asserts

In addition to specifying a range tolerance as a fixed value you can also specify it as a percentage:

Assert.That(i, Is.EqualTo(40).Within(5).Percent); // pass

Assert.That(i, Is.EqualTo(40).Within(4).Percent); // fail "Expected: 40 +/- 4 Percent"

Specifying a Range for DateTime Objects with NUnit Asserts

When working with DateTimes you can specify the tolerance as a TimeSpan instance:

var newYearsDay2019 = new DateTime(2019, 1, 1);

Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(TimeSpan.FromDays(1))); // pass

Or instead of using a TimeSpan you can use one of the convenience modifiers:

Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(1).Days); // pass

Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(24).Hours); // pass
Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(23).Hours); // fail

var numberOfMinutesInADay = 24 * 60;
Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(numberOfMinutesInADay).Minutes); // pass
Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(numberOfMinutesInADay - 1).Minutes); // fail "Expected: 2019-01-02 00:00:00 +/- 23:59:00"

// Also Within(n).Seconds .Milliseconds and .Ticks
To learn more about NUnit 3 check out my Introduction to .NET Testing with NUnit 3 Pluralsight course to learn everything you need to know to get started, including asserts, categories, data-driven tests, customizing NUnit, and reducing duplicate test code.

SHARE:

Making 2019 Your Best Year Yet

Whilst I’m not personally a fan of New Year’s Resolutions, preferring instead to adopt a mindset of continuous improvement, the end of a calendar year is as good a time as any to do a 12 month retrospective and think about the future.

Below are a number of resources you may find helpful to make 2019 your best year yet, along with some of the things I’m currently learning/implementing in my quest for personal development.

2018 Book List

These are some of the books I read in 2018 that I found helpful and that have influenced my thinking and personal growth:

  • Discipline Equals Freedom: Field Manual (Jocko Willink). Learn to understand that discipline in the present gives you more freedom in the future.
  • The Inner Game of Tennis (W. Timothy Gallwey). More about physical development/performance but the ideas transcend into anything you want to learn.
  • Man's Search for Meaning (Viktor E. Frankl). Harrowing account of Nazi concentration camps infused with what it means to have real meaning in your life.
  • Principles: Life and Work (Ray Dalio). Ray Dalio is the founder of one of the biggest and highest performing hedge funds in the world, this book shows the life and work principles the author has developed and also introduces the fascinating idea that you can run your life based on a set of your own written down principles.
  • Extreme Ownership: How U.S. Navy SEALs Lead and Win (Jocko Willink  & Leif Babin). This should be required reading for anyone managing/leading other people, and for anyone wanting to feel like they are more in control of their life by not making excuses.

Personal Growth Podcasts

My 4 go-to podcasts (in no particular order) for physical, mental, and career growth:

Don’t Code Tired Articles

The following are articles on this blog:

What I’m Working On Implementing At the Moment

These are things I’m trialling, working on implementing, working on improving, or plan to implement in the near future:

  • Regular daily meditation (I’m currently trying the Mindvalley 6 Phase Meditation Quest)
  • Early to bed, early to rise (somewhere between 4:30 AM and 6AM – not sure yet) Effectively swapping some mindless TV/YouTube/XBox gaming at night for productive time in the morning.
  • Learning to deadlift and working on flexibility to allow my to do this
  • Generally: reducing waste, simplification, balanced minimalism in regards to ownership of things, saving/investing more money, reading more, learning more.

Living Your Best Life

I hope the preceding things prove to be useful, inspirational, or interesting to you.

I wish you all the best for the coming year and hope that your 2019 brings you closer to being able to live your best life.

Best wishes, Jason.

SHARE:

Developing Tizen Samsung Galaxy Watch Apps with .NET and C# - Getting Started

This article assumes you have set up the Tizen/Visual Studio development environment as outlined in this previous article.

Installing the Watch Emulator

The first step is to install the relevant emulator so you don’t need a physical Samsung Galaxy Watch. To do this open Visual Studio and click  Tools –> Tizen –> Tizen Emulator Manager

This will bring up the Emulator Manager, click the Create button, then Download new image, check the WEARABLE profile, and click OK. This will open the Package Manager and download the emulator.

Installing the Tizen Wearable emulator in Visual Studio

Once the installation is complete, if you open the Emulator Manager, select Wearable-circle and click Launch you should see the watch emulator load as shown in the following screenshot:

watchemulator

Creating a Watch Project

In Visual Studio, create a new Tizen Wearable Xaml App project  which comes under the Tizen 5.0 section.

Once the project is created and the with the emulator running, click the play button in Visual Studio (this will be something like “W-5.0-circle-x86…” ).

The app will build and be deployed to the emulator – you may have to manually switch back to the emulator if it isn’t brought to the foreground automatically. You should now see the emulator with the text “Welcome to Xamarin.Forms!”.

This text comes from the MainPage.xaml:

<?xml version="1.0" encoding="utf-8" ?>
<c:CirclePage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:c="clr-namespace:Tizen.Wearable.CircularUI.Forms;assembly=Tizen.Wearable.CircularUI.Forms"
             x:Class="TizenWearableXamlApp1.MainPage">
  <c:CirclePage.Content>
    <StackLayout>
      <Label Text="Welcome to Xamarin.Forms!"
          VerticalOptions="CenterAndExpand"
          HorizontalOptions="CenterAndExpand" />
    </StackLayout>
  </c:CirclePage.Content>
</c:CirclePage>

Modifying the Basic Template

As a very simple (and quick and dirty, no databinding, MVVM, etc.) example, the MainPage.xaml can be changed to:

<?xml version="1.0" encoding="utf-8" ?>
<c:CirclePage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:c="clr-namespace:Tizen.Wearable.CircularUI.Forms;assembly=Tizen.Wearable.CircularUI.Forms"
             x:Class="TizenWearableXamlApp1.MainPage">
    <c:CirclePage.Content>
        <StackLayout HorizontalOptions="CenterAndExpand" VerticalOptions="CenterAndExpand">
            <Label x:Name="HappyValue" Text="5" HorizontalTextAlignment="Center"></Label>
            <Slider x:Name="HappySlider" Maximum="10" Minimum="1" Value="5" ValueChanged="HappySlider_ValueChanged" ></Slider>
            <Button Text="Go" Clicked="Button_Clicked"></Button>
    </StackLayout>
  </c:CirclePage.Content>
</c:CirclePage>

And the code behind MainPage.xaml.cs:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

using Xamarin.Forms;
using Xamarin.Forms.Xaml;
using Tizen.Wearable.CircularUI.Forms;
using System.Net.Http;

namespace TizenWearableXamlApp1
{
    [XamlCompilation(XamlCompilationOptions.Compile)]
    public partial class MainPage : CirclePage
    {
        private int _happyValue = 5;

        public MainPage()
        {
            InitializeComponent();
        }

        private async void Button_Clicked(object sender, EventArgs e)
        {
            HttpClient client = new HttpClient();

            var content = new StringContent($"{{ \"HappyLevel\" : {_happyValue} }}", Encoding.UTF8, "application/json");

            var url = "https://prod-29.australiasoutheast.logic.azure.com:443/workflows/[REST OF URL REDACTED FOR PRIVACY/SECURITY]";

            var result = await client.PostAsync(url, content);
            
        }

        private void HappySlider_ValueChanged(object sender, ValueChangedEventArgs e)
        {
            _happyValue = (int)Math.Round(HappySlider.Value);

            HappyValue.Text = _happyValue.ToString();
        }
    }
}

The preceding code essentially allows the user to specify how happy they are using a slider, and then hit the Go button. This button makes an HTTP POST to a URL, in this example the URL is a Microsoft Flow HTTP request trigger.

The flow is shown in the following screenshot, it essentially takes the JSON data in the HTTP POST, uses the HappyLevel JSON value and sends a mobile notification to the Flow app on my iPhone.

Microsoft Flow triggered from HTTP request

Testing the App

To test the app, run it in Visual Studio:

Xamarin Forms app running in Samsung Galaxy Watch emulator

Tapping the Go button will make the HTTP request and initiate the Microsoft Flow, and after a few moments, the notification being sent to the phone:

Microsoft Flow notification on iPhone

SHARE:

Starting Where You're At

Someone says to you: "you must be doing agile, continuous integration, continuous deployment, automated testing, test driven development, etc. etc. etc. or you're doing it wrong".

For any sufficiently complex application you’re building it's likely that the above things, and more, will be beneficial.

The problem is if you’re not doing any of these things and you feel overwhelmed where do you start?

You start where you're at.

build on each success with subsequent success

I remember watching Ray Mears on TV once and he used the acronym STOP to remember what to do if you get lost or stranded:

  • (S)top
  • (T)hink
  • (O)rient
  • (P)lan

If you feel overwhelmed this may be a useful acronym to help you start where you're at.

Accepting where you are now and starting where you're at can help remove negative feelings and the feeling of being overwhelmed and not being "good enough" at your job.

This of course doesn't mean that you should accept unprofessional practices and not try to improve things, it simply means acknowledging without judgement where you are now and then moving forward to improve things for the future.

You can start with the "big rocks", the more important or foundational things such as making sure you're using adequate source control. Maybe then move to implementing a basic continuous integration build. Maybe then start to add some automated tests, etc. etc.

Do things incrementally and build on each success with subsequent success.

To create the change you desire, you may have to invest in you along the way, develop an understanding that discipline equal freedom, and also ask yourself the question “what would easy be like?”.

SHARE:

Discipline Equals Freedom

One of the books I read this year was Discipline Equals Freedom: Field Manual by Jocko Willink.

The overarching concept in the book is that if you have discipline now, in the present, this will result in greater freedom in the future.

This could be having the discipline to work out/lift weights; in the future this will most likely result in the freedom to move more with less pain/lift heavy things/go trekking/kayaking on holiday, etc.

This could be having the financial discipline to regularly save/invest money rather that spend everything you earn; in the future this will most likely result in having the freedom to (semi)retire early, not have to work in a job you don't like, etc.

I think the reason this book resonated with me personally is that it helped focus the outlook I already had on investing in the future/thinking longer term.

This concept can also be applied to software development. Disciplined software development now, will most probably lead to greater freedom in the future to make changes or add new features.

As an example, having the discipline to create automated tests can help give you the freedom in the future to make changes without having to perform a load of manual testing or worry about what may have broken unknowingly.

Another example: having the discipline to refactor code as you are working on the current feature/bug to keep the code as clean as possible will most probably give you the freedom to change it more easily in the future.

If you are a manager: having the discipline to allocate time for you team to train/learn/get better even when faced with pressure to deliver will most probably result in the freedom to deliver more in the future.

The concept can also be applied to non-coding practices such as having the discipline to hold a stand up meeting every day, engage business/users/stakeholders regularly, etc.

You can also flip this concept on its head and instead ask: in the future, what aspects would I like more freedom in: in one month, in one year, in 10 years? Let the answers to this question guide you when deciding on what it is you need to be more disciplined on today or this week to make manifest those desired freedoms in the future.

SHARE:

Developing Samsung TV Apps with .NET - Getting Started

In 2018, Samsung started to release Smart TVs that support apps written in .NET. These TVs run on the Tizen operating system which is “an open and flexible operating system built from the ground up to address the needs of all stakeholders of the mobile and connected device ecosystem, including device manufacturers, mobile operators, application developers and independent software vendors (ISVs). Tizen is developed by a community of developers, under open source governance, and is open to all members who wish to participate.” [Tizen.org]

This means that apps can developed in Visual Studio using .NET, tested locally on a TV emulator, tested on a physical TV, and published/distributed on the Samsung Apps TV app store.

In this article you’ll learn how to set up your development environment, create your first app, and see it running on the TV emulator.

Part I - Setting Up Visual Studio for Samsung TV App Development

There are a number of steps to setup Visual Studio.

1.1 Install Java JDK

The first thing to do is install Java, the Tizen tools under the hood require Java JDK 8 to be installed and system environment variables setup correctly.

To do this the hard way, head over to Oracle.com JDK 8 archive page and download the Windows x64 installation. Note the warning before deciding whether or not to go ahead: “WARNING: These older versions of the JRE and JDK are provided to help developers debug issues in older systems. They are not updated with the latest security patches and are not recommended for use in production.” [Oracle.com] Also note “Downloading these releases requires an oracle.com account.” [Oracle.com]

To do it the easy way, open the Visual Studio Installer, check the Mobile Development with JavaScript payload and in the optional section tick the Java SE Development Kit option as shown in the following screenshot. (You may also want to install the Mobile Development with .NET payload as well as you can use Xamarin Forms to develop Samsung TV apps)

Installing Java 8 from the Visual Studio Installer

Once Java is installed you’ll need to modify system environment variables as follows:

  1. Add a system variable called JAVA_HOME with a value pointing to the path of the Java install, e.g: C:\Program Files\Java\jdk1.8.0_192
  2. Add a system variable called JRE_HOME with a value that points to Java JRE directory, e.g: C:\Program Files\Java\jdk1.8.0_192\jre
  3. Modify the Path variable value and add to the end: %JAVA_HOME%\bin;%JRE_HOME%\bin;”"

1.2 Install Tizen Visual Studio Tools

The next job is to install the Visual Studio Tizen related tools.

First, open Visual Studio and head to  Tools –> Extension and Updates. In the online section, search for “Tizen” and download the Visual Studio Tools for Tizen extension. Once downloaded, you’ll need to close Visual Studio and follow the prompts to complete the installation (it may take a little while to download the Tizen tools). Once complete re-open Visual Studio.

Next, in the Visual Studio menus, head to Tools –> Tizen –> Tizen Package Manager. This will open the Tizen SDK installer. Click the Install new Tizen SDK option as the following screenshot shows:

Tizen SDK Installer in Visual Studio

Choose an install location – you will need to create a new folder yourself – for example C:\TizenSDK

Follow the prompts and you should see the SDK installation proceeding:

Tizen SDK intallation in progress

You will also be asked to install the Tizen Studio Package Manager, once again follow the prompts. Be patient, the Package Manager install may take some time “Loading package information”.

Once complete, all the dialog boxes should close and you can head back to Visual Studio.

1.3 Install the Samsung Studio TV Extensions

In Visual Studio, head to Tools –> Tizen –> Tizen Package Manager.

Head to the Extension SDK tab and install the TV Extensions-4.0 package:

Install Tizen TV Extensions in Visual Studioe

Once again be patient (the progress bar is near the top of the dialog box).

Once installed, close Package Manager.

1.4 Verify Samsung TV Emulator Installation

Before trying to use the TV emulator check out the requirements (including turning off Hyper V  https://developer.tizen.org/development/visual-studio-tools-tizen/installing-visual-studio-tools-tizen

Back in Visual Studio, head to Tools –> Tizen –> Tizen Emulator Manager.

You should see HD 1080 TV in the list of emulators:

Tizen TV Emulator installed

1.5 Install Samsung TV  .NET App Templates

Head back to Visual Studio’s Tools –> Extensions and Updates menu, once again search online for Tizen, and this time install the Samsung TV .NET App Templates extension. This will give you access to the project templates. You may need to restart Visual Studio for the installation to complete.

Part II – Creating Your First Samsung TV .NET App

2.1 Create a new Samsung TV Project

Re-open Visual Studio and click File –> New Project.

In the Tizen Samsung TV  section, choose Blank App (Xamarin.Forms) template:

Blank App (Xamarin.Forms) Visual Studio Template

Click OK. This will create a very basic bare-bones app project that uses Xamarin Forms.

It is a good idea to head to NuGet package manager and update all the packages such as the Xamarin Forms and Tizen SDK packages.

Head into the “STVXamarinApplication1” project that contains the “STVXamarinApplication1.cs” file which in turn contains the App class. Inside the app class you can see the following code:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

using Xamarin.Forms;

namespace STVXamarinApplication1
{
    public class App : Application
    {
        public App()
        {
            // The root page of your application
            MainPage = new ContentPage
            {
                Content = new StackLayout
                {
                    VerticalOptions = LayoutOptions.Center,
                    Children = {
                        new Label {
                            HorizontalTextAlignment = TextAlignment.Center,
                            Text = "Welcome to Xamarin Forms!"
                        }
                    }
                }
            };
        }

        protected override void OnStart()
        {
            // Handle when your app starts
        }

        protected override void OnSleep()
        {
            // Handle when your app sleeps
        }

        protected override void OnResume()
        {
            // Handle when your app resumes
        }
    }
}

Modify the line Text = "Welcome to Xamarin Forms!" to be: Text = DateTime.Now.ToString()

Build the solution and check there are no errors.

2.2 Running a .NET App in the Tizen Samsung TV Emulator

In the Visual Studio tool bar, click Launch Tizen Emulator.

Launching the Tizen Emulator from Visual Studio

This will open the Emulator Manager, click the Launch button and the TV emulator will load with a simulated remote  as shown below:

Samsung TV Emulator

Head back to Visual Studio and click the run button (which should now show something like T-samsung-4.0.x86…):

Deploying an Samsung TV app to the emulator in Visual Studio

Once the button is clicked, wait a few moments for the app to be deployed to the emulator. You may have to manually switch back to the emulator if it’s not automatically brought to the front.

You should now see the app running on the emulator and showing the time:

.NET app running on the Samsung TV emulator on Windows 10

If you want to read more about the Tizen .NET TV Framework, check out the documentation.

SHARE:

Azure Functions Continuous Deployment with Azure Pipelines: Part 8 - Using Gates to Run Smoke Tests on Deployed Function Apps

This is the eighth and final part in a series demonstrating how to setup continuous deployment of an Azure Functions App using Azure DevOps build and release pipelines.

Demo app source code is available on GitHub.

In the previous instalment we added functional end-to-end testing to the release pipeline.

As one final check for the release, we can execute a smoke test on the deployed production Function App. This smoke test won’t modify any data and will simply allow us to check that the Function App is responding to HTTP requests.The smoke test function won’t be disabled in production but it will be protected by a function key.

namespace InvestFunctionApp.TestFunctions
{
    public static class SmokeTest
    {
        [FunctionName("SmokeTest")]
        public static async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "post", Route = Testing.TestFunctionRoute + "/smoketest")] HttpRequest req,
            ILogger log)
        {            
            log.LogInformation("C# HTTP trigger function processed a request.");

            string requestBody = await new StreamReader(req.Body).ReadToEndAsync();

            dynamic data = JsonConvert.DeserializeObject(requestBody);

            string azureDevopsReleaseUrl = data?.releaseurl ?? "[releaseurl value not supplied in json request]";

            log.LogInformation($"Smoke test for release {azureDevopsReleaseUrl}");

            // We could add extra smoke testing code here, this simple version just allows us to 
            // verify that the deployment succeeded and the Function App is responding to HTTP requests

            return new OkObjectResult("Smoke test successful");
        }
    }
}

The preceding smoke test function returns a 200 OK result and also allows the URL to the AzureDevOps release to be provided, which is written to the logs. This allows ops to see from which release the smoke test function came from.

The first step is to add a variable to the release pipeline called “prodSmokeTestKey” with the value being the function key from production for the smoketest function. Once this key is copied from the Azure Portal it can be copied into the variable value and the padlock icon clicked to mark this as sensitive data:

Adding a sensitive Azure Pipeline variable

“Gates allow automatic collection of health signals from external services, and then promote the release when all the signals are successful at the same time or stop the deployment on timeout.” [Microsoft]

Gates can be evaluated before a stage executes and/or after a stage executes.

“Approvals and gates give you additional control over the start and completion of the deployment pipeline. Each stage in a release pipeline can be configured with pre-deployment and post-deployment conditions that can include waiting for users to manually approve or reject deployments, and checking with other automated systems until specific conditions are verified. In addition, you can configure a manual intervention to pause the deployment pipeline and prompt users to carry out manual tasks, then resume or reject the deployment.” [Microsoft]

We can add a post-deployment gate that will mark the release as unsuccessful if the smoke test function does not respond successfully. To do this we can use the “Invoke Azure Function” task. Other tasks that can be executed as part of a gate include “Invoke REST API”, “Query Azure Monitor Alerts”, and “Query Work Items” tasks.

To add a post-deployment gate to the “Deploy to Production” stage, click the post-deployment conditions icon, enable the Gates switch, and click the + Add button.

Enabling post-deployment gates in Azure Pipelines

Choose “Invoke Azure Function”. Paste in the URL to the production smoke test function and for the function key specify “$(prodSmokeTestKey)” to retrieve the key from the pipeline variable that we set up earlier. Select the method as POST and in the body enter “{ "releaseurl" : "$(Release.ReleaseWebURL)" }”. This is the JSON payload that will be sent to the smoke test function.

Invoke Azure Function Deployment Gate

Hit save and queue a new release.

Once the production deployment is complete, there will be a 5 minute delay before the gate(s) are evaluated for the first time and a 15 minute wait between re-evaluation of gates which ultimately means a lengthy delay between the deployment and smoke test execution. To improve this you can set the the “The delay before evaluation” to 1 minute, and in the Evaluation options section at the bottom set “The time between re-evaluation of gates” to5 mins and “The timeout after which gates fail” to 6 mins.

As an alternative to using gates in this way (especially if you need to add multiple gates or complex smoke testing requirements) could be to have another post-production-deployment task/job/stage that calls the smoke test function(s)  in another source-controlled test project in a similar way to how the end-to-end tests were executed against the test Function App.

That bring us to the end of this 8 part series.To be notified of future posts subscribe to the blog feed or follow me on Twitter.

SHARE:

Azure Functions Continuous Deployment with Azure Pipelines: Part 7 - Running Functional End-to-end Tests in a Release Azure Pipeline

This is the seventh part in a series demonstrating how to setup continuous deployment of an Azure Functions App using Azure DevOps build and release pipelines.

Demo app source code is available on GitHub.

In the previous instalment we created the release pipeline and now have continuous deployment working. Currently if the unit tests pass (and the rest of the build is ok) in the build pipeline, the release pipeline will automatically execute and deploy the  Function App to Azure.

In this instalment of this series we’ll add some additional stages in the release pipeline to deploy first to a test Function App in Azure, then run some functional test against this test deployment, and only if those tests pass, deploy to the production Function App.

Deploying a Function App to Test

The first step is to edit the release pipeline that we created earlier in this series and add a new stage. A quick way to do this is to click the Clone button on the existing “Deploy to Production” stage:

Cloning a stage in a release Azure pipeline

Change the name of the cloned stage to “Deploy to Production” and the original stage to “Deploy to Test”, it should now look like the following screenshot:

Release Azure Pipeline

Next edit the tasks in the “Deploy to Test” phase, click the Disable Testing Functions task and click Remove to delete the task from the test stage.

We need to change the deployment target, so click the Azure App Service Deploy task and change the App Service name to “InvestFunctionAppDemoTest” – we want to deploy to the test Azure Function App not the production one.

Creating Functional End-to-End Tests

If you check out the demo source code on GitHub you can see the end-to-end test project.

The AddToPortfolioFunctionShould test class contains a test called BuyStocks. This test performs the following logical steps:

  1. Create a new test Investor in Azure Table storage (by calling the test Azure function CreateInvestor)
  2. Call the Portfolio function to add funds to a portfolio
  3. Wait for a while
  4. Check that the test Investor created in  step 1 has had its stock value updated – this is done by called the test function GetInvestor

Side Note: In this example we’ve create 2 additional Azure Functions to help facilitate testing, one to create a test investor and one to retrieve Investor details so we can assert against the final result. We could have just manipulated Azure Table storage directly within the test methods but I wanted to show this approach to demonstrate a number of features such as automating function disabling as part of deployments and passing pipeline variables to test code. Normally we don’t want to deploy testing-related items to production for a whole host of reasons (performance, security, data integrity, etc.), but this approach if properly managed and secured can be quite a useful quick win. If you do implement these kind of test functions you must ensure that they cannot be called if deployed to production by restricting them at multiple layers: first by securing the functions with a secret function key and second by ensuring as part of the deployment the testing functions are disabling in the app settings. You could even add a 3rd level of checking by asserting that the function is executing in a testing environment with like an AssertInTestEnvironment being called at the start of each test function. All that being said, deploying test functions to production adds all this additional complexity and risk and  so is best avoided.

There’s a few things that this end-to-end test needs.

Firstly it needs to know the Azure Function keys for the two test functions and also the Portfolio function. We don’t want to commit these keys to source control, so we can instead create pipeline variables for them and then access them via environment variables in the C# test code by using Environment.GetEnvironmentVariable(variableName).

Secondly there is an extra level of checking around the test functions being able to be called in production. The functions will be disabled in production, in addition to being protected by a function key. Whilst these two things should make it impossible for them to be called, there is an extra check implemented in the following class:

internal class Testing
{
    internal const string TestFunctionRoute = "testing";
    internal const string TestEnvironmentConfigKey = "InvestFunctionApp.IsTestEnvironment";

    /// <summary>
    /// Testing functions should be disabled in Azure, this is an additional level of checking.
    /// </summary>
    internal static void AssertInTestEnvironment(ILogger log)
    {
        var value = Environment.GetEnvironmentVariable(TestEnvironmentConfigKey);

        var isTestingEnvonment = value != null && value == "true";

        if (!isTestingEnvonment)
        {
            log.LogError("This function should be disabled in non-testing environments but was called. Check that all testing functions are disabled in production.");
            throw new InvalidOperationException();
        }
    }
}

The AssertInTestEnvironment method is called in the test functions that should be disabled in production, for example:

namespace InvestFunctionApp.TestFunctions
{
    public static class CreateInvestor
    {
        [FunctionName("CreateInvestor")]
        [return: Table("Portfolio")]
        public static async Task<Investor> Run(
            [HttpTrigger(AuthorizationLevel.Function, "post", Route = Testing.TestFunctionRoute + "/createinvestor")] HttpRequest req,            
            ILogger log)
        {
            Testing.AssertInTestEnvironment(log);

            log.LogInformation("C# HTTP trigger function processed a request.");

            string requestBody = await new StreamReader(req.Body).ReadToEndAsync();

            return JsonConvert.DeserializeObject<Investor>(requestBody);
        }
    }
}

For the test functions to be enabled in the test Azure Function App, there needs to be an application setting called “InvestFunctionApp.IsTestEnvironment” set to “true”.

Adding Function Keys as Pipeline Variables

As we did earlier in the build pipeline, we can add variables to the release pipeline, the variables are called CreateInvestorFunctionKey, GetInvestorInvestorFunctionKey, and PortfolioFunctionKey. The values of these should be the function keys of those functions deployed to the test environment. You may need to create a release and run the release pipeline first to deploy the app to test before you can get the test function keys (a bit of the chicken and the egg here).

Release Azure Pipeline variables

Notice in the preceding screenshot that we’re not marking these test keys as secret, though we should ideally go and do that but it introduces a little extra complexity if we want to access them as environment variables in the C# test code.

Adding a Functional End-to-End Test Stage

Now we have the app deployed to test, we want to run the functional tests against it.

To do this we create a new empty stage called “Run Functional Tests” and modify the “Deploy to Production” stage trigger to be run after the new testing stage completes as the following screenshot shows:

Stages in a release Azure Pipeline

Notice here that we’re creating a completely new stage to run the functional tests, this is for demonstration purposes to show the flexibility of being able to design your release pipeline however you want though this approach doesn’t align fully with the conceptual idea of a stage being a: “logical and independent entity that represents where you want to deploy a release generated from a release pipeline.” [Microsoft] . It does however conform to the idea that a stage is a “logical entity that can be used to perform any automation”[Microsoft]. In any case, it is more likely that in a real scenario we wouldn’t create a new stage just to run the functional tests. What we could have instead are a couple of stages, one called “QA” (a testing environment deployment) and one called “Production”. We could then run the functional tests as a separate task in the “QA” stage. You should make sure you read the documentation  to fully understand what stages are and the nuances such as “Having one or more release pipelines, each with multiple stages intended to run test automation for a build, is a common practice. This is fine if each of the stages deploys the build independently, and then runs tests. However, if you set up the first stage to deploy the build, and subsequent stages to test the same shared deployment, you risk overriding the shared stage with a newer build while testing of the previous builds is still in progress” [Microsoft].Another option would be to create a new Function App in Azure (with a unique name) for each execution of the stage, run the functional tests against it, and then delete the Function App.The great thing about Azure Pipelines is the flexibility they offer, however this flexibility comes at the cost of potentially shooting yourself in the foot.In the future I intend to write more about good practices and concepts when designing pipelines.

Continuing with the demo scenario, we now need a task in the new testing stage to execute dotnet test on the functional end-to-end test project.

To do this we can add a .Net Core task, set the command to test and the path to project as “$(System.DefaultWorkingDirectory)/_InvestFunctionApp/e2etests/InvestFunctionApp.EndToEndTests/InvestFunctionApp.EndToEndTests.csproj” (notice in this path we’re accessing the e2etests artifact created in the YAML build).

Setting Test Azure Function Application Settings

When deploying to the test Function App in Azure we need to set the application setting “InvestFunctionApp.IsTestEnvironment” to “true”. Rather than using Azure CLI we can do this as part of the Azure App Service Deploy task in the Application and Configurations Settings as the following screenshot shows:

Setting Azure Function App settings when deploying from Azure Pipelines

Testing the Updated Release Pipeline

Once you’ve made all these changes and saved them you can queue up another manual release to see if everything works. Just click the + Release button and choose “Create a release”. Specify the latest build in the artifacts section and click Create.

This will queue and start a new release:

Azure release Pipeline executing

After a while the release should complete and all stages should complete successfully:

Azure release Pipeline executing

Clicking on the “Run Functional Tests” stage and then the Tests tab you can see the “AddToPortfolioFunctionShould.BuyStocks” test executed and passed:

Passing tests in an Azure Pipeline

 

In the final part of this series, we’ll see how to execute a smoke test against the deployed production Function App to verify at a high level that all is well with the deployment.

SHARE: