Random data generation in F# with AutoFixture

EDIT 06/30/2020: I have now consolidated these features and published two Nuget packages:

• AutoFixture.Community.FSharp
• AutoFixture.Community.FSharp.Xunit.

I had used AutoFixture with xUnit in the past, and it worked great, especially the AutoDataAttribute feature: to write a test with random input data in C#, you just had to do something like this:

// Assuming you have a dto defined like this:
public class MyDto
{
    public int Foo { get; set; }
    public string Bar { get; set; }
    public DateTime Baz { get; set; }
}

public class Updater
{
    public static void DoubleFoo(MyDto dto) => dto.Foo *= 2;
}

public class MyTestClass
{
    // Instead of this:
    [Fact]
    public void VerboseFactMethod()
    {
        // Manually construct instances of data even though you don't really care
        // _what_ the values are, only what the program does with them
        var dto = new MyDto
        {
            Foo = 1,
            Bar = Guid.NewGuid().ToString(),
            Baz = DateTime.Now
        };

        // perform assertions
        Assert.Equal(2, Updater.DoubleFoo(dto))
    }

    // You just declare you need a value of a certain type, and it's just provided for you
    [Theory, AutoData]
    public void MyTestMethod(MyDto dto)
    {
        // perform assertions
        Assert.Equal(dto.Foo * 2, Updater.DoubleFoo(dto))
    }
}

Now dto would be generated with random data. Boom, write tests like magic and you only need to worry about the important stuff.

The goal

So when I started working on F# projects, I quickly grew tired of the verbose nature of creating the many record types needed by the code during unit tests. In C#, it's common to have DTO types as classes, where often there will be a constructor to set default values. But in F#, when working with plain data, the better practice is often to utilize records for brevity in type declarations. However, this can lead to the issue of declaring instances of these types to be cumbersome and verbose, which can be a problem in the realm of unit testing where we ideally want to make it clear what exactly we are trying to test.

The equivalent in F# would be something like this

type MyDto =
    { Foo : int
      Bar : string
      Baz : DateTime }

let doubleFoo dto = { dto with Foo = dto.Foo * 2 }

[<Fact>]
let ``Verbose fact method``() =
    let dto =
        { Foo = 1
          Bar = Guid.NewGuid().ToString()
          Baz = DateTime.Now }
    let actual = doubleFoo dto
    Assert.Equal(2, actual.Foo)

But I wanted to be able to do it like this:

[<Theory>]
[<AutoData>]
let ``Succinct theory with AutoData`` (dto: MyDto) =
    let actual = doubleFoo dto
    Assert.Equal(dto.Foo * 2, actual.Foo)

That worked like a dream! Through testing around, I determined that the following cases worked as expected:

1. Passing a single value in a theory AutoData parameter
2. Passing a tuple value in as theory AutoData parameters
3. Passing curried arguments as theory AutoData parameters

So far, seems like it works! But I encountered a few issues arise when dealing with the F# type system.

The Problem

The specific issues I ran into were the following:

1. AutoFixture was unable to natively generate an F# list with any members, i.e. it only ever created an empty list
2. When generating a discriminated union, it only seemed to be able to create the first case declared

So for example, if I wanted to pass an int list as a parameter or I had it as the member type of a record, I would only ever get the empty list [].

And if I wanted to use an int option, I would only ever get Some values-- it would never generate None.

This is important because although I didn't care what values are rendered, there may be cases that arise that depend on these differences, and if I never encounter a None, for example, that could hide a bug that I would otherwise find.

The Fix

The first thing I needed to do was figure out how to instantiate items of the types I needed. Making lists was pretty straightforward, but generating discriminated unions proved a little more tricky.

F# lists

I was able to find an existing implementation for F# lists from here:

// Needs to be on a type for reflection purposes
type internal ReflectiveList =
    static member Build<'a>(args : obj list) =
        [ for a in args do
                yield a :?> 'a ]

    static member BuildTyped t args =
        typeof<ReflectiveList>
            .GetMethod("Build", BindingFlags.Static ||| BindingFlags.NonPublic)
            .MakeGenericMethod([| t |])
            .Invoke(null, [| args |])

let (|List|_|) candidate =
    match candidate |> box with
    | :? Type as t when t.IsGenericType &&
                        t.GetGenericTypeDefinition() = typedefof<list<_>>
        ->
            let t = t.GetGenericArguments().[0]
            let create repeatCount (ctx: ISpecimenContext) =
                [ for _ in 1..repeatCount -> ctx.Resolve t ]
                |> ReflectiveList.BuildTyped t

            List(create)
            |> Some
    | _ -> None

Discriminated Unions

The bulk of getting this functionality to work came from the need to randomly generate instances of a discriminated union. To properly create random instances of a discriminated union, you must account for each different potential union case. Do do that, I utilized functions in FSharp.Reflection, which is built in to the core library, and an active pattern, for easy readability in the customization class.

open FSharp.Reflection
open AutoFixture.Kernel

let rand = Random()

// This helper function will randomly select
// one of the possible Discriminated Union cases to generate
let getRandomElement cases =
    let length = Array.length cases
    let index = rand.Next(0, length)
    Array.get cases index

/// Randomly choose a union case
let (|Union|_|) candidate =
    match candidate |> box with
    | :? Type as t when FSharpType.IsUnion(t) ->

        let cases = FSharpType.GetUnionCases(t)
        let case = getRandomElement cases

        let args =
            case.GetFields()
            |> Array.map (fun f -> f.PropertyType)

        // Here I opted to return a function that receives the ISpecimenContext
        // And uses FSharp.Reflection to create the union case
        let create (ctx: ISpecimenContext) =
            let args = args |> Array.map ctx.Resolve
            FSharpValue.MakeUnion(case, args)

        Some <| Union (create)
    | _ -> None

Putting it all together

Once I had the functions needed to generate the different types

Luckily, AutoFixture was designed with extensibiility in mind, so adding the ability to handle these type differences was actually deceptively simple.

All you really need to do is add an ICustomization that accounts for these cases. Then, you just need to account for the different possible types that we want to handle.

open AutoFixture
open AutoFixture.Kernel

type FSharpSpecimenBuilder(f: IFixture) =
    interface ISpecimenBuilder with
     member this.Create(request, ctx) =
        // Here we utilize active patterns defined below to
        // account for and handlethe lists and unions
        match request with
        | List create -> create f.RepeatCount ctx
        | Union create -> create ctx
        | _ -> NoSpecimen() |> box

type FSharpCustomizations() =
    interface ICustomization with
        member this.Customize fixture =
            FSharpSpecimenBuilder(fixture)
            |> fixture.Customizations.Add

/// A Fixture that adds support for common F# types
type FSharpFixture () as this =
    inherit Fixture()
    let customization = FSharpCustomizations() :> ICustomization
    do customization.Customize(this)