xUnit-Jet – Open Sourced

At Jet.com, we’re big fans of xUnit for unit testing (along with some other products). xUnit is primarily designed for C#, and while it can detect inequality between native F# types, it doesn’t have a true understanding of their structure. As a result, we had unit tests that could detect an issue, but not give us detailed information on what exactly the issue was. This meant that to solve the problem, we had to do extra digging (or sometimes write extra code to provide a better answer). In a culture where you want to push code fast with adequate risk reduction, our use of xUnit wasn’t meeting our needs.

This issue was compounded by the fact that F# offers a rich type structure (mostly made possible by discriminated unions and tuples), which allows types of different shapes to be co-mingled. In effect, a single type can be deeply expressive and flexible without the use of polymorphism. Our type definitions are intended to express the explicit possibilities for values with no room for error or excess baggage. Some values end up being quite large because of this, and we therefore need granular feedback on where the issues are.

xUnit-Jet is our solution to this problem! We’ve been using this internally for the better part of a year, and are excited to make it public. It has lots of room to grow, but even what’s been built so far has been invaluable to us.


Examples

Here are some sample unit tests to express the differences in running native xUnit vs. our extension. xUnit is able to find inequality between the values, but its response isn’t very informative. This leads to follow-up work in digging deeper to find the differences. These types are fairly simple, so you can imagine how arduous this would be with larger / more complex types.

The following examples are not exhaustive, but should give a sense of what the extension does.

type SampleTypeInner =
    { primitiveOption : int option
      stringList : string list
      intTuple : int * int }

type SampleTypeOuter =
    | SomeLabel
    | SomeData of SampleTypeInner list

let expected =
    [ { SampleTypeInner.primitiveOption = None
        SampleTypeInner.stringList = [ "stringOne" ; "stringTwo" ]
        SampleTypeInner.intTuple = (101, 102) }
      { SampleTypeInner.primitiveOption = Some 3
        SampleTypeInner.stringList = []
        SampleTypeInner.intTuple = (201, 202) } ]
    |> SampleTypeOuter.SomeData

Record Values

let [<Fact>] ``inequality of record value`` () =
    let actual =
        [ { SampleTypeInner.primitiveOption = None
            SampleTypeInner.stringList = [ "stringOne" ; "stringTwo" ]
            SampleTypeInner.intTuple = (101, 102) }
          { SampleTypeInner.primitiveOption = None
            SampleTypeInner.stringList = []
            SampleTypeInner.intTuple = (201, 202) } ]
        |> SampleTypeOuter.SomeData

xUnit’s Assert.Equal result:xunitrecordvalue

xUnit-Jet’s Assert.equalDeep result:recordvalue


Discriminated Union Types

let [<Fact>] ``inequality of discriminated union label`` () =
    let actual = SampleTypeOuter.SomeLabel

xUnit’s Assert.Equal result:dulabel

xUnit-Jet’s Assert.equalDeep result:dulabel


Collection Lengths

let [<Fact>] ``inequality of list size`` () =
    let actual =
        [ { SampleTypeInner.primitiveOption = None
            SampleTypeInner.stringList = [ "stringOne" ; "stringTwo" ]
            SampleTypeInner.intTuple = (101, 102) } ]
        |> SampleTypeOuter.SomeData

xUnit’s Assert.Equal result:listsize

xUnit-Jet’s Assert.equalDeep result:listsize


Values Within Collections

let [<Fact>] ``inequality of value within a list`` () =
    let actual =
        [ { SampleTypeInner.primitiveOption = None
            SampleTypeInner.stringList = [ "stringOne" ; "stringXXX" ]
            SampleTypeInner.intTuple = (101, 102) }
          { SampleTypeInner.primitiveOption = Some 3
            SampleTypeInner.stringList = []
            SampleTypeInner.intTuple = (201, 202) } ]
        |> SampleTypeOuter.SomeData

xUnit’s Assert.Equal result:listvalue

xUnit-Jet’s Assert.equalDeep result:listvalue


Values Within Tuples

let [<Fact>] ``inequality of value within a tuple`` () =
    let actual =
        [ { SampleTypeInner.primitiveOption = None
            SampleTypeInner.stringList = [ "stringOne" ; "stringTwo" ]
            SampleTypeInner.intTuple = (101, 102) }
          { SampleTypeInner.primitiveOption = Some 3
            SampleTypeInner.stringList = []
            SampleTypeInner.intTuple = (201, 999) } ]
        |> SampleTypeOuter.SomeData

xUnit’s Assert.Equal result:tuplevalue

xUnit-Jet’s Assert.equalDeep result:tuplevalue

6 comments

  1. This is so beautiful. Are there plans to put it on Nuget? (Unless I’m missing something, I couldn’t see it there.)

    1. Yes – we’re planning to put it on NuGet, but we want to give the owners of xUnit a chance to take a look at it first. It would be cleaner to just roll it into xUnit if possible – if that doesn’t shake out, we’ll formalize the publication process and get it in a state that’s easier to consume.

  2. Nice work!

    Have you thought about how you might decouple the `deepEquals` implementation from xUnit? i.e. I’m thinking a function like `diff: expected -> actual -> option` would be generally very useful (where `Diff` is a type which describes differences between `expected` and `actual`, if any). Namely, it could make complex record comparison output easier to decipher in Unquote… instead of `test ` you could do `test `.

Leave a Reply

Your email address will not be published. Required fields are marked *