When testing becomes part of your library
Thursday 30 November 2017
Being a haskell library provider means you have a contract toward your user: you provide them with a minimum degree of confidence your code work.
This is quite important. And thankfully Haskell maintainers are quite good with that. The language provides the abstraction to avoid the common mistakes; the semantic is checked at compile time. But what about the implementation itself? Or the properties? Is it correct? Is your functor actually a functor? Will it behave like everyone is expecting? Is your JSON parser parsing the data as expected?
There is an Haskell library for that: Quickcheck. We all know about it, most of us are using it. It has been doing the job okay so far.
Now, other languages like Rust, have decided to include testing as a core feature. I believe it is a good idea to include the testing framework as early as possible in the core library; or within the language itself.
Some would rather like a more fragmented libraries. At the Haskell Foundation we believe it is better to unify, to bring together core libraries, to facilitate the standardisation of haskell libraries and have inter operational types. The foundation unifies modern Haskell concepts and real word programming. And because foundation is one of the core library if not the core library. This is why it comes with a complete testing framework.
Foundation's types come with instances of
Just like you would find in Quickcheck (Foundation does not aim to re-invent the wheel):
class Arbitrary a where arbitrary :: Gen a
Arbitrary available at the foundation of the ecosystem is really
useful and it prevents from seeing the most infamous compilation warning to see:
If you are using Quickcheck (and didn't want to see the all its dependencies
added to your package) you have certainly seen this compiler message.
With Foundation's Check you can add
Arbitrary to your types. Allowing you to
use it in your tests but also allowing the users of your library to re-use
the instance of
Arbitrary in their own tests.
Foundation is not re-inventing the wheel here. It is mostly what you
would have with Quickcheck, except that in Foundation you use the
constructor instead of specific functions. In Foundation, it is called
data Test where Group :: String -> [Test] -> Test Property :: IsProperty prop => String -> prop -> Test CheckPlan :: String -> Check () -> Test
so we have 3 constructors:
Groupwill allow you to build some sort of hierarchy within your tests;
Propertytest any kind of property (see below);
Propertywill allow you to validate properties but unlike
Propertyit allows you to acquire resources first.
Well this is easy:
main :: IO () main = defaultMain $ Test "Properties" [ Test "Functor"  , Test "Applicative"  , Test "Monad"  ]
This constructor regroup the 2 different kind of test you will want to do most of the cases:
convertToBase Base64 "" === ""
convertFromBase Base64 . convertToBase Base64 === id
And to do that, no need for different constructor, the
class will help you.
To do a test once, and only once without generating data (unit test):
main :: IO () main = defaultMain $ Property "base64 of nothing is nothing" (convertToBase Base64 "" == "")
The type check understood this is a
Bool type. The test has no parameter,
it only need to run once.
When you want to test a property is true for any arbitrary value of a given type you will want to write a Property Test:
main :: IO () main = defaultMain $ Property "unbase64 . base64 == id" $ \d -> Right d === convertFromBase Base64 (convertToBase Base64 d)
The type checker see that you need a parameter, checks it is an instance of
Arbitrary and generate it for you.
Foundation provides handy function and operator. Here
=== will provide you
with a nice little error display in case of something is different.
Prime Common ✗ Time failed after 1 test ✗ parseJSON . fromJSON === id failed after 1 test use param: --seed 14494984503458014416 parameter 1 : 1999-09-03T06:46:42+00:00 parameter 2 : 1997-02-12T23:17:46+00:00 Property `a == b' failed where: a = Right 1999-09-03T06:46:42+00:00 :: Either [Char] Time ^ ^ ^^ ^^ ^^ ^ b = Right 1997-02-12T23:17:46+00:00 :: Either [Char] Time ^ ^ ^^ ^^ ^^ ^
In this case the display is really neat. It won't be perfect all the time but it can provide a useful hint in case of error. Foundation's Check does not only fail the test, it provides you with details that can help you same time here.
Below is the code that generated the error above:
main :: IO () main = defaultMain $ Property "fromJSON . toJSON == id" $ \a (b :: Time) -> Right a === parseJSON (toJSON b)
So now, this is not a finished business. Foundation's community is still building up. But the progress so far are astonishing and it is a pleasure to use advanced haskell in real world project.