PBT Haskell -- QuickCheck + Hedgehog

Framework Selection

Framework	Shrinking	Stateful	Choose When
QuickCheck	Type-based (manual)	Limited (open-source)	Default for most projects. Universal ecosystem support.
Hedgehog	Integrated (automatic)	Yes (parallel too)	Need automatic shrinking composition or parallel stateful testing

QuickCheck is the original PBT framework (2000). Hedgehog is the modern alternative with better shrinking.

Quick Start (QuickCheck)

import Test.QuickCheck

prop_reverse_involutory :: [Int] -> Bool
prop_reverse_involutory xs = reverse (reverse xs) == xs

-- Run: quickCheck prop_reverse_involutory
-- Or in test suite: testProperty "reverse" prop_reverse_involutory

Generator Cheat Sheet (QuickCheck)

Via Arbitrary Type Class

arbitrary :: Gen Int       -- any Int
arbitrary :: Gen String    -- any String
arbitrary :: Gen [Int]     -- any list of Ints
arbitrary :: Gen Bool

-- Custom Arbitrary
data Color = Red | Green | Blue deriving (Show, Eq)

instance Arbitrary Color where
  arbitrary = elements [Red, Green, Blue]
  shrink Red = []
  shrink _   = [Red]

Gen Combinators

choose (0, 100)              -- bounded int
elements [1, 2, 3]           -- pick from list
oneof [gen1, gen2]           -- union
frequency [(80, gen1), (20, gen2)]  -- weighted

listOf arbitrary             -- list
listOf1 arbitrary            -- non-empty list
vectorOf 5 arbitrary         -- fixed-length list

-- Map
fmap (* 2) arbitrary         -- even integers

-- Bind (dependent generation)
do xs <- listOf1 arbitrary
   x  <- elements xs
   return (xs, x)

-- Sized
sized $ \n -> resize (n `div` 2) arbitrary

Shrinking

instance Arbitrary MyType where
  arbitrary = ...
  shrink (MyType a b) = [MyType a' b | a' <- shrink a]
                     ++ [MyType a b' | b' <- shrink b]

Stateful Testing (QuickCheck)

Not supported in open-source QuickCheck. Use Hedgehog for stateful testing, or commercial Quviq QuickCheck (Erlang).

Quick Start (Hedgehog)

import Hedgehog
import qualified Hedgehog.Gen as Gen
import qualified Hedgehog.Range as Range

prop_reverse :: Property
prop_reverse = property $ do
  xs <- forAll $ Gen.list (Range.linear 0 100) Gen.alpha
  reverse (reverse xs) === xs

-- Run: check prop_reverse

Generator Cheat Sheet (Hedgehog)

Gen.int (Range.linear 0 100)
Gen.int (Range.constant 0 100)    -- no size scaling
Gen.double (Range.linearFrac 0 1)
Gen.string (Range.linear 0 50) Gen.alpha
Gen.bool
Gen.element [Red, Green, Blue]

-- Collections
Gen.list (Range.linear 0 50) Gen.alpha
Gen.nonEmpty (Range.linear 1 50) Gen.alpha
Gen.set (Range.linear 0 20) Gen.int

-- Map
Gen.int (Range.linear 0 100) <&> (* 2)

-- Filter
Gen.filter (> 0) (Gen.int (Range.linear minBound maxBound))

-- Recursive
Gen.recursive Gen.choice
  [ Gen.constant Leaf ]
  [ Node <$> genTree <*> Gen.int (Range.linear 0 100) <*> genTree ]

Hedgehog Ranges

Range.linear 0 100       -- scales with size parameter
Range.constant 0 100     -- fixed bounds, no scaling
Range.linearFrac 0.0 1.0 -- fractional, scales with size
Range.singleton 42       -- always 42

Stateful Testing (Hedgehog)

Hedgehog supports sequential and parallel state machine testing.

import qualified Hedgehog.Gen as Gen
import qualified Hedgehog.Range as Range
import Hedgehog
import Hedgehog.Internal.State

-- Define state and commands
newtype ModelState (v :: * -> *) = ModelState { items :: Map String Int }

data Put (v :: * -> *) = Put String Int deriving (Show, Eq)
data Get (v :: * -> *) = Get String     deriving (Show, Eq)

instance HTraversable Put where
  htraverse _ (Put k v) = pure (Put k v)
instance HTraversable Get where
  htraverse _ (Get k) = pure (Get k)

cPut :: (MonadGen n, MonadIO m, MonadTest m) => Command n m ModelState
cPut = Command
  (\state -> Just $ Put <$> Gen.string (Range.linear 1 10) Gen.alpha
                        <*> Gen.int (Range.linear 0 100))
  (\(Put k v) -> liftIO $ storePut realStore k v)
  [ Update $ \(ModelState m) (Put k v) _out -> ModelState (Map.insert k v m)
  ]

cGet :: (MonadGen n, MonadIO m, MonadTest m) => Command n m ModelState
cGet = Command
  (\(ModelState m) -> if Map.null m then Nothing
                      else Just $ Get <$> Gen.element (Map.keys m))
  (\(Get k) -> liftIO $ storeGet realStore k)
  [ Ensure $ \(ModelState before) _after (Get k) out ->
      out === Map.lookup k before
  ]

prop_store :: Property
prop_store = property $ do
  actions <- forAll $ Gen.sequential (Range.linear 1 50)
    (ModelState Map.empty) [cPut, cGet]
  executeSequential (ModelState Map.empty) actions

For parallel testing, replace

Gen.sequential

with

Gen.parallel

and

executeSequential

with

executeParallel

. Checks linearizability of concurrent operations.

Test Runner Integration

QuickCheck

-- With HSpec
describe "sort" $ do
  it "preserves length" $ property $
    \(xs :: [Int]) -> length (sort xs) == length xs

-- With Tasty
testGroup "sort" [ testProperty "length" prop_sortLength ]

-- Cabal/Stack: depends on QuickCheck, hspec-discover or tasty

Hedgehog

-- With Tasty
testGroup "sort" [ Hedgehog.testProperty "length" prop_sortLength ]

-- Cabal/Stack: depends on hedgehog, tasty-hedgehog

Unique Features

QuickCheck

• The original: 25+ years, every Haskell test framework integrates with it
• Arbitrary type class: One generator per type, automatic derivation via Generic
• Quviq commercial version: Most complete PBT implementation ever built (Erlang)

Hedgehog

• Integrated shrinking: Automatic via rose trees, composes through applicative
• No orphan instances: Explicit generators avoid type class problem
• Range combinators: Fine control over value scaling with size
• Parallel stateful testing: Built-in linearizability checking