Writing concept - crumpled up paper wads with a sheet of white paper

Don’t throw away invalid data

Recently I saw a wonderful talk about domain modeling in F#, by Scott Wlaschin. Here’s an older version of it. It was a superbly executed presentation, crystal clear and compelling. But I had a minor quibble. 

Wlaschin drew on Yaron Minsky’s quote:

Make illegal states unrepresentable

This strategy is extraordinarily attractive to developers wanting to build robust, predictable systems. But based on both personal experience and many discussion with colleagues about their projects, it comes in direct conflict with actual end-user needs time and time again.

What inevitably happens is that end users attempt to use the software in legitimate but unforeseen (or previously unobserved ). The system then rejects their inputs and throws away their data. Finding the software to be an obstruction to work, they then route around it. If we are very lucky, they also complain to us, the developers. More often, the system is simply quietly sidelined from important but invisible (to us) workflows.

The canonical representation of this, in my mind, is the terminal in a busy workplace that is plastered with sticky notes documenting how to “fool” the software into facilitating an unanticipated workflow—including which fields to fill with fake data in order to proceed.

The problem shows up in microcosm as well. When I fat-finger a form on a website, the last thing I want to see is blanked-out fields accompanied by an error message. The client-friendly thing to do is to re-display the form with the original “illegal” data still in place along with hints on how to fix it. The same goes for API responses, if you want to be compassionate to the developers who are learning to code to your API. This is why Ward Cunningham documented the Exceptional Value pattern, to preserve invalid data from request to response.

Most business “transactions” grow to be processes, with needed information supplied piecemeal as a part of a conversation. And we can’t predict in advance which datatypes in a system will need to to be preserved in an “illegal” state… or how long they will need to be preserved that way. We may need to store an incomplete entry in the database for days or weeks until a human can review it and apply their judgement. Throwing any part of that information away because it is “illegal” is negligent system design.

It feels so clean and satisfying to type data as valid or nothing:

type EmailAddress = 
  | Some EmailAddress of String
  | None

But most of the time, the graceful choice is to preserve data regardless.

type EmailAddress =
  | Valid EmailAddress of String
  | Invalid EmailAddress of String