A couple years ago I wrote a library called AlterEgo. It’s an implementation of the State Pattern for Ruby. I consider the library to be retired at this point—other libraries, such as State Machine and ActiveModel, have since incorporated all of its important features.

Every now and then, though, I still receive unsolicited compliments on the AlterEgo spec suite. Let me say that again: I get compliments specifically for the tests, not for the library itself.

When I sat down to write AlterEgo, I decided to try an experiment. I decided to write “literate specs” for the code—specs that would double as the library’s executable documentation. My workflow went like this:

  1. Write some introductory verbiage to an AlterEgo feature. Put the text inside a comment block.
  2. Immediately following the comment, write a spec that clearly demonstratred the feature in idiomatic Ruby and RSpec.
  3. Update the library so that the spec passed.
  4. Go back to step 1.

Judging by the responses I’ve received, the experiment was a success. Here’s a snippet from the AlterEgo specs, to give you a feel for the format (note, this was in an older version of RSpec):

# It is possible to get a list of currently handled requests, as well as a list
# of all possible requests supported in any state.

describe TrafficLightWithRedCountdown do
  before :each do
    @it = TrafficLightWithRedCountdown.new
  end

  it "should know what requests are supported by states" do
    @it.all_handled_requests.should include(:cycle, :color, :seconds_till_red)
  end
end

This “literate spec” style was inspired by Donald Knuth’s “Literate Programming” methodology. Knuth’s idea was to approach programming the same way one would approach writing a paper or a book:

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reınforce each other.

Knuth had another explanation for the name he chose, one which has always amused me, and which inspired the title of this article:

I must confess that there may also be a bit of malice in my choice of a title. During the 1970s I was coerced like everybody else into adopting the ideas of structured programming, because I couldn’t bear to be found guilty of writing unstructured programs. Now I have a chance to get even. By coining the phrase “literate programming,” I am imposing a moral commitment on everyone who hears the term; surely nobody wants to admit writing an illiterate program.

By choosing RSpec for the AlterEgo specs, I was deliberately choosing a syntax which would build naturally from the prose descriptions of the functionality being described. Could I have written the same specs using Test::Unit? Certainly. But I daresay I would have been forced to use more in-line comments to explain the particular behavior I was demonstrating. The RSpec, in most cases, reads very similarly to the English explanations I would have otherwise employed.

TDD (and by extension, BDD) is first and foremost a design discipline; it is a tool for thought which enables us to tease out the needed code structure in a very organic way. But there are side-benefits to TDD which we also find very helpful to the software process. The most obvious is the fact that it produces comprehensive regression test suites as a by-product.

Another fringe benefit of TDD which has fallen somewhat by the wayside, in my opinion, is the notion that the tests can double as the documentation—documentation which, by its nature, can never fall out of sync with the source code. This feature of TDD was a Big Deal back when I was first getting into open source; it was very popular to reply “just read the tests” in response to the question “where is the documentation”, and this was a novel change from the heavyweight documentation which had up until then been considered one of the requisites of software development.

I don’t see this as often in Ruby land these days. More often, the documentation consists of the README, a blog post or two, perhaps a screencast, and some Wiki pages on GitHub. I often find that the test/spec suite for a given class is essentially a “bag o’tests”, with no particular organization or attention to readability. The tests presumably served their purpose as part of the design process, and are now doing time as regression checks—but they are in no way ambassadors for the library they describe.

There are exceptions. And more often than not, when I find exceptions, they are written in RSpec or Cucumber, not Test::Unit. This is anecdotal, of course, but it makes sense to me. The choice of RSpec is, at some level, the choice to think about code from the perspective of English prose, rather than from the perspective of code. I can’t say for certain which is cause and which is effect, but in my experience “literate” and RSpec/Cucumber tend to go hand-in-hand.

One of my favorite recent examples is the specs for Myron Marston’s wonderful VCR library, which you can read on Relish. Personally I think those specs should be preserved and enshrined as some sort of international monument to executable tests. They are sublime.

Just as a counter-example to my own evidence, however, I will also note that Ara T. Howard accomplishes a similar effect in pure Ruby; no test frameworks needed:

require 'arrayfields'
#
# the class Array has only a few added method, one is for setting the fields,
# when the fields are set for an array THIS INSTANCE ONLY will be modified to
# allow keyword access.  other arrays will not be affected!
#
  a = [0,1,2]
  fields = ['zero', 'one', 'two']
  a.fields = fields                # ONLY the Array 'a' is affected!
#
# keyword access is now allowed for many methods
#
  p a['zero']                        #=> 0
  p a['one']                         #=> 1
  p a['two']                         #=> 2
  p a.at('one')                      #=> 1
  p a.values_at('zero', 'two')       #=> [0, 2]

There is a different, but related, effect of using RSpec. Not only does it seem to encourage a literate test-writing style, but it also tends to encourage slightly more idiomatic Ruby, at least among its more conscientious users.

Here’s a short example of what I’m talking about. Let’s say I have a class “Frood”, which defines a couple of predicates:

class Frood
  def knows_where_its_towel_is?
    true
  end

  def is_hoopy?
    true
  end
end

Let’s take a look at the Test::Unit tests for the predicates:

f = Frood.new
assert(f.is_hoopy?)
assert(f.knows_where_its_towel_is?)

Straightforward enough. Now let’s take a look at the equivalent idiomatic RSpec:

f = Frood.new
f.should be_is_hoopy
f.should be_knows_where_its_towel_is

BLEAAAARGH!!! That looks awful!

Why does it read so poorly? Because we used some non-idiomatic naming conventions for our predicates. If we rename them to the idiomatic #hoopy? and #aware_of_where_its_towel_is?, respectively, then we get some much better-looking RSpec:

f = Frood.new
f.should be_hoopy
f.should be_aware_of_where_its_towel_is

This is just one example. There are a lot of little ways that RSpec makes non-idiomatic Ruby code look glaringly obvious. It tends to break, for instance, when you implement #method_missing without a matching #respond_to?.

And I think this is one of the reasons it gets a lot of grief: a lot of people write mildly non-idiomatic code—I know I do, from time to time—and RSpec makes this kind of code look very ugly. It’s similar to the complaints I often see about Mock Objects: in most cases, when I see someone complaining about how elaborate and brittle their mocks are, I find that the code in question is littered with SRP violations. Mocks, and to some degree RSpec, are like the Hydrogen Peroxide of programming: they fizz up where they encounter subtle technical debt.

I know, I know, I can hear what you’re thinking. “I don’t need no RSpec to tell me if my code sucks!”. Maybe you’re right. TDD, BDD, RSpec, Literate Programming—they are all just tools for thought, and if we can think real good we don’t need them.

Practically every hacker seems to go through a phase where he or she reads about JWZ writing Netscape without any Unit Tests and says to him or herself “yeah! tests are overrated! I’m going to be just like JWZ and just code it right!“. The flaw in that plan is that they aren’t JWZ. Neither am I, and statistically speaking, neither are you. There’s a reason that even seasoned airline pilots step through written checklists during preflight. Even masters make use of disciplines, and tools for thought.

The point of this article is not to convince you to use RSpec. If anything, I just want to provoke you to ask yourself some questions about your testing discipline. Are your tests literate? Are you writing test suites that read like manuals to your code? Does the code you are working on warrant tests like that (the answer may be “no”)?. Are you writing example code that looks awkward and forced, or that is flowing and idiomatic? How would your classes and methods change if you were writing an essay on them for publishing in a printed book? Whose eyes are you looking at your tests through—the eyes of the newest member of your team, or your own seasoned eyes?

If you know of more examples of literate specs—whether in RSpec, Test::Unit, Shoulda, or any other form—I’d love to see them! Please leave a link in the comments.

Published by Avdi Grimm

22 Comments

  1. I agree a lot with what you say (which is different from agreeing with a lot of what you say).

    In defense of the unidiomatic code, however, you would probably write it as

    f.knows_where_its_towel_is?.should be_true

    But your point is taken.

    Reply
    • The predicate matchers of RSpec are one of the best features (imho).

      I love it when I can express an idea so clearly, that I can write it down in very easy specs, like these:

      it { should be_busy }
      it { should have_queued_items }

      I actively try to write my specs like this before writing code, which leads to simple interfaces and design.

      Reply
    • I prefer Mark’s idea but as someone who hasn’t started using RSpec I could be wrong. I think trying to rewrite the method names to suit the tests is not the right idea. I’d rather my code read nicer since that’s where these methods will be used more.

      f.aware_of_where_his_towel_is is not as nice to look at as f.knows_where_his_towel_is

      (e)

      Reply
      • Just to be clear, I was talking about a replacement for

        f.should be_knows_where_its_towel_is

        I actually do like the be_*, has_* RSpec dynamic methods, I do use them where appropriate, and I would rename a predicate method to better suit them.

        But if I had to test a method called #knows_where_its_towel_is? I would use the alternative that I mentioned.

        Reply
  2. I often write specs/tests for a method and find it helpful to read it as such.

    describe ‘Frood’ do
    subject{ Frood.new }
    describe ‘#hoopy?’ do
    context ‘when hoops are present’ do
    before { subject.hoops << Hoop.new }
    specify{ subject.hoopy?.should be_true }
    end
    end
    end

    That seems like a lot for a simple method, but it tells me what the methods are, how each behaves and under what conditions. But looking at that now, it seems a bit backwards.

    What I don't like about your test above is that I don't know what the methods are. Seeing that something "should know about" something else only implies the concept, but it doesn't tell me how I'll be able to access what it knows about.

    Am I missing something?

    Reply
    • That class is a reference to The Hitchiker’s Guide to the galaxy:

      A hoopy is a realy together guy. A frood is a really amazingly together guy.
      Hey you sass that hoopy, Ford Prefect, there’s a frood who really knows where his towel is

      Although it seems now that I have been misusing “hoopy” all these years; I thought it was an adjective, but it is apparently a noun..

      Reply
  3. This is probably the best piece of writing about testing that I’ve read this year. Thank you.

    My first introduction to literate programming with Ruby was via your RPCFN challenge last year. I vividly remember running the Cucumber features you supplied as acceptance tests after bumbling my way through TDDing the solution using RSpec. I remember feeling extremely grateful for those thoughtfully written features. They allowed me to refocus on the important bits of the solution.

    That experience was part of the inspiration for some work I’ve been contributing to recently on a step-def-free Cucumber library. Our specs could use a little more literacy, but our Example project’s Cucumber features are reading pretty well, I think. Would really appreciate your feedback:

    https://github.com/RiverGlide/CukeSalad

    Reply
  4. While it’s a convincingly worded read, I don’t see much gain between….

    f = Frood.new
    assert(f.is_hoopy?)
    assert(f.knows_where_its_towel_is?)

    …. and …..

    f = Frood.new
    f.should be_hoopy
    f.should be_aware_of_where_its_towel_is

    I am tending to listen to DHH on this issue, being a fence sitter for some time.

    Reply
    • You might want to give it another read, then. The point is that a method called #is_hoopy? is idiomatically incorrect by Ruby conventions, and somewhere down the line that may confuse or annoy another programmer. The idiomatic form is #hoopy?, without the “is_”. The syntactic vinegar of the RSpec syntax for testing predicates makes that glaringly obvious, while (as you demonstrate in your comment), the Test::Unit syntax lacks this red flag.

      Reply
      • “#is_hoopy? is idiomatically incorrect by Ruby conventions …”

        “test”.is_a?(String)=> true

        This is standard and seems natural to me. I’m trying to learn rails and I find the syntax of rspec very annoying at the moment. It doesn’t seem ruby like at all to me + I feel like I have to learn a whole new language, which I don’t have the time for…

        Reply
        • One of the greatest things about Ruby is that it is sometimes inconsistent. In other words, given a choice between consistency and usability, Ruby goes for the latter.

          In your example, consistency would say to use “test”.a?(String) to be consistent with its close companion “test”.kind_of?(String). But that’s not as readable. So Ruby inconsistently sticks an “is_” in there.

          If you don’t have time to learn RSpec (which is not “a whole new language”), then you don’t have time to improve your productivity. That’s all RSpec, or any other tool, is – a way to get good code out faster. There’s no other reason to use it. If you don’t have time to learn some basic industry-standard tools, then you’re like the woodsman who has no time to sharpen his axe because he has all these trees to cut down.

          Reply
        • One of the greatest things about Ruby is that it is sometimes inconsistent. In other words, given a choice between consistency and usability, Ruby goes for the latter.

          In your example, consistency would say to use “test”.a?(String) to be consistent with its close companion “test”.kind_of?(String). But that’s not as readable. So Ruby inconsistently sticks an “is_” in there.

          If you don’t have time to learn RSpec (which is not “a whole new language”), then you don’t have time to improve your productivity. That’s all RSpec, or any other tool, is – a way to get good code out faster. There’s no other reason to use it. If you don’t have time to learn some basic industry-standard tools, then you’re like the woodsman who has no time to sharpen his axe because he has all these trees to cut down.

          Reply
    • describe Frood do
      it “is hoppy” do
      assert { Frood.new.hoppy? }
      end

        it "knows where its towel is" do
          assert { Frood.new.knows_where_its_towel_is? }
        end
      end
      

      Then the output looks like this:

      Frood
      is hoppy
      knows where its towel is

      You can scan that (or a spec with dozens of examples) very quickly and understand what’s going on. These days you can get with t/u with extension libs like TURN, but RSpec’s been doing this for years.

      Reply
  5. @avdi: I love rspec, and I’ve been using it for years, but I still get the sinking feeling that I’m structuring my specs wrong. Are there any projects you think have well written specs?

    Reply
  6. This seems like an appropriate venue for a shameless plug: wrong

    https://github.com/sconover/wrong

    Alex and I strongly believe that pure ruby asserts/expects contribute to readability, it’s much of what motivated us to write wrong. And it plugs into rspec.

    RSpec: BlueCheese.new.smell.should be > 9000
    RSpec + wrong (BDD style): expect { BleuCheese.new.smell > 9000 }

    test unit: assert_equal time, money
    Wrong: assert { time == money }

    We only have one assert method, and you express predicates in plain ruby. We let plain ruby do the talking.

    Reply
  7. Awesome post. Love your writing style.

    Reply
  8. Contest is for the literate that like clean, fast software: https://github.com/citrusbyte/contest

    I left RSpec years ago because it was causing too many errors itself. I also didn’t appreciate that they decided to completely refactor the interface on me (pre-bundler days, that was a nasty thing to deal with). And their documentation frankly was lacking, a problem the heavy use of magic didn’t help. Honestly, RSpec to me is more of a testament to how overboard ruby coders will go with software rather an example of how literate they are. In the above example, you get almost all of the way there on Test::Unit with one hundred lines of code.

    I agree and loved your article, though. Literacy is a staple of the Ruby world, and I’m not sure I’d ever want to go back. Not going to name names, but I’ve been working with an event-driven language that is all the rage right now. I’ve noticed their developers are horrible at spelling. It bothers me a little bit, makes me uncomfortable.

    I’ve been using MiniTest’s spec on ruby 1.9 and I’ve also found that satisfactory. I’m thinking of combining it with Wrong for a project I’m working on. Great thing about ruby.. even if you don’t agree on the software choice, you’ve got a dozen alternatives!

    Reply
    • Personally, I don’t think nested contexts are really the primary value proposition of RSpec, so I think it’s missing the point a bit when people point out that you can get nested contexts without much effort. I’ve worked with a few different nested context plugins for Test::Unit, and while they are handy, they are just a small part of the overall “RSpec experience”.

      With that out of the way, I’m really glad you saw that this isn’t really an article about RSpec so much as an article about how one approaches writing tests. Thanks for the comment, and happy testing!

      Reply
  9. I guess this can be an example of literate specs: https://github.com/cs3b/steak_fm

    Sorry if i didn’t get your idea 😉

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *