Boolean Externalities

The other night I spent a long time trying to figure out why an acceptance test was failing. Eventually I tracked it down to the fact that a particular predicate method was returning false , when I expected it to be returning true.

Ultimately I would find that the test failure pointed back to a legitimate oversight in my code. But I wasn’t there yet. First I had to work my way back to the source of the unexpected false.

Down the rabbit hole

The proximal source of my frustration was this method:

class Episode
  # ...
  def available_to_user?(user)
    free? || user.current_subscriber?
  end
  # ...
end

Simple enough method. It turned out that of the two possible reasons, the false return was coming from user.current_subscriber?, which looks like this:

class User
  # ...
  def current_subscriber?
    fully_authenticated? && subscription.open?
  end
  # ...
end

This time, the offending value was stemming from the left side of the &&. Digging in, I found this:

def fully_authenticated?
  authenticated_accounts? && verified_email?
end

I dutifully checked the return values, and followed the right-hand fork to this:

def verified_email?
  email_verification.confirmed?
end

Following one step further down the rabbit hole led me here:

class EmailVerification
  # ...
  def confirmed?
    !!confirmed_at
  end
  # ...
end

At last, I was at the end of the boolean chain. I hadn’t yet discovered why confirmed_at had not been set for the test user in question, but now I knew what I was looking for.

Debugger Blues

Following this trail was not easy, or particularly pleasant. I used a debugger to do it. Past experiences have left me gun-shy when it comes to debuggers; generally when I find myself contemplating using the debugger, I start questioning my recent life choices. Nonetheless they are sometimes a necessary evil.

Using the debugger meant first ensuring that I could catch the software at just the right moment to examine the values in question. Fortunately I was dealing with a deterministically failing test, not an occasional runtime error. Even more fortunately I was using RubyMine, which does a pretty decent job of taking the pain out of debugging in Ruby, even when working with the split client/server processes of a Capybara testing session.

Which is not to say that the debugging process was enjoyable. Thankfully this problem could be traced at a single point in execution, and didn’t require stepping. But I still had to methodically evaluate each predicate in turn to see which one(s) were returning unexpected values. In case you weren’t keeping count, that’s eight different methods.

Evaluate episode.available_to_user?. Click through to the method definition. Evaluate free?. Evaluate user.current_subscriber?. Click through to the method definition. Rinse. Repeat.

I could have done this without a debugger, of course. I could have annotated the methods with good-ole’ print statements instead. This is always a little tricky with query methods, because you have to make sure to preserve the method’s return value. One annotated method might have looked something like this:

class Episode
  # ...
  def available_to_user?(user)
    result = (free = free?) || (current_subscriber = user.current_subscriber?)
    puts "episode.free? == #{free}"
    puts "user.current_subscriber? == #{current_subscriber}"
    result
  end
  # ...
end

This is a huge alteration to the method. If I had annotated seven more methods like this, I would have had a complete picture of why that boolean return value was false. Of course, I probably also would have had a screen full of output, since the method is probably called more than once in a single test, and I would have had to figure out which output corresponded to the point in the test I actually cared about…

Why does this hurt?

Ordinarily when I have a painful development experience, I reflect and come away with some conclusions about how I could have written the code better in order to avoid the problem. But here I was left scratching my head.

Because this is good code by most conventional measures. Every method is short and meaningfully named. Each deals only with one object’s limited view of the world, and respects the encapsulation of other objects. The methods have no need of extra documentation, because their meaning is self-evident. What constitutes a “current subscriber”? Why, a fully authenticated account and an open subscription, of course. It’s self evident.

class User
  # ...
  def current_subscriber?
    fully_authenticated? && subscription.open?
  end
  # ...
end

Looking at this code, I can’t imagine wanting to structure it any other way, at least not while still remaining within the broad object-oriented paradigm. This code feels right.

And yet, it hurts. When I needed an answer, it took a long time and a lot of steps to get it. And I did not enjoy the process.

Information discarding

The fundamental issue is that while this code makes it very easy to ask “is a user allowed to view this episode?”, when the answer comes back “no”, there is no way to ask “well, why not?”.

One of the basic OO principles is information hiding: “I’ll answer your question, but you don’t need to know how I arrived at the answer.” Outside of the #current_subscriber? predicate I don’t want to be distracted by the details of how that answer is calculated. From a static point of view, this program does a great job at information hiding.

But from a dynamic perspective, this program isn’t just hiding information. It’s shredding it, burning it, and scattering the ashes. At every step in the chain of predicate methods, knowledge about the provenance of information is ruthlessly discarded in favor of cold, clean boolean yes-or-no responses.

And the problem is, as a developer sometimes I do need that information. And not necessarily only as a developer. One way to help users help themselves is to give them clues as to why something isn’t working as expected.

A frustrated user is staring at a disabled video player. Could that user make use of a hint, indicating that the reason they can’t watch a video may be related to the fact that their email address was never verified? Quite possibly. But with the system as it stands now, surfacing that information to the user would be a major project. It would require manually piecing together the chain of boolean consequences from start to finish.

Externalities

This all reminds me a bit of the concept of “externalities” in economics. According to some theories, prices in a market are an elegant way of dealing with the problem of limited information. No one can have an omniscient view of the entire market. But if individual buyers and sellers each make bargains based on their own limited sphere of knowledge—poor rainfall affecting crop yields; a fad diet increasing demand for bacon—in the end each product’s price will reduce all of those individual bits of knowledge into one essential number.

The problem is that sometimes the production of a good imposes a cost on someone who is neither the buyer or the seller. If manufacturing a widget has a side effect of pumping mercury into a nearby river, that might not affect the producer or the consumer in a meaningful way. But it has a value impact on the people living near the factory, one which may never be reflected in an increased price for widgets. These so-called “externalities” throw a wrench into the works of market-based economies.

I see the information lost along the steps of the logic chain in the code above as a kind of “boolean externality”. In theory, all the important information has been rolled up into one convenient boolean value. In reality, vital knowledge has been lost along the way.

Looking back, I realize that I’ve run into debugging problems like this one a lot over the years. And I’ve encountered similar situations where I discovered a need to expose why a decision was made to the user, but I didn’t have a clean way of tracing that information without building some very brittle special-purpose code.

This bothers me a lot, because as I said before, there is nothing wrong with this code as far as I can tell. It exhibits good OO style. Yet it falls short in making important information accessible. And I can’t imagine any changes short of a sweeping architectural shift—and a lot of extra code—which would change the situation.

Where to from here?

Is there a better way? I’m half assuming that readers from the functional programming camp have been quietly giggling to themselves for several paragraphs already. I suspect that this is one of those things that are well addressed by the data-centric viewpoint advocated by folks like Rich Hickey. I’d love to hear from someone with more practical FP experience than I have, demonstrating exactly how this problem would be addressed (and/or wouldn’t exist in the first place) in e.g. idiomatic Clojure.

I’m also mindful of the Functional Reactive Programming model, in which values are explicitly traced from source to destination. I don’t know for sure, but I imagine that an FRP environment might take a lot of the pain out of this code.

What do you think? Is this a pain you’ve felt as well? Do you have ideas about how to solve it? Is there something obvious I’m missing? Are there paradigms or technologies that better address the problem of boolean externalities? I’d love to hear your thoughts on this in the comments or on your own blogs.

EDIT: Kevin Rutherford replied with a thought-provoking post about replacing booleans with a state model.

48 comments

  1. With a Result or Either type, you can communicate more information about why something happened. In Ruby, this would mean returning a tuple of (true/false, String) instead of just a bool. You could include something more complex than a String too, but that’s the simple case.

    def confirmed?
    if confirmed_at
    (true, “You have been confirmed”)

    else
        (false, "You are not confirmed)
    
    end    
    

    end

    and likewise, up the stack. You can build up operators to just work on the first value when you need it, discarding the others.

    This would be very, very odd in Ruby, though…

    1. Yeah, that’s the rub, isn’t it? It’s odd in Ruby to build something that could give me the info I need. Moreoever, it’s more work, and by the time I discover the pain it’s some serious retrofitting work to fix it.


      Avdi Grimm
      http://avdi.org

        1. Can you show/explain some sample code? I’m having a hard time imagining how it would simplify this code. (I understand that storing external information about a value is the big deal with macros, but I’m unsure how it would improve this specific situation).

          1. This is not Haskell code I’d want to ship, but it’s perhaps good enough.

            If you run the code, it will spit out “Woah baby, error: Wrong input”

            If you take off the line marked with the comment from the someThing function (the one that passes in 17 to sub_meow) then it will say “Success! Big Meow”

            In this case the return type is a string, but it could be whatever.
            https://gist.github.com/psycotica0/31ca4ac945f5203fc0e7

          2. These aren’t monads, though. These are just types. Would be analogous to Either.new(:left, “something wrong”), Either.new(:right, “something right”).

            This still makes for ugly ruby code. I guess you could do something like include Either and then do Left.new or Right.new.

            Makes me think of something akin to:

            (logic { first_condition? } .and { second_condition? }).or { third_condition? }

            … where the logic system knows how to handle condition metadata.

            def first_condition?
            logic(:false, {here: :is, some: :metadata})
            end

            … etc.

    2. I’m unclear how this solution actually helps – a method like fully_authenticated? is still going to need to decide which message to include in the output. Something is still getting lost in that case, and there’s a significant increase in complexity.

      1. Typically this is described as having a need to “combine errors” somehow. Typically, within a particular problem domain the error space is known and finite and can all be wedged together. If you have a lot of error generating code then this gets painful, though.

        In Haskell, you typically avoid this by sectioning off effects and computations which are error producing so that you just don’t have to do all this rigamarole. That’s often a good program structure as well.

  2. This was a very interesting read and I’ll try to come up with a more profound answer, but for now: It seems like Disqus comments are broken on your site in Chrome 33 on Ubuntu. Had to use Firefox to view comments.

    1. Yeah, I lost my comment when i posted and had to take a heap snapshot to recover it. It worked on the second try.

  3. This reminds me of Douglass Adams’ 42 story. (TLDR; After a century, a world size computer returns the answer to life the universe and everything as 42; Bigger computer needed to determine the question).

    I’m wondering if the standard practice of returning naked booleans is a premature optimization. Maybe the convention should be to pass a boolean and a justification.

    I don’t know of any language level support for justifying a return value. I’d see something like ‘justify(retVal) -> reason’ being useful, as long as I could still pass around retVal as it’s normal type.

    On the other hand, maybe this a case in a wider problem: losing information when compositing data. Say we kept a tree around with all the data/functions used in compositions. Could we keep this tree around without horrendous performance problems?

    There’s a similar situation in an async environment. When you have a problem you can usually identify the event, but the event generator and the state of the event generator is usually lost. There are solutions where they store metadata that goes back a few events, but keeping spent events around uses up a ton of memory. Also, you almost always need one more frame to find the problem. Eight deep would be rather excessive.

    In tracing composition, I suppose you only need to trace back to mutable data, global state, or input parameters. Hmm.

    This is interesting and worth playing around with.

  4. On a side note, p with a hash would make that print debugging slightly nicer:

    p free?: free?, current_subscriber?: user.current_subscriber?

    You get convenient labeling with a minimum of syntax, and the values are automagically inspected. You can switch to pp if the values are complex enough.

  5. One first thought if we stay in realm of standard Ruby: If confirmed? instead would have been verify_confirmed! the reason for the failure could have been propagated as some (probably re-wrapped) exception. While using exceptions for control flow is an anti-pattern, it as least is a built in sort of “return value with context”.

  6. I’m not sure that FP/FRP would help in this situation. A user would be a struct and these functions would call each other in the same way. FRP doesn’t apply as there aren’t event streams coming in.

    Incidentally, this is a situation where macros would come in handy. define an and macro, with it returning both information on the value and the code/context that produced it.

  7. Generally, predicate methods are a shortcut to something more useful. Here’s a simple refactor to tease out the information that predicates typically smash:

    def unavailable_reasons
    [:some, :list, :of, :reasons, :why, …]
    end

    def unavailable?
    unavailable_reasons.any?
    end

    1. Go also extensively uses this. I guess they have exceptions now, but initially they used that for all error handling, even exceptional.

      It would mean something along the lines of

      class EmailVerification
      # …
      def confirmed?
      return !!confirmed_at, “EmailVerification#confirmed_at: #{confirmed_at.inspect}”
      end
      # …
      end

      def verified_email?
      email_verification.confirmed?
      end

      class Episode
      # …
      def available_to_user?(user)
      puts free? || user.current_subscriber?
      end
      # …
      end

      [false, “EmailVerification#confirmed_at: nil”]

      Pretty neat!

  8. This sounds like something where you’d return a Validation object that returns a list of errors — usually you’d see this in an Either, a disjunct union structure.

    1. Isn’t the fundamental “problem” here (I’m not convinced it’s a problem) that as Rubyists we’re both (a) discouraged from using exceptions for control flow, and (b) prevented from allowing an arbitrary object to be falsy. Exceptions can be handled and contain useful information; a complex object that evaluates to false could convey reason-data without affecting the flow of the code.

      I wish I had a solution instead of a complaint…

  9. Hello Avdi,

    First I want to say that I’m not a professional programmer so the following thoughts you might find irrelevant, and maybe they are.

    I think your code is not yet at the level of evolution where it can behave so that it follows your subjective expectations.
    See, the code computes (more or less) perfectly the needed result. It does so by the collaboration of several objects, each of which has no idea why it behaves the way it does. They are created with a purpose in your mind, but they themselves have no purpose. All they do is interact with each other, where their interaction is similar to calculations like this:
    (0,0,1,0) -> 1
    (1,1,1,1) -> 1
    (0,1,0,1) -> 0
    etc. Stripped from the semantics given by the text of the code, it is much harder to ask the “why” question. Is the user now happy when the result of the calculation 1? Or zero? The discrepancy is between the interaction of the brain cells of your (potential) subscriber and the interaction of your objects.
    Boolean values are value-free. Value in the human sense is subjective.
    Coming back to your analogy with externalities. The way the phenomenon is taught in most schools end after the second interaction (a third party not part of the original transaction is harmed). Mostly this is cited as an example why markets don’t work and government action creates a better society.
    But of course the chain of interactions does not stop here. The 3rd party harmed by the original transaction is going to act upon his interest. It is expected that the harmed party will use possibilities available to him to incorporate the costs incurred to him to the original parties in the transaction. (For example, by suing them) As a result, a new price for future transactions is born. Which will change the behaviour of future parties potentially willing to transact, which will result in a different transaction, which might lead to another 3rd party effected by the transaction, and so on. What is the meaning of the price at each transaction? Hard to say, but somehow it incorporates the values of each involved party on its evolutionary path.
    So the story in externalities should not be, how markets do not work, but this is exactly how they work. Given the universe we live in, there are always unintended consequences. Prices reflect our journey in discovering reality.

    So yes, your objects do not have a purpose. Asking them why do they behave the way they do makes no sense to them, the answer is “because it is the way I am”. They are the result of the evolutionary selection of your intentions. They do not care if the subscriber is happy. But once they are selected, adapted, arranged for long enough, their sum (your program) will show an emergent behaviour as if they did.

  10. I’m a Python programmer who recently had a similar problem with a large code base making use of lots of dynamism. I had an object and needed to inject information into it’s calling context and didn’t know all the places it could be created, so I didn’t know where I needed to make the context injection.

    Anyway, I banged out a quick hack to examine the execution stack and tell me both what-called-what, and the values of particular variables along the way. (Super shitty PoC at http://github.com/jrbl/python-wtf). If you had a similar utility for Ruby, then in your outermost method you would insert a call to the tracer and get a dump of all the calls along the way. I haven’t instrumented calling parameters and return values in python-wtf but now that I read your post, I see that that’s a really good idea, so I’ll probably do that RSN.

    As far as disciplinary changes… I don’t know what to tell you. I think this kind of thing just happens sometimes when you’re living in a highly dynamic world. My money’s on tools to audit after the fact.

    1. Actually on further reflection I realize that wouldn’t help you here, this is the opposite problem. You don’t want to know where your context came from, but where your locals came from. It seems like it shouldn’t be hard to write a similar tracer though that lets you wrap a method or attribute and watch its resolution, though…

  11. I also prefer simple output before resorting to a debugger when possible. I like to use Object#tap.

    free?.tap {|b| puts(“free?: #{b}”)} || user.current_subscriber?.tap(&method(:puts))

    2nd use of tap is easy to tack on if you don’t need a lot of other output noise, Both preserve return value. And it’s slightly simpler to undo than the multi-line alteration you demonstrated.

    1. Yeah, I always start like like. I made two handy Object#taplog & Object#tapputs that I can place into the code flow.

      free?.taplog("free?") || user.current_subscriber?.taplog("user.current_subscriber?")

      That outputs me an “[taplog] free? : false” into my dev logs.

      Having dedicated methods is also useful to Ctrl+Maj+F and find the forgotten ones (Object#tap being used at legitimate places).

  12. I think this would be addressed by Tell, Don’t Ask. That keeps errors localised to the place where they can be best handled. FP is not a huge help here, lazy evaluation tends to have a similar effect as your externalities.

  13. First, let me say that I share your feeling that sometimes we not only want to know about a truthy information but sometimes we also need to take some actions on some of them and to be honest I don’t have a single solution to the problem in my applications.

    Sometimes I use earlier returns when handling invalid data in a controller’s method, sometimes I use some filter, sometimes I use throw-catch, other times raise-rescue and in other contexts I use events/promises. Each of the approaches has their pro and cons and I usually have to resort to my common sense to decide what to do depending on the situation.

    If your code is too modular and you don’t want each module to be aware of the controller for instance, you could use some thread-local or request object to store some object to which you could register the events you’re interested on in the controller’s action and trigger the exceptional situations you want to handle in each module if you think it worths the effort.

    Now, with regards to code quality, you say the code is self-evident. I’d just like to point out that it may be self-evident to you, but it wasn’t to me as I was reading. For example “current_subscriber?” is not a clear name to me and I guess it would mean something like “has_valid_subscription?”. Then I have no idea on what “fully_authenticated” could mean and even less idea on what would be an open subscription. And the pause was longer as I was reading “authenticated_accounts”. Why would an user have multiple accounts? I simply assumed that it would be called “authenticated?” instead and then I’d get rid of “fully_authenticated?” and implement “has_valid_subscription?” as “subscription.valid? && verified_email?” and would move the authenticated check to some filter in the controller since I don’t really think it makes sense to ask a user whether it’s authenticated because this is not something inherent to the user but it’s something you extract from the request environment instead.

    Also, as I was reading the code I noticed that the user wasn’t being notified about an e-mail that has not been confirmed. From my point of view, access verification is responsability of the controller serving the content. As such, I think it would be fine to add such verification as some kind of before filters. If you want to be able to test them in isolation, fine, just move them to some module and test the module and make sure you include the right modules to your controller.

    Rails controller’s filters are already a pattern supposed to not only validating data/access but also to take some action, so I think of them as being a great fit for such scenarios.

    Other possibilities would include adding some rescue_from statements to the controller and raise the exceptions in your verifying modules even though they should use throw-catch ideally. Or you could override “process” in your controller or in ApplicationController to wrap it in a “catch” block.

    If you ask me, from the code readability point of view according to my preferences I find regular simple filters much easier to grab the whole flow quickly.

    As for the print debugging, there’s no need for much confusion, you can simply use “p” which returns the value being printed. Ex.:

    def method
    puts “method output:”
    p(statement here)
    end

  14. Another option, aside from the functional approach you’re looking for, would be to refactor your code to work more in-line with the Validator concept. As you know, validators contain an error collection which is usually displayed to a user.

    In one of my projects, I’ve created my own generic validators which are not tied to Rails, though they behave in a similar fashion. For my use case, I have a custom report language DSL which my user can validate before submitting a report job. The validation itself runs as a background job and will send the user an e-mail containing any validation messages.

  15. So I think the primary problem is you seem to be doing a lot of asking on your public interfaces. So you’ve got all these objects working together asking each other if they can do something. So you are quite right, you don’t get any good information other than yes/no.

    Taking more of a Tell Don’t Ask approach, you could instead have a useful class who’s responsibility it is to Authorize access to the episode for the user.

    AuthorizeEpisode.new(episode, user).call

    And it would be swell if that returned some kind of response that had the answer you wanted as to whether the user is authorized or not, and if not, perhaps a message why. A nice little response struct would work well here.

    Taking it a step further we can layer in some Ruby magic and get to a pattern my co-worker Brian Hughes came up with. He calls it Response State and we build a gem around the idea. (https://github.com/Originate/response_state)

    Ruby makes it so nice to have little predicate methods to ask if something is true or false, however, having the public interface on your objects hosting a whole slew of these methods leads to a lot of code that is constantly asking questions and then creating if/else structures to deal with the answer.

  16. In general, I’m not very confident that you can really do much to make debugging acceptance (integration) tests easier. When you’re troubleshooting something that consists of a long chain of links, any of which might be the cause of the failure, there’s nothing that can (generally) save you from searching the chain.

    This seems like a pretty general problem with ‘large’ tests too – think of failures in large or complex mechanical or electrical systems, or even reconciling a bank account. Sometimes there are clues as to the likely location or identity of the failure, but often only a methodical search is able to discover it.

  17. I tried to write a probing proxy, storing every method call and its result, but stucked with booleans. I think my proposition to a ruby language (https://bugs.ruby-lang.org/issues/8273) possibly could make it possible.

    Concisely, if conditionals took care of object’s #to_boolean method, one could design a proxy object behaving exactly like an underlying one. Another way to do it is a superclass NullObject which is a simple Object in any regards but conditionals where it’s falsy.

  18. https://gist.github.com/baweaver/e77f80e4a84925ff0044

    We can be slightly cheeky about it:

    is_authorized?.false_tap { |v| puts “#{current_user} was not authorized” }

    …which will always return self, but if the condition is false it’ll evaluate the code in the block. The same is true conversely for the other method in that gist, true_tap.

    Conditional_pipe is far more of just being annoyed at having to break out of line to do things.

  19. I think the “rabbit hole” metaphor is quite apt for this example because as you describe the process of traveling deeper and deeper into the chain of predicate functions to get at the solution there is this sense of falling.

    Maybe the problem here is that this whole idea of the predicate function is wrong; no, not entirely of course, just some subset of these that do not take arguments. I think about some of the core predicate operators, for instance ==, >, 1), (2 == 4).

    So in functional programming there is this concept the pure functions that essentially equates a function to a map with a 1 to 1 correlation between arguments and results (i.e. the function always returns the same exact thing given the same input). Well, there’s more to it than that but the important thing is that if a function is given no argument, then the answer should always be the same for no arguments.

    Are there predicate operators that take no arguments? The only example I can think of for a predicate operator that does not take an argument is “else”, since else will always return :true (I’m mixing apples and oranges here because operators aren’t functions but for the sake of this argument I think it’s OK to think of them that way).

    In your example, these predicate functions aren’t being passed anything. Functionally speaking they should be returning the same thing. But you may argue that you are passing something; because we’re talking objects here, we can think that the implicit argument “passed” to these predicates is always the object it self.

    OK then, so we have a way to inspect the object and get the answer? There is the crux. these predicates aren’t asking questions of self they are asking things about some other self. And when these things get chained together then it’s hard to know what thing it was that was being asked a question about.

    I haven’t really answered how to solve this in a better way; and sorry for being so hand wavy about my attempt at explaining some of what I think is important to consider. Clearly this is a hard question.

    Thanks for the post.

  20. Just a quick note on my comment above that some of the text of the second paragraph seems to be mysteriously gone, so it may not make sense. I would add it back here except that I suspect it would only get removed again.

  21. This won’t help on Ruby, but its food for thought: I’ve seen debugging environments that support “reverse debugging” (such as GDB – http://sourceware.org/gdb/wiki/ReverseDebug). You debug to the point where the error occurs, then essentially put the debugger “in reverse” and step backwards through the code until you find the part that failed. If Ruby had that, you’re debugging experience would have been quite a bit more delightful.

Comments are closed.