Avdi Grimm, Code Cleric
Joseph Goguen authored Algabraic Semantics of Imperative Programs, and was professor of computing systems at Oxford. Given those credentials, one might expect him to have been firmly on the “formal methods” side of programming language research. But in an essay in Software Development and Reality Construction entitled “Denial of Error“, he wrote this:
Hello friends,
It’s been a weird week for all of us, I think. I don’t have a lot more to say about that, so I’m just going to proceed forward as usual.
I'll Trade Ya!
Hey there! Archived SIGAVDI letters are for newsletter subscribers only. All it costs to join (and unlock this post) is an email address! I'll write to you weekly-ish with a few interesting links, some updates, and some reflections on the intersection of software and life. And I'll respond to your replies! Whattya say?
And now a word from our sponsor: me! Do you like SIGAVDI? Do you enjoy my other publications like that video with Eric Normand? Consider tossing a buck or two my way on Patreon. Thanks! 🙏
What to write of my life in these strange days? As a programmer, my life is your life is our life now: sequestered in our houses, adjusting to a world in which work-from-home is no longer a perk but a mandate.
One minor-but-real risk we confront almost immediately is the mushing-together of time: what is to differentiate the days when our surroundings are the same 24/7? A first line of defense: a new appreciation for, and dedication to, the seasonal holidays. Sláinte, friends.
In The Closed World, author Paul Edwards says that there are “two major genres of historiography” when it comes to the computing revolution, but that:
…the tropes and plotlines of both genres impose requirements that lead authors to ignore or downplay phenomena outside the laboratory in the mind of the scientist. Both versions of the story explain developments in a given field solely from the perspective of actors within it. As Mahoney puts it, the authors of this “insider history… take as givens… what a more critical, outside viewer might see as choices”. There is little place in such accounts for the influence of ideologies, intersections with popular culture, or political power. Stories based on the tropes of progress and revolution are often incompatible with these more contingent forms of history.
This puts me in mind of The Californian Ideology:
Because these core workers are both a privileged part of the labour force and heirs of the radical ideas of the community media activists, the Californian Ideology simultaneously reflects the disciplines of market economics and the freedoms of hippie artisanship. This bizarre hybrid is only made possible through a nearly universal belief in technological determinism.
(Emphasis mine).
The common theme here is the belief that technological progression—and the resulting business, economic, societal, and political upheavals stemming from it—flow out more or less inevitably and mechanically from technical advances. But what if we have more agency in this process than we imagine? More importantly: who benefits from us, the technologists, adhering to this deterministic view of blind, inevitable change?
Priority one is still finding a new gig. I’m also taking advantage of the enforced down-time to up my rate of publishing, do more collaborations, and tackle a bunch of unshaven yaks on my websites.
No movement this week.
That’s it for this week. Thanks for reading, and as always, feel free to write back!
Cheers,
Avdi
Eric Normand (of LispCast and PurelyFunctional.tv) and I have been trading notes about the developer education business for years. The other day we caught up and talked about business models, subscription vs. product pricing, courseware software, and a lot more. We recorded it in hopes it might be helpful to anyone else who is in, or considering jumping into, this line of work.
Thoughts on strong opinions and terms for complexity; some booknotes from Reactive Design Patterns and Domain-Driven Design; notes from Jessica Kerr.
OK so I understand you think teams should stick to the tech stack they have the most familiarity with.
Yes, under most circumstances.
Riddle me this though: I have a freshly-assembled team. One dev is fresh off a Rails bootcamp. One is a veteran Java developer. One has spent most of their time with PHP and Laravel. And the last one is a Node/Express fan.
What is the right language and framework for this new project?
What language and framework is your team most familiar with?
I guess… most of us have worked on Java/Spring projects…
Then those are the right technologies to use.
What about Ruby and Rails, though? Aren’t you a big fan of those?
Is your team a big fan of them?
Well, no…
Then it’s not the right choice for this project.
Wait, are you saying that we should always just use what we are familiar with?
Almost always.
What’s an exception?
Some domains have strongly converged on a particular tech stack. If there is an overwhelmingly better ecosystem for the kind of work you are doing, the it may be worth it to take the productivity hit of getting up to speed on a new stack and its accompanying idiosyncrasies and idioms.
Can you give me an example?
Well, a lot of scientific work is done in Python. So the ecosystem of science tools and libraries is particularly mature, and there are a lot of people who can help you if you get stuck. Also, a lot of scientists have used some Python, so if they are your users, choosing Python may give you a leg up working with them.
What about scaling issues, though? Wouldn’t we be better off choosing Go or Elixir?
I don’t know. Would you? Do you have enough information right now to be certain that the types of scaling issues you are going to run into are going to be the type that Go or Elixir would make a crucial difference to your project’s success?
I don’t know…
Do you have confidence that your team will design a Go or Elixir system in a way that it will benefit from those language’s strengths when the time comes? What did your team’s first Java/Spring system look like, architecturally speaking?
Oof, I see your point. But… it’s just that some of the team members have been playing with Elixir lately, and they’ve been really impressed with it.
Ah! I understand now. Your team is experiencing Tech Fatigue.
What?
Your developers have been using Java and Spring for years, and they are bored. They are watching people give talks about other languages and frameworks and different approaches to problems, and they are discontent.
So you’re saying we’re just chasing a new shiny object?
Yep. And you’re hoping I’ll give you a solid technical justification to pursue that shininess.
Ouch. So should we just suck it up and use what we know?
No!
Programmers are creatives. Creatives need new media and new constraints to get their juices flowing. New languages and tools can help developers break out of their creative ruts, and regain excitement for their work.
Wait… so should we go ahead with new tech after all?
Yeah! I say go for it. But! Be honest with yourselves. You’re not doing this because you know it’s the right choice architecturally. You’re not addressing a technical deficiency. You’re addressing an enthusiasm gap.
And for developers, enthusiasm is crucial! We don’t do good work when we’re bored.
But what if it doesn’t work out after all? What if we run into unforeseen issues with these unfamiliar technologies?
There’s a good chance you will. That’s why you should time-box your experiment with new languages or frameworks. Schedule regular check-ins and retrospectives to ask: are we “getting this”?
Are we feeling competent and confident? Or are we spinning our wheels on questions about the “right” way to structure things in this unfamiliar stack?
Are we delivering stories? Or do we keep re-writing the same ones over and over?
When we get stuck, are we finding the help we need?
Are the developers on the team who were dubious of this technology warming to it? Or are they increasingly frustrated?
Set a decision date somewhere between three and six months out. Get together and decide as a team whether to move forward with this new stack, or whether to chalk it up to research and use something you understand better.
It seems like it would be pretty demoralizing to throw all that work away.
It could be. But you’ll probably find that even if you do, the break and the fresh perspectives you’ve explored will give your team some new energy that they can apply to their old familiar stack. And they’ll have a newfound appreciation for just how productive they are with the tools they are comfortable with.
Lately Jess and I have been talking about the work of software development, particularly project setup.
Some of the work that we do when is purely experimental. “What happens when I push this button?” “What sub-commands are available?” “What is possible with this library?” “Will this fix it?” That’s exploring.
Exploring can be aimless. It can also be deliberate. We can make hypotheses and prove or disprove them. “I don’t understand this framework, but I hypothesize that a change to this file will be be reflected in the UI”. This kind of code science can help us build a model: a limited theory of how software works.
This is like looking at the map and saying “I think that I’m here, and if I’m right that means when I walk two hundred yards East-South-East I’ll hit a stream”. This is science, but “science” means a lot of different things to different people. So let’s call this orienteering.
Exploring is only part of the work. Often we know approximately where we are, we know that a destination exists, and we are trying to get there. “The frob tool should be installed” “There should be customers in the database” This is traveling.
Traveling should be repeatable. It should be reproducible. Traveling should not involve a map and compass and some scribbled notes about “taking a left at the big tree”. The transition between exploring and traveling is accomplished with pathmaking.
Pathmaking can be as rudimentary as trailblazing. (Trailblazing is not as destructive as it sounds. It simply means delineating a path with “blazes”: periodic markings on trees.) This can mean writing a basic README.
Or pathmaking can be more robust, like clearing brush and shoring up a trail with stone stairs and logs for erosion management. This can take the form of of formal documentation, automation scripts, utilities, code generators, and many other conveniences.
The important thing about pathmaking is that the path exists in a known, shared, canonical location. Either the project repository (usually the best place) or a team wiki.
But sometimes we don’t make a conscious distinction between exploring and pathmaking. We get lazy and force other developers (including our future selves!) to repeat the same exploration over and over again. This is traipsing. We expect others to just traipse behind us, without any assistance.
Coding is inherently pathmaking. We figure out what needs to be done and and then we make a path that the computer can follow. Sometimes we even write it such that human readers can skim the individual steps and still see the overall journey. We call this readable code.
Setting up a new development environment for an established project often feels like exploring, but it shouldn’t. It should be traveling.
Jess has been exploring how to use Dockerized development environments to make setup repeatable. After some hesitance I think I’m on board. Computer horsepower is more of a commodity than it used to be, and being able to move a dev environment from one machine to another via checked-in configuration is a clear win.
Another, less implication of the exploring/traveling distinction: Data in development databases should be truly ephemeral and erased often. Is there data that is often useful to have? Then it should be captured in a seeds file, which is checked into the repository.
What are you doing right now? Are you exploring, pathmaking, or traveling?
Tell me if you’ve heard this one: four white guys ask a fifth white guy to join their podcast.
Being invited to Ruby Rogues in 2011, first as a guest and then as a panelist, was a thrilling moment for me. It was one of my first experiences of feeling like I’d “arrived”; like I’d achieved the acknowledgment and respect of my peers. It felt like a validation that I was competent and that I belonged. And over the following years it opened a lot of doors for me.
At the time, I didn’t think so much about how this is a privilege not everyone shares: the expectation that eventually, if you work hard, you’ll receive that kind of validation. How not everyone spends their life being implicitly told “that could be you, some day!”.
Fast-forward a few years. The Rogues panel had evolved considerably, but I had reluctantly quit because I had too many other calls on my time.
A few months after leaving, I became aware of the influence of John Sonmez on Charles Max (Chuck) Wood, the show’s founder. John, Chuck, and two other men hold weekly “mastermind” sessions where they talk about their lives and businesses, which they broadcast on a podcast called “Entreprogrammers”.
Listening to the show, I heard Chuck and the others publicly disparage, mock and plan to screw-over a friend of mine who was then working for him as a contractor. The man who took the lead in egging Chuck on: John Sonmez.
Sonmez’ creepy comments about women also stuck out to me. I remember being surprised that Chuck was so close with someone whose values seemingly clashed with (what I knew of) Chuck’s values.
For a while I also subscribed to Sonmez’ business partner Josh Earl’s newsletter. It was there I learned about some of the questionable marketing practices Sonmez was proud of. For instance, I learned about how John openly employed the tactic of directing mob pile-ons at people who upset him. I recently learned of another case of this, in which he used his mailing list to mass-downvote a less-than-stellar review of his book.
Over the years I would periodically think: at what point is Chuck going to finally get fed up with all this sleaze, and stop taking business and personal advice from this dude? Evidently the descent into full-on MRA pick-up-artist creepiness wasn’t enough. But when Sonmez recently went on a vicious, hateful tear telling black women on Twitter to “shut your mouth”, surely that was a wake-up call?
I went over to Chuck’s Twitter profile the other day, thinking I would DM him and ask him where he stood. I didn’t get very far though, because what I found were tweets defending Sonmez. Including a retweet of Robert Martin, snidely mocking everyone who was upset.
There was also a video. I watched it in full to make sure I understood Chuck’s position. He spends most of it complaining about how people aren’t being civil, and demanding that if people want to be heard they have to be nice and explain things carefully to Chuck and people like Chuck. He says he won’t defend what Sonmez wrote; but he can’t seem to bring himself to actually condemn some of the most hateful speech I’ve ever seen from a prominent developer. The whole thing comes off as petulant at the fact that people who have been held back by systemic sexism and racism for their entire lives, and are then told to “shut their mouths”, won’t behave themselves properly.
His proposed solution is to get John, a man who has publicly stated that he speaks in bad faith, together with the people he attacked for a “conversation”.
At this point, I thought back on discussions with Chuck in the past. I thought about my Rogues days, and behind-the-scenes discussions of who to invite onto the show, or not. I thought about Chuck’s tweets and other public statements over the years.
I thought about a recent case I’m aware of. In which a white guy was invited onto one of Chuck’s shows. Noting the lack of representation in the panel, the invitee proposed a woman who was more experienced than him on the same topic to go on in his place. He got silence in response.
And then I thought about silent gatekeeping. About those conferences that just happen to have stages dominated by white men year after year. And the organizers throw up their hands and say “nobody else submitted, these must be the only people available”. Even while other conferences show, over and over and over again, that they can achieve far more diverse speaker slates by widening the circles the organizers reach out to.
And then I thought about complicity. Specifically, my complicity.
I thought about the guy that everyone in a group of friends knows is a little bit creepy toward women, but they hang out with him anyway and they don’t say anything because he’s basically a good dude, ya know? Or the guy who when he gets a few drinks in him at a party likes to bring up The Bell Curve and how different races have different IQs, it’s just science, but it’s not like he’s racist racist and so we all put up with him and don’t say anything.
Chuck isn’t either of those guys. But Chuck is nice-ist. He likes people to be well-behaved. Apparently without acknowledging that being nice just happens to be easiest for the people at the top of the heap.
With his position as the owner of popular podcasts for the Ruby, JavaScript, and freelancing communities, Chuck is in a position of great gatekeeping power in who he invites to be guests, and who he invites to be hosts. It’s a particularly influential position because, in my observation, developers typically do their most avid podcast-listening early in their careers. This is the period when they are absorbing norms. It’s the period in which they are learning what a Ruby developer, or JavaScript developer, etc. looks like, and sounds like.
And me… I’ve lent a little of my legitimacy to that empire. There are people who started listening to Ruby Rogues because I was on it. And at least in the back of my mind I’ve known for a while how those shows are implicitly curated.
In a world where marginalized people can sense that things are subtly weighted against them, but can never quite put their finger on how… silence is complicity.
For years I’ve had a silent policy that I won’t go on any of his shows. I’ve turned down or ignored multiple invitations. I’m making that policy un-silent now.
I can’t magically withdraw any legitimacy I’ve lent to Chuck’s shows. But I can go on-record in saying that I reject any association between myself and those shows, other than my appearance in some of their back-catalogs. I can publicly state that after the mass panel departure two years ago, Ruby Rogues is not the same show I was once proud to appear on.
OK, enough salving of my conscience. Now for the important bit.
I went on a hike today. Along the trail there was a tree that was so completely rotted out inside that I could fit myself, with my backpack, in the open hole in its side. But remarkably, there were still leaves on its branches. The end for that tree is unavoidable, but it may continue to shade out younger trees and inhibit their growth for another decade or more.
Prominent people and resources in the developer community can feel like that tree. It can feel like “ah, this thing already exists, there is no reason to make one like it”. Many aspiring bloggers despair of what to write, because they are afraid that anything they might write about has already been said.
The truth is, just because a thing exists doesn’t mean it’s the best thing or the authoritative thing. For instance many have noted that John Sonmez’ book on soft skills and career development is… not actually very good. There are others who write far more deftly and wisely on that topic.
Monoculture promotes laziness and mediocrity. One of the reasons I am thrilled to see the software industry gradually becoming more diverse is because of how much the quality of writing and speaking about software has improved as a result.
I’m not here to try and get Chuck to change. The only way we’re going to have reliably better representation of diverse voices in software is if marginalized people are the ones creating and owning the resources. My role here is to lift up the next generation of bloggers, authors, speakers, show hosts, and screencasters.
So, a few things:
Eight years ago when I arrived on Ruby Rogues, I felt like I’d arrived. I want that feeling to be readily available to everyone who has something to say in the developer community. I want to help make sure more saplings get the sunlight they need to thrive.
I started reading Vehicles, by Valentino Braitenberg. Braitenberg asks us to consider the implications of extremely simple notional machines. For instance, the very first machine has only a single motor controlled linearly by a single sensor.
If we say that the sensor senses heat, then this machine will move faster in warm areas (seemingly trying to escape them) and slower in colder areas. If we then add small “perturbations” to its environment—say, water currents, or other objects randomly nudging it—it will appear to wander around in a rather “lifelike” manner. It will appear to have a kind of purpose, even though we know it is nothing more than sensor and motor.
Braitenberg demonstrates how adding only a few more elements to such a vehicle vastly increases the richness of its behavior.
Vehicles with only two sensors and two motors can appear to display tendencies towards an energy emitter that we might term fear, aggression, or love, if we didn’t know we were looking at a simple machine.
What if we again add some sources of random perturbations and throw a swarm of these ultra-simple machines together and allow them to also perturb each other? We get a system of extraordinarily complex emergent behavior… even though the individual elements are about as simple as we could imagine.
To restate: this system is not complicated. But it is complex.
Now consider a Chad Fowler-style microservice architecture (or Fred George-style, if you prefer). E.g. microservices so small that they fit on a single screen. We don’t rewrite or add to them; we throw them away and make new ones.
These services are simple. But throw them all together, interacting with each other and perturbed by real-world inputs, and what do we get? Emergent, unpredictable behavior and interactions.
This is a system that is not complicated. But it is complex.
Or in other words: Simplicity leads to complexity.
This isn’t a bad thing. Complex systems are rich systems. But living with them requires a graceful approach. We very quickly pass the point at which it is feasible to understand the system, let alone predict it. Instead, we need to be able to deeply observe the system, mitigate failures (but only after the fact!), note trends, and nudge it in new directions.
Recently I saw a wonderful talk about domain modeling in F#, by Scott Wlaschin. Here’s an older version of it. It was a superbly executed presentation, crystal clear and compelling. But I had a minor quibble.