Coaching Letter #235
Scaling, part the fifth
Hi, how are you? Thank you for subscribing to the Coaching Letter. You rock. Yes, I know, another CL about scaling, I’m sorry about that. But the previous ones have got me so much mileage and interaction with folks, and knowing how way leads on to way, I find myself committed to finishing up the series, and I’m kind of on the clock, so I’m working my way through them as fast as I can. Some Coaching Letters have more practical utility than others, and part of the desire to write these ones on scaling is that we are using these ideas to ask our partner districts to think about how they plan, and those meetings are happening right now. So to that end, Andrew got our AI partners to generate a handout on scaling based on the last four coaching letters—we’ll update it when the rest are written. If you use it, or if you have any feedback, or if you need anything else, please let us know.
You’ll remember that I recently used the term “failure modes”, with a little off-the-cuff remark about how I learned that term from ChatGPT but didn’t know much about it? Well, sometimes I forget who I’m talking to. Within minutes of posting that CL, I got an email from Marta, who wrote,
“Going back a career, failure modes began in the military and it was adopted into the engineering design and reliability process, although I am sure other fields have adopted it. It asks the “how” of failure - a part breaks, jams, etc. Not the cause (why did it break/jam?) and effect (what happens - does it matter?). In education, we can ask how will this policy/tool/system harm the stakeholders or outcome. Then you would ask why, followed by the so what?”
Super useful. So this Coaching Letter is about identifying failure modes, and since we’re talking about scaling, specifically about identifying potential failure modes as we scale up a program or practice through a process called a pre-mortem. A pre-mortem is a recipe for identifying potential problems and pitfalls before they sink your plan.
So here’s my favorite pre-mortem story. Many years ago now, we were working with a district on strategic planning. (Coincidentally, this was the same district with the superintendent I wrote about in CL #233 who didn’t understand the function of Instructional Rounds and therefore went through the motions without changing anything or learning anything new.) Anyway, my partner Roberta and I went in for a meeting with the superintendent to lay out a plan for upcoming work, and she changed everything. So Roberta and I had to go from her office to a meeting with the central office instructional team, which was probably about 15 people, with a new plan in hand and having done no prep for the meeting, because what we had done was now out the window and had flown to wherever good plans go after they die. We had already worked with this team, and a few weeks before had used a pre-mortem protocol with them regarding some aspect of the strategic plan, so they knew how we worked.
They were sat in a large square, and I explained that there was a new approach, and we needed their help in figuring out how to make it work. And without our asking, they just started telling us what could go wrong, in a very orderly, disciplined way. It was like they had the protocol in front of them. The person immediately to my right, at right angles to me, started by saying “This plan will not work because…” and then the person sitting to his right used the exact same sentence stem and added his contribution, and then they just kept going round the square as Roberta and I took notes. It was brilliant. It gave us more insight than we could ever have generated on our own, and saved us a mountain of grief. I was so impressed, and so grateful.
A pre-mortem is a tool for identifying failure modes—the ways in which something—anything—could go wrong. This gives you the opportunity to forestall the failure—if you’re smart and you handle it well. The key is to be able to leverage what you must already have demonstrated—that it is safe to tell you bad news.
The actual recipe is very simple: You come in with a plan, and you ask an assembled group with varied perspectives on the system (this is important!) to imagine that it’s a year or two from now and it’s obvious the plan failed—did not deliver on the theory of action behind it, not even close. And you ask them to tell you why it failed. And you let them tell you, without showing how you feel on your face, without getting defensive, and without even explaining anything, because what you think is an explanation appears to be, to someone listening, a rejection of the feedback or a defensive reaction. If you want people to tell you bad news, you have to ask for it, and you have to be very careful to express appreciation and follow up. People are Bayesians—if their previous experience of you is that you will do neither of those things, they are not about to start truth-telling now.
Chapter 2 of Instructional Rounds includes one of my all-time favorite lines:
In our experience, this level of optimism about the direct relationship between a policy and student learning is common, and the people farthest from the daily interactions of the instructional core are most likely to unknowingly subscribe to the “and then a miracle happens” improvement theory.
The point of this Coaching Letter is to help you to not be one of those people. I gathered the best resources on pre-mortems below. If you need anything else, please don’t hesitate to reach out. Thanks, as always, for reading the Coaching Letter, and if you have any feedback, I would love to hear it. Best, Isobel
Resources on pre-mortems
My first CL on pre-mortem was #34, well before they were on Substack. It tells the story of Gus, one of the original 5 recipients of the Coaching Letter, and how he handled a pre-mortem—his follow-up email is a model of how to handle feedback, because it is entirely focused on how the work will be better as a result of people sharing their thinking. (None of the links in that CL work any more, by the way, since our website migrated, but I’ve listed them below.)
A very elaborate pre-mortem recipe. It’s great, but it looks overwhelming. Remember that a pre-mortem is really only one key move: asking people with varied experience and points of view to tell you why something might fail.
Aggregation of useful stuff on pre-mortems Turns out that I’ve written about pre-mortems a lot. So I fed all the relevant CLs to ChatGPT and asked for a categorized list of the separate points I’ve made about pre-mortems, in my own words. It did a really good job of the list but after several passes I still can’t promise that it gave me what I’d actually written and not a paraphrase or a ChatGPT-slick version—but it’s pretty close. You may find it more useful than this CL, but be warned that it’s very dense; there are 140 points on the list.
The great psychologist Daniel Kahneman talking about pre-mortems. 3 minute video. Super useful introduction—the kind of thing you can include in an email inviting people to a pre-mortem.
Chapter 2 of Instructional Rounds. This chapter starts with a story about a meeting about implementing a new algebra course sequence. It’s written as a cautionary tale about the risks of a weak theory of action, but the way I see it, this leadership team has walked into a pre-mortem without realizing it. They didn’t ask for it, but it’s what they got. Here’s an excerpt that includes the magical thinking line quoted above.
Gary Klein, the originator of the pre-mortem, has written about it in this HBR article, but it’s behind a paywall. But I highly recommend reading one of Klein’s books—Streetlights and Shadows is a great one to start with.
The Red Team Handbook. You can get this on Amazon but the Army allows free access to the electronic version. This is great! I printed it and put it in a binder and I refer to it often. 5 stars, highly recommend.
Red Team: How to Succeed by Thinking Like the Enemy, by Micah Zenko. Very good read, a lot like books by Charles Duhigg and Dan Coyle. One for your summer reading list.