Iterate, Marry, Kill
In the spring of 2015, a single social media post launched Crisis Text Line’s volume into the stratosphere. The post, in which a user of Crisis Text Line described our service as some of the best help they had received in their life, was viewed over one million times in 24 hours. Over the next month, our daily texter volume jumped 60%. On staff, our reaction was a mix of “Woohoo!” (to see our service reaching more people in need) and “Eek!” (to keep up with demand, we had to scale our capacity quickly).
To scale, companies have two options: hire or build. Most not-for-profits and foundations choose hiring, believing that people make impact, while data and tech make overhead. Many not-for-profits use tech as 21st century office buildings, rented digital shells in which people can make impact. At Crisis Text Line, we see products as vehicles. They accelerate our Crisis Counselors’ abilities. They allow them to help more texters in crisis: more accurately, with less effort. Products can provide guidance, or eliminate distractions, allowing our Crisis Counselors to focus on what they do best: talking to texters in crisis.
“Products can provide guidance, or eliminate distractions.”
When we experienced that soaring volume, a staff member came up with a simple product idea to support our Crisis Counselors: sending them text reminders for their upcoming shifts. On the surface, it was an obvious win - Crisis Counselors are volunteers; they have busy lives of which crisis counseling is a small piece. Why not help them remember their weekly shift? Yet only a week after launching shift reminders, we decided to pull the product.
Pulling the product? Not that interesting: failure happens. What’s more interesting is how we knew, in less than a week, pulling the product was the right move. That was because of a product process we use at Crisis Text Line to hold products accountable. We call it Iterate, Marry, Kill.
Iterate, Marry, Kill (IMK) has three stages: (1) Define Success, (2) Measure Success, (3) Act (IMK).
“If you can’t measure it, it doesn’t exist.”
1. Define Success. IMK is built on the idea that, for products, success is quantifiable. There’s a famous quote in statistics: “If you can’t measure it, it doesn’t exist.” In other words, if you don’t know if a product was successful, you don’t know if it’s worth anything. For our reminder product, we defined success as the percentage of Crisis Counselors taking their scheduled shifts after receiving the reminder. Agreeing on a definition of success before launching a product is critical; otherwise, what I call “builder’s syndrome” - the idea that “we keep it because we built it” - wins.
2. Measure Success. For any product, we evaluate success as soon as we have enough data - usually within two weeks. In the week after launching our reminder tool, the percentage of Crisis Counselors taking their scheduled shifts fell by 1%. Oof. However, what we walked away with was accountability, and accountability leads to action.
3. Act. Data drives action. Depending on a product’s success, we can take three actions:
Iterate. The product is good, but could be better. This means we go back and build the next version of a product; there’s untapped value.
Marry. The product is perfect as is. No iteration needed.
Kill. The product failed. This was the case with our reminder product. But failure doesn’t have to be a bad thing. While successful products are gold, at Crisis Text Line, we treat failures as silver. Failures can be mined for learnings.
“While successful products are gold... we treat failures as silver.”
After killing shift reminders, we went back to the drawing board. Our next product aimed at boosting shift attendance was a winner: it drove up weekly rates by 10%. (More on that in a future post.) Ultimately, product-driven organizations measure success in cycle times: how fast can we roll out products, evaluate their success, and move on to the next build. Organizations thrive based on their cycle times. In periods of rapid growth, like we started to experience in 2015, and continue to experience now, fast product cycle times means keeping pace with the growth in demand from texters who need us in their moments of greatest crisis.
One more surprising benefit: it turns out our Crisis Counselors appreciate the speed that IMK gives our product process too. They don’t mind the failed products; what they care about is that we are constantly working to make their experience better. IMK is a product process that leads to constant improvement.
Bob Filbin
Bob Filbin is Chief Data Scientist at Crisis Text Line. He has written about using data to drive action for the Harvard Business Review and Medium, and was named one of the Chronicle of Philanthropy’s 40 Under 40 in 2016 who are making their mark in the nonprofit world.