👋 Hey, it’s András here! Every two weeks, I write about product management concepts, hot takes, and frameworks to help you build better products. Subscribe to get each article delivered to you when it’s published!

Subscribe now

In the spring of 2022, I had the opportunity to participate in Reforge’s Scaling Product Delivery program. This article will highlight a few key concepts from the materials, with relevant examples from real-life products.

I’m not aiming to provide full coverage of everything shared in the course but rather a subjective selection of unique concepts and frameworks. Also, a fair bit of warning: it’s not a short article, and it contains several different concepts. If you don’t have a focused 15-20 minutes right now, feel free to bookmark the page for later.

If you do, let’s dive in!

What Reforge and the program are about?

Reforge is a training platform that offers specialized programs for senior marketing, engineering, and product professionals. As part of a yearly subscription, members get live course participation 2-3x a year, access to all program contents, and access to the community. Reforge is somewhat similar to MasterClass, but the company provides more specialized content, the subscription is more expensive, and it includes live sessions.

Their Scaling Product Delivery course focuses on delivering big bet product initiatives. Highlighting some of the systems that make delivering major solutions easier on a scale. Things like how to work together with different stakeholders, or how to keep up an alignment between the team and senior leadership. The program is not about designing a product organization, nor how to ship features faster—although parts of the materials might be useful there.

From hills to climbing the Mount Everest

The course often compares the effort and preparedness of climbing a hill (launching a feature) to climbing Mount Everest (launching a major new solution). Different practice, time, tools, cooperation, and mindset is needed for each. And for the latter (big bet innovations), Reforge offers a four-part structure to guide product and engineering professionals on the journey that consists of the following:

Building product conviction
Fabricating a dynamic delivery plan
Execution and adoption
Forming a launch strategy

Below, I’ll explore key concepts from each section, with some own commentary and added examples of real-life product events.

The four components of scaling product delivery. Source: Reforge

But before we do that, it’s important to clarify what we mean by “big bet innovations” first.

These innovations are much bigger than small-scale functionality improvements. They can manifest as new products, new product lines, or major additions to an existing product. Twitter’s spaces, Slack’s huddles, or Amplitude’s customer data platform can be considered as such.

Building product conviction

The first section focuses on discovering and validating opportunities, so building the product conviction. No matter how efficient an engineering organization is, if the initial product idea is flawed, the company’s bottom line will notice it.

While relying on the “product sense” of the founders or early employees might work for a startup (considering it’s fueled by market & customer knowledge), it might not work for a scaled organization. In those places where product decisions are decentralized, companies need an efficient machine that discovers problems and validates solutions.

And validating the best ideas might not always be fast.

As Matt Greenberg, former VP of Engineering at Credit Karma puts it:

“Great teams know that they have to go slow to go fast. The best thing you can do to make products ship faster is to know when to slow down and make sure you get all the right information.”

But before talking about validating the solution, it’s important to know whether we’ve found a vegetable, vitamin, or painkiller-level problem.

What’s that, you ask?

Customers value both vegetables and vitamins, but they don’t address painful or short-term needs. In contrast, painkillers do.

When judging which group our problems belong to, we shouldn’t simply accept our own assumptions, but we should pressure-test them with customers.

And be warned, customers might lie to us.

Not the purposeful, trying-to-cause-harm type of lies, but rather white lies. Our customers want to be nice to us, but that means that they might avoid giving really honest feedback. So even if the proposed idea is the worst, they’ll just agree, stating that it’s “not bad”.

When validating with customers, we shouldn’t just ask whether they like the idea, but we should ask for specific commitments to verify their buy-in.

Reforge recommends asking for a soft commitment first, a light promise if they’ll use the functionality in the future. If they say “yes”, we shouldn’t stop there. Then the request for hard commitment should follow.

We could ask for a monetary commitment from customers (“Can I take your credit card details to deposit some money for the functionality?”), but that’s likely not feasible in most real-life scenarios. So instead, we could ask them to commit X number of hours in the next month. If they agree, then we’d send event invites that they need to accept.

When this exercise is complete, we should count how many committed users we had in the two rounds. If the drop after the first round is significant, it’s likely that our users just wanted to please us, but wouldn’t value the product much.

Real-life example: Tesla is clearly asking for a hard commitment. When ordering their Cybertruck pickup car, the company is asking for a $100 deposit, even though the car’s specifications are not yet finalized. A different angle of this model: this way, customers are providing an interest-free loan to the company.

The last vital takeaway from building the product conviction is that it doesn’t only include the product team.

It’s key to involve all relevant stakeholders early on: engineering, design, commercial departments, and the executive team. The organization needs to be on the same page in terms of what customer value we want to provide, the estimated business impact, and the effort required to build the thing.

Four aspects of building product conviction. Source: Reforge

And once the initial connection is established, the project stakeholders should maintain the dialogue. As misalignment can cause a late-stage blowup—more on that later.

Fabricating a dynamic delivery plan

After successfully building the product conviction across the main stakeholders, we should develop a flexible delivery plan. One that includes work estimation, sequencing, delegation, and risk mitigation for big bet initiatives.

About estimation: neither overestimation nor underestimation is good.

Overestimation can cause resistance from leadership to approve a project, while underestimation might lead to scope cuts, delays, or an understaffed team trying to achieve the impossible.

To battle this, leaders need a way to forecast and manage the work reliably. And Reforge argues that agile or waterfall are suboptimal approaches for this kind of planning. The first is too flexible and short term, and the second is too rigid and relies on early assumptions being correct.

To ensure a solid foundation for planning, product and engineering people need alignment on the customer problem to be solved, the target user profiles, and key functionalities the solution should contain.

For a successful exercise, the team has to assess the complexity first, and the following four ballpark ranges provide a good starting point to that:

Straightforward: “we know how to do this”.
Normal: “we’ve done something like this before”.
Complicated: “we’ve never done anything similar and don’t know about it”.
Very complicated: “no one has ever done something like this before”.

Out of these, big bet initiatives typically fall into the last two.

Beyond categorizing the work, there are other complexity factors to be considered: scale, magnitude, stability, and access.

Complexity factors for building bit bet initiatives. Source: Reforge

Once we’ve thought about the complexity angle, it’s time to explore work sequencing.

Reforge compares sequencing to cooking instructions when making some delicious food. In order to amaze the audience, the cook would need the right ingredients, the correct amounts, plus the right order of using those to make the dish.

Think about making a hamburger: you don’t want to fry the tomatoes, cut the raw hamburger patty to pieces, or put the sauce on the top of the bun with the seeds, right? If so, why would you make similar mistakes when building a new solution?

Beyond sequencing, setting well-understandable milestones is also important.

We shouldn’t just plan the work and hope that everything will turn out great in the end. Instead, we should continuously monitor the health of the initiative, and whether the milestones are achieved on time. If not, it’s much easier to course-correct earlier in the project.

And when a plan needs adjustment, usually, we can play with three components: scope, time, or resources.

By changing the scope, we can reduce the number of features included in the first version, reduce the feature complexity (5 colors instead of 20), reduce the quality, or reduce the number of user segments we’re targeting.
By tweaking the time, we could push the launch date further, if possible. If not, we can still reduce the time spent on quality (fewer tests, messier code…), or shift some of the work to post-launch.
Last but not least, by adding resources, we can somewhat increase the development speed. But in these cases, remember: 9 women still cannot deliver a baby in 1 month.

Beyond the things above, one more factor can increase complexity: 3rd party partners.

When working with others outside of our organization, we lose some of the process control we have on the inside. The partner might see different priorities or have other processes to tackle the same. In addition, contractual terms, technical limitations, and slow feedback cycles can all play a part in increasing our complexity.

Real-life example: Apple announced in June 2021 that digital ID cards will be available within its Wallet app with the release of iOS 15, an update that rolled out to customers in September 2021. The functionality was heavily off-forecast as it took the company until late March 2022 to announce the first state to utilize digital IDs. And it’s likely that the reliance on US government contributed.

Execution and adoption

When it comes to execution, Reforge compares that activity to driving a speedboat versus commanding a large ship.

Stakeholders who lived through many functionality improvements might be perfectly capable of controlling a speedboat where it’s quick to change directions. But it’s an entirely different game to be in charge of a heavy cruise ship that might need an entire crew to sail to a remote destination.

As with other parts of this framework, constant alignment with relevant teams and senior leadership is important for a smooth ride. The problem, the vision, the chosen direction, and the way to address obstacles all need to be aligned.

One way to view alignment is to study the Vector Math of Teams framework, developed by Hubspot’s Dharmesh Shah and Tesla’s Elon Musk.

According to this idea, people on any given team are vectors that have a magnitude and a direction. If we quantify the impact on a scale of 1 to 10, some people’s vectors might be lower, some might be higher based on their impact.

Not just the magnitude is important, but the direction too.

One nine-value vector pointing in one direction (refine existing product) can be extinguished by another nine-value vector pointing in another direction (build a new product). And then the total sum is zero, and we’ve wasted efforts.

For maximum impact, the vectors should follow each other to build momentum. One practical example is when a team builds a technical enabler solution in the first month; then another team builds on top of that in the second month.

Maximum impact: when vector magnitudes and directions are aligned, and building on top of each other. Source: Reforge

Transitioning from impact to collaboration, Reforge highlights three important gatherings that help to build transparency across the organization. All of them have a different purpose and participants:

Kick-off meeting: Setting the right context about the problem and the selected solution, communicating the whys—including customer and business value, pricing details, etc. Relevant to all key stakeholders for building a shared vision and understanding.
Product reviews: Recurring sync between the executive team and core team members building the solution. A no-blame forum for focused feedback, learnings, raising blockers, and getting additional input from the leadership.
Functional and dependency team touchpoints: A platform for core and other dependent teams to verify the current state, make decisions, communicate those to others, and take the necessary next steps.

By clarifying expectations, these occasions can build a stronger alignment with critical stakeholders, which in turn can help to avoid late-stage blowups.

The last concept in this section I found particularly useful is related to interpreting executive feedback.

Senior leaders give all sorts of feedback during the development lifecycle, but not all of them are instructions to change things. Therefore, when hearing them out, we should isolate the following feedback categories:

Ideas: “you could consider this”—not an instruction, and the person providing the idea might not even have domain knowledge in the field, they might be simply stating their personal preference.
Recommendations: “I’d recommend you to do this”—guidance from someone who might have relevant experience in the topic. Can be freely challenged if the team has more insights into the matter or sees something the executive doesn’t.
Commands: “this is what you need to do”—no room for negotiation; the team needs to follow what was said.

With this approach, even small feedback like “make the button blue” can be easily resolved. We shouldn’t be afraid to ask back whether something was a command or just an idea.

Real-life example: Another case from Apple, its AirPower wireless charger product. The device was announced in September 2017 but it never launched to general public. The device failed to be commercialized due to rumored overheating issues, and then in 2019, Apple official axed their plans to ever release it.

Launch, communication & measurement

Let’s suppose we’ve aligned our organization to deliver a great solution to a relevant problem, and we’ve mostly been on time with the initiative, meeting the major milestones. Finally, as we arrive at the product launch, we’ve reached the top of Mount Everest. But we cannot stop now; we need to get down—and that takes effort too.

Post-launch activities are one of the most underestimated parts of big bet initiatives. It’s not enough to deliver well on a great problem, but we want to be prepared for the moment of launch. And ensure that everything adds to the product value, not substracts.

The first question to address around the topic is how to launch.

There are several different approaches, but in most cases, we want to let users gradually take our product into use, not letting everyone in from the very first minute. This is to avoid unexpected bugs, issues with scalability, or use cases that we haven’t discovered fully.

Reforge breaks down the different launch strategies into three groups:

Stage by percent of users: The most common method is to let an increasing percentage of customers use the product. For others, it’s not available, or they need to join a waitlist.
Stage by customer segment: Go live with the new solution for limited customer segments only—restricting it to USA, small businesses, or heavy users. This enables a company to survey the reception of the product on a test group, and if all goes well, scale it to others. This is why Meta/Facebook preferred testing in New Zealand in the past.
Stage by technology platform: Prioritize the launch on the platform with more users, compatible API functionalities, or where it’s easier to develop the solution. For example, Android dominates in India, so skipping the iOS release is a reasonable choice for applications launching in that market. But often, the choice is whether to launch on the web, iOS, or Android.

From these strategies, companies can also opt to follow multiple at the same time.

Real-life example: Clubhouse, the then-popular live audio app, used two of the three strategies when they launched in March 2020. The product was only available on iOS initially (technology platform), and users needed to sign up for a waitlist first (percent of users) to get access.

After nailing down the launch strategy, companies often find themselves in a tight spot in the weeks leading up to the release. Should they prioritize finishing the last features they’ve planned to include, or focus on increasing the quality, thus eliminating bugs?

To answer this, it’s better to view the remaining work through three lenses: launch blockers, critical items, and non-critical items.

While bugs might be annoying, if a foundational feature needs some extra time (like logging in to the app), it’s clear what the focus should be on—as it’s a launch blocker.

Another common need for big bet initiatives are launch checklists.

As commercial pilots go through a set of standardized checklists as a 200+ passenger airplane takes off, they’re also essential in coordinating product launch events.

These checklists are built by product, engineering, customer support, marketing, and other departments participating in a launch. The goal is to define who does what at a particular time and in what order. It’s easy to recall past events when details of a new product have leaked because a retailer pulled the trigger too early.

And it’s a good reminder that these checklists are not limited to internal stakeholders—external partners, resellers, and agencies might play a part too.

Post-launch measurement is also a key part of the launch strategy, but there is something that flies more often under the radar: disaster planning.

Let’s admit it; new product launches are not always astounding successes.

In the case a new product is not meeting our expectations, there is room to act. But the action is dependent on how sensitive the problem is, and how difficult it is to solve. Reforge presents four “failure modes” in their program, corrective actions that the organization needs to take if sh*t hits the fan.

These modes are:

Optimize in place: if the flaws are not critical, don’t remove the product, instead fix the issues with additional development work.
Throttle: minimize the traffic to the solution to limit impact, and gradually roll out the improvements over time.
Temporary block: if the problem is sensitive and it’s not straightforward to fix it, take down the product temporarily until the new version is out.
Rollback: when it’s both a highly sensitive problem and a very difficult one to solve, roll back the changes, and consider rethinking the idea.

The course illustrates these options on two axes, severity and solution difficulty:

And the post-launch work should be considered as a part of the product/feature cost.

Real-life example: Slack rolled out a feature in March 2021, enabling anyone to message others as long as the sender knows the email address of the recepient. Then, after customer outcry over privacy concerns and spam, they ended up ditching the feature soon after, triggering a “rollback” situation.

And with the end of this section, we’ve reached the part of a successful product launch. However, we should still ensure that our product provides value over time to customers from this point forward.

Conclusion, remarks

In this article, we’ve covered key concepts from Reforge’s Scaling Product Delivery. We’ve learned the importance of building product conviction, ways to fabricate a dynamic delivery plan, guide the execution, and shape the launch strategy.

But at best, this was only a high-level overview of what the program has to offer, it’s not a complete or detailed guide for any of the concepts—the course has a lot more depth.

One question others ask when I mention Reforge is whether it’s worth the price.

And my answer is: it depends!

If you want to explore a specific topic (like Scaling Product Delivery) in more depth, but not looking for anything else, then the $1.995 yearly fee is way too expensive.

But if you’re looking to level up your knowledge by participating in multiple courses, want to build a wider professional network, and considering using some of the frameworks and materials provided by the company, then it justifies its price.

At last, I want to thank Andy Johns and Matt Greenberg for assembling the course, Louis Bennett for providing great facilitation for the live events, and all the speakers & participants that made the content feel alive!

Also, special thanks to the Reforge team for giving permission to use the images above.

Disclaimer: I was not incentivized by Reforge to write this article, or not affiliated with the company in any way.

Learnings From Reforge’s Scaling Product…