Value Driven Design Case Study: Acme Streaming

Case introduction

The case study concerns a project done for a company called Acme Streaming. They provided high-quality audio streaming over the internet. They were able to stream very high-resolution and low-distortion audio music over the internet at very fast speeds. This was a substantial win over the competition that could also send high-quality audio sources over the internet but with a noticeable loss in quality to the discerning listener.

Acme Streaming invented a system that created a better audiophile music experience than the competition, but required a component installation on the user’s computer. This special requirement slowed down their sign-up process, and their sign-up wizard had a significant churn rate so that many potential users never even got to hear what the technology was capable of.

Understand phase

At the initial project kickoff, we met with the founders of the company, the CEO, the CTO, and other key engineers handling both the back office as well as the front-end sign-up wizard. They wanted this wizard redesigned.

They first explained that their service was one of a kind and, because of that, fairly lucrative. But some lesser-quality but cheaper competition did start to appear. Consequently, they were concerned about the churn rate in their sign-ups. They wanted this reduced by at least 10%. They figured that in this way, the project would easily pay for itself and continue to deliver increased value for the company.

I asked what the churn rate was, and they admitted it was about 70%. I asked why it could not be reduced more than 10%, and they explained that the company had technically driven processes in place that made the processing of new accounts very complicated and could not be altered. Given what they saw as little room for change, they wanted to immediately reduce the churn rate from 70% to something closer to 60%.

At this point in the design process, we started to reframe the issue for the client to see what larger issues were actually causing the problems they were having and whether their assumptions were accurate.

We looked at their churn rate and asked the client if the churn rate was the only thing that concerned them about the sign-up process. They admitted it was embarrassing that a company dedicated to speed was so slow to sign up for, because the process could take up to a week for all the clearances to finalize.

I also asked what percentage of these “lost users” were lost for good. They were unclear. They admitted that they were better than the competition, but when a user signed up for a service, they got attached to it because they would spend time creating their music library, including playlists and favorites that are not easy to replicate in another service. Moreover, while their sound was better than other services, they admitted their search algorithms did not work well for smaller segments like classical music and show tunes.

This started a design thinking workshop whose main focus was reframing the problem and assessing project success metrics. In the space of one hour, we quickly identified the ways the entire product was potentially losing money. Of those, we spotted issues where design could play a key role and then prioritized them. Then we grouped them into related objectives. Items that were not measurable but important ideas were kept aside.

The related objectives now had related key results added to them, making them the first draft of OKRs. The other important ideas, if they affected design, we rephrased into design principles, which were our first draft of KPIs for the project.

This helped establish baseline values for the project.

The OKRs were further refined to these two:

OKR1: Create a connection with the customer

• Key result 1: Sign-up process 1-2 days faster than current blue sky benchmark
• Key result 2: User product adoption, collected at least 50 albums after one month

OKR2: Increase sales

• Key result 1: Churn rate less than 60%
• Key result 2: Free trial conversion 50%
• Key result 3: Sales growth 50K Q2

The Design Guidelines we defined with the client were:

• The sign-up process should reflect the company vision of speed and musical elegance.
• The sign-up process should appeal to music lovers and audiophiles (which, it turned out, were not necessarily the same thing).
• The new sign-up process only uses engineering resources approved by VP Engineering (currently zero).

The company emphasized the need for the consumer to be fully provisioned before listening to music because otherwise the service would be indistinguishable from the competitors.

Their marketing lead also insisted that to get user product adoption required this “wow” experience when they heard this pristine audio stream from the service. Otherwise, without full provisioning, they would get a similar experience as any streaming service.

For example, marketing and an audiophile expert both stated they wanted the users to get the full benefit of the music from the time they first listened. For the sake of brevity, for the rest of the case study we will focus on the OKR “Increase Sales.”

Concept Phase

In this phase, we strove to come up with a concept for the sign-up flow that would address all of the objectives and key results. With some sketch concepts we took to the user, we realized that a major problem was the broken communications in the provisioning phase, which took several days and up to a week (back then).

In follow-up discussions with engineering, they explained the provisioning process took long due to required technical checks and connections that had to be done with a home server installation, and a one-time process was required to make a verified connection.

This was also necessary for their verification that the user was an actual customer through a handshake with the server instead of a password login, which was viewed by engineering and marketing as an essential differentiator. Moreover, this provisioning process was necessary to get the full advantage of the Acme Streaming service. Otherwise, it was no better than any other service.

Discussions with users as well as some stakeholders who were expert audiophiles revealed that the delay was more annoying than the advantages of delaying user access until full provisioning. We looked at the user’s OKRs, and they fell into two easily understandable objectives: Enjoy the service and be able to trust the company. Waiting for provisioning did not fit with either of these objectives. Moreover, a key result for the user was immediate access to the service they signed up for.

We quickly came up with a couple of concepts when we allowed a user “fast track” access to the service with the understanding that full quality would come in 4-5 days when fully provisioned, but in the meantime they could enjoy the service at a quality rate as good or better than rival services. Though the professional subject matter experts were a little leery, they made it clear that the point when the provisioning was completed needed to be clearly communicated.

We tested the low-fi concepts and quickly found that offering a “7-day free trial” without the server at this reduced quality for a week, in order to “get familiar with the service,” was more than acceptable to the users we tested.

We did notice there were many issues with communication at the end of the “trial,” which included ignored spam messages or alerts that weren't seen. But the simplest solution was to display the company logo, which appeared on their media player, in grayscale, and when fully provisioned, the logo would “light up” in full color indicating full functionality. This was a concept that worked similar to the way other parts of the application functioned, so it worked seamlessly.

This concept got spectacular results. So we went back to the client and wanted to revise our OKRs. A lead engineer who accompanied us on some of the concept test sessions was enthusiastic enough to help us rally the engineering team to give this top priority. With this increased knowledge from the user, we could reiterate on the OKRs based on a better expectation of the user’s behavior and their values. The project engineering scope expanded, and the OKRs were likewise raised so that success would be measured by a more realistically higher value that could be attained.

This OKR “Increase sales” was changed to the following:

OKR2: Increase sales

• Key result 1: Churn rate less than 20%
• Key result 2: Free trial conversion 75%
• Key result 3: Sales growth 250K Q2

Related User OKR: Quick access

• Key result 1: 85% quicker access to music of lower but acceptable quality
• Key result 2: Clear communication of the status of the service

Create Phase

During the specification of the concept into a detailed design, we also discovered additional factors that contributed to the churn rate. The first was that unless you were an expert audiophile, you often did not understand what you were signing up for and why. We came up with a design for more straightforward messaging with links for the audio expert.

Their front-end developer was assigned to the project to allow us to get quick feedback and quickly iterate on user feedback on the live system at the same time we were fleshing out the concept with high-fidelity prototypes, and he was also able to add a help system to the sign-up.

Moreover, it seemed that in order to get the full benefit of the improved sound, one needed to own very high-quality equipment, which not everyone could afford, even if they were an audiophile. This also led to some of the churn rate. We learned new values for both the company and the user.

The company realized their current audience was smaller than they realized and needed to expand it, first via better sign-up experience. The company also wanted to make help desk access one click away, and users wanted to feel the system was transparent and wanted immediate help desk assistance. When this idea was prototyped, we also learned that expert audio users were even willing to pay for special audio support.

Observe Phase

During the observation phase, we worked iteratively in detailing the design with the users and other stakeholders. We found some assumptions of the end users were not accurate. For example, some users were unaware of the necessity for an added server and were perfectly fine with performance that equaled other services. Even audiophiles who switched services sometimes did so for issues unrelated to the musical stream itself. The most cited problems were not being able to find an album they wanted or recurring technical bugs in the song management software.

Moreover, when shown detailed sign-up screens detailing the 3-7 day delay, users found this pointless. The drop-off rate in those tests was over 70%. The feedback we heard from users who were switching streaming services complained that they’d have no streaming music at all until the local servers were confirmed; it seemed too much trouble to switch.

Others new to the service were more likely to go to a competitor offering more instant gratification. We realized no music was worse than music with less quality. Selling them on a free interim period was an easy sell and solved the speed-in-sign-up issue.

Users liked the idea of a 7-day free trial to cover the period when they did not have access to the full streaming technology. Some added they thought they should have 7 days to also review the new service. Engineering seemed inclined to this view as they thought, given high-end audio equipment, the new service’s value would be easily noticed. And the significant cost of the service was also a factor in making it a financially safer choice for the consumer to try out.

The same flow we had tested with just one change: elimination of the waiting period for access to music, but still keeping the delay for the local server-based solutions, the churn rate almost disappeared. We could not say for certain what the new churn rate would be, but we suggested to the client that it was going to be so low that it might be better to think of a different, more challenging key result to measure success by.

Implement phase

During implementation, there was a phased approach in a continuous release cycle. The first release was the immediate access to streaming music without the local server, and user feedback was extremely positive. It turned out very few users compared this new service with their previous services as was originally feared. Moreover, some migrated from services that had lower quality streaming, so even with the “lower quality” service, most users had positive experiences. Next, development released the extended “7-day free trial”: It was called extended because they added an additional 7-day free trial when the high-quality service began. These two additions alone accounted for a 50% reduction in the churn rate and an immediate 120K bump in sales.

This positive reaction to even their server-less offering led the company to offer just the server-less offering as a reduced-cost option, offering less audiophile users an upgrade path and a competitive price. Engineering also found a way of capturing degradation over the music file and visualizing it. This visualization would also show the dips in quality that naturally occur and then the improved performance with their premium offering. This then launched an entire series of new OKRs on tiered services and feature adoption.

By the time the Reflection phase came to the project, the original “Increase sales” OKR was almost unrecognizable. Since churn rate was solved, they replaced the old OKR with “Upgrade sales.”

• Key result 1: Premium conversion rate 20%
• Key result 2: New user increased 175% (130% entry level, 40% premium)
• Key result 3: Sales growth 400K Q2

In two quarters, the entry-level service became the main money maker, while the premium service became the reason why users signed up for the free trial, regardless of whether they used the server option or not.

Conclusion

The Acme Streaming case study demonstrates the transformative power of Value Driven Design when applied systematically to real-world business challenges. What began as a modest goal to reduce churn by 10% evolved into a complete business model transformation that not only solved the original problem but uncovered entirely new revenue streams. By reframing the challenge from a narrow technical constraint to a broader user experience opportunity, the design process enabled Acme Streaming to discover that their perceived weakness (the provisioning delay) could become a competitive advantage through tiered service offerings. The methodology’s emphasis on continuous iteration and value measurement allowed the team to pivot from defensive problem-solving to offensive market expansion, ultimately creating a sustainable business model where the “compromise” solution became more profitable than the original premium offering. This exemplifies how Value Driven Design, when properly executed, can transcend its initial scope to reveal hidden business potential that might otherwise remain unexplored.