Off-Brand Testing

In Brief

Off-brand testing is a technique that helps established companies run experiments while minimizing risk to their brand equity. Off-brand testing prevents skewing the customers' response (up or down) due to their brand. In a nutshell, the product or service being evaluated is presented using a no-name brand or without branding at all. Typically, these are executed as very small scale tests to get a statistically meaningful result. If it comes back positive, the data justifies a larger batch of effort.

Helps Answer

Is there merit in the idea itself, or is there interest because of high brand value?
Would our existing customers be interested in this type of product, even if they didn't know it was from our company?
Would there be alternative market segments that might be interested in this product?

Description

While the numbers vary based on product and market type as well as survey, most customers need on average 8-21 exposures to a brand message before they'd consider purchasing a product. As these numbers are typically quoted as averages, they presumably describe the behavior of the majority, not of the early adopters. With repeated exposures to a specific message, customers start to associate certain emotions or thoughts with a specific product or company. By increasing the number of exposures, a marketing team can create a familiarity with the product, to at least consider it when purchasing a product from that category. When evaluating branding initiatives, the key metric is the number of exposures a consumer has to the core message behind the brand.

In truth, most "businesses earn almost all their profits from a small number of brands — smaller than even the 80/20 rule of thumb suggests. In reality, many corporations generate 80 percent to 90 percent of their profits from fewer than 20 percent of the brands they sell, while they lose money or barely break even on many of the other brands in their portfolios," according to Nirmalya Kumar writing for the Harvard Business Review.

With off-brand testing, you are essentially trying to discover which early adopters are most likely to respond to an offer (without adding the persuasive influence of a known brand to the mix). This is trying to measure the level of passion or pain the particular group is feeling with respect to this problem or your solution. So the key unit of measurement is typically the conversion rate. If you show this product (for example on a landing page) to 1000 ideal prospects, what percent take action?

If these customers have been repeatedly exposed to the brand, particularly with positive associations, the conversion rate obtained will likely be higher than if it were not related to the brand. It may even skew results enough to give you a "false positive." You walk away from a test thinking you have a good product idea, but in fact you obtained a positive result because the customer knows and trusts your brand.

Alternatively, if a company or brand has a bad association within a particular segment, it can use off-brand testing to anonymously evaluate products or features.

Another common application of this approach is to launch an app in an app store in a similar country to your final target market. Tongue-in-cheek, this technique is referred to in Silicon Valley as "Test in Canada."

Time Commitment and Resources

Negligible, as this is done alongside other techniques mentioned in the Real Startup Book. For example, on a landing page smoke test, you'd avoid using corporate logos, color, or fonts that could hint at the identity of the experimenter.

Interpreting Results

Off-brand testing is similar to:

Single-blind studies in scientific research. The researchers know who is being experimented on and who isn't, to help remove bias that may affect the subject's perception and behavior.
Blind taste tests are usually used to compare one brand to another, albeit in a more mature product category.

Potential Biases

As this is a technique used to reduce bias, there are no major additional biases. It's worth noting that you can still go overboard with selling instead of validating, even without branding. There are a number of techniques that will influence the outcome but will not necessarily tell you about true demand. For example, heavy scarcity tactics will increase "sales," but will not help you understand the underlying demand. On the flip side, having too many options will depress response.

Field Tips

Fight the urge to throw a big bang corporate-style launch when validating an idea using off-brand testing. It works, and it gives you data you can stake your career on. - @LaunchTomorrow
There is a time for brand compliance, security, and all of the other goodies you're used to. First do a smoke test using off-brand testing - @LaunchTomorrow
Corporations do not lack ideas, but rather have a hard time deciding on what ideas to invest in. - @davidjbland https://medium.com/@davidjbland/lean-startup-comes-home-8f205993da40
Got a tip? Add a tweetable quote by emailing us: [email protected]

Case Studies

Tools

Got a tool to recommend? Add a link by emailing us: [email protected]

References

Medium - David Bland: Lean Startup Comes Home
Harvard Business School: Kill a Brand, Keep a Customer
Wikipedia: America's Test Kitchen
Got a reference? Add a link by emailing us: [email protected]

A/B Testing

Scorecards

Off-Brand Testing

Off-Brand Testing