On April 23, 1985, Coca-Cola Co. famously announced that it was trading in its Coca-Cola (aka Coke) soft drink’s nearly century-old secret formula for a new, sweeter one. Certain taste tests had shown that many consumers preferred the sweeter taste of Pepsi, so “New Coke” was the company’s answer to that particular problem.

The new product received a not-so-enthusiastic public reception, however. In fact, many Coke drinkers were so upset that they launched campaigns across the country to try to persuade Coca-Cola to bring back the original Coke. And that’s exactly what the company did less than three months later, on July 11 — under the new Coca-Cola Classic brand.

Many observers still consider New Coke to be one of the biggest product development blunders in history (although the product did stick around a while under the “Coke II” moniker).

New products are the lifeblood of a company, and no company intentionally sets out to launch the next “New Coke.” Instead, the new product development goals are a positive consumer reception and sales that grow year after year, leading to product longevity. By following some new-product-development best practices, dairy processors could improve their odds of attaining those goals.

Do your research

Successful new dairy product development begins with a great idea. And great ideas for new dairy products are rarely born in a vacuum. That’s why smart companies look outside their own walls first for inspiration.

“Consumer insight and research are truly at the heart of our innovation because we’re always looking to meet unmet needs in the marketplace,” explains Takoua Debeche, chief innovation officer for White Plains, N.Y.- and Broomfield, Colo.-based Danone North America.

Danone North America recently harnessed consumer insights to develop and launch an innovative product line: Oikos Blended nonfat Greek yogurt. As Debeche explains, the Greek yogurt category had been losing household penetration, among younger millennials in particular.

“To uncover why, Danone North America and Kantar conducted a massive study exploring consumers’ relationship with yogurt and found the category underperforms when it comes to providing ‘new and interesting taste experiences,’” she notes. “This inspired us to restage the Oikos line to differentiate in taste, tone, look and feel, and answers consumers’ need for an experience built around more exciting flavor and interesting taste.”

The new line boasts a creamy texture, Debeche explains, and offers 50% more fruit than the previous Oikos Strawberry, Blueberry, Peach and Cherry Greek yogurts. It’s so thick that the company calls it “forkable yogurt.”

Joe Baird, CEO of Willows, Calif.-headquartered Rumiano Cheese, noted that his company not only uses SPINS data and simple segmentation to pinpoint nutrition trends across different demographic groups, but also looks beyond the dairy segment for new product trends.

“How are consumers thinking about wellness or convenience in other aspects of their lives? We tend to identify target segments and to think about trends and growth drivers around key seasonal occasions,” he says.

For its part, Cincinnati-based Graeter’s Ice Cream doesn’t limit its research to the dairy space, either — sweet and indulgent flavors that are trending in other product areas can spur ideas, too, says Robert Graeter, vice president and one of the fourth-generation owners of the company. Graeter’s also gleans insights from market research firms such as Mintel and pays attention to flavor forecasts and flavor trends published by well-known flavor companies.

“And then we’re fortunate in that we have retail stores,” he adds. “So we have consumer contact that we can mine and receive customer input.”

State-specific dairy industry promotion and marketing boards such as the California Artisan Cheese Guild, the California Milk Marketing Board, the Vermont Cheese Guild and many more can inspire great ideas, too, suggests Felice Thorpe, former president of the California Artisan Cheese Guild and founder of Felice Foods, a consultancy for both established food brands and early-stage food companies.

“All provide ongoing education, events and communications that consolidate research from data companies like IRI, Nielsen and SPINS,” she says. “Also, IDDBA has excellent virtual education and training and networking opportunities that often highlight consumer trends.”

Dig deeper

Dairy processors might want to look to some more sophisticated methods and sources, too, to find new product inspiration.

“One key is getting more comfortable with hyper-segmentation — identifying emerging needs, not just large existing needs within key populations,” says Adam DeVito, an innovation process expert with the Glasgow Consulting Group, a food and beverage product development consultancy based in Narragansett, R.I. “This requires an agile mindset and agile systems to support your mindset.”

He recommends deep exploratory dives into the “shifting needs, wishes [and] aspirations” of millennials — who now make up most of the workforce — as well as Generation Z and even Generation Alpha (born after 2010).

“The goal should be to understand the underlying issues that are driving changes in behavior,” DeVito says, “and then seek to understand the underlying values shifts shaping those issues. It’s like peeling through layers of onion to uncover what really matters to people.”

Dairy processors also might want to study and learn from mature categories outside the dairy industry that are being “reshaped by disruptive innovation” but with little to no “change to consumer occasions or need states,” he adds, pointing to the car industry as an example.

“Category innovation was initiated and led years ago by a large player — Toyota — then real disruption really came from Tesla,” DeVito points out. “The more established players are participating but catching up. There are definitely going to be winners and others that will largely catch up.”

External research is critical, but ultimately, “inspiration comes from within,” says Adam Mueller, president of and fifth-generation butter maker at Minvera Dairy, Minerva, Ohio. He says it is important for companies to visualize their capabilities for innovation, too.

Danone North America uses a team approach here.

“In addition to extensive research, consumer focus groups, and trend tracking, our R&I (research and innovation) and marketing teams get together and utilize several different inspiration activities to inspire the teams to come up with new ideas or concepts that we can start prototyping against,” Debeche says.

Go or no-go?

Once a dairy brand comes up with what it deems a great idea for a new product, it then must determine whether it makes sense to move forward with that idea. Thorpe says a starting point is determining whether there is interest in the product on the part of retail buyers and consumers.

“You can do an informal survey with your most honest stakeholders, along with market research,” she says. “Companies also need to design a product for competitive advantage, not for uniqueness. Cheese with chia seeds might be unique, but how many consumers will actually purchase it? Finding a unique point of differentiation that will scale with enough people is an important qualifier.”

Mueller advises taking a look at the category’s data to get the growth picture for both volume and SKU expansion.

“It’s important to take time with the data to pinpoint if you are filling a regional or national SKU space” he points out. “Hyper-target your product’s most desirable region and customer, and then partner with a retail store space to get direct consumer feedback.”

Baird recommends focusing more on the development process and putting into place stage gates at each point of that process.

“The key is to begin by implementing rapid prototypes across a number of categories, then full-recipe product, then determine manufacturing feasibility as quickly as possible,” he says. “Prototypes should be tested very early with consumer panels to get an indication as you move through the process. Manufacturing feasibility at the right scale and the right price is the biggest challenge for most.”

Speaking of prototypes, Graeter’s small-batch French Pot process allows the company to make small runs of any new ice cream, Graeter explains. The company first tests any new concept internally among its leadership team, employees and flavor panels. If a product makes the cut, the company then is able to test it quickly and simply within its own retail ice cream stores.

“We track customer response and get ratings,” he says. “Depending on the customer response, we would possibly move forward to the market or not. [Some concepts] become a full-line flavor or a seasonal flavor or a monthly flavor. We’ve got different layers of products that we offer to customers based on the popularity of the flavor and the reception it gets.”

To really streamline efforts related to a go/no-go decision, DeVito advises dairy brands to evaluate and use a design thinking process that allows them to iterate new product ideas earlier in the process, when “risks and costs are low,” while receiving market feedback from the beginning.

“This early iterative process involves ‘fusing’ identified emerging needs, tied to a defined target, with early concept descriptions and early-stage, low-cost prototypes,” he explains. “We need to nurture cultures that promote a continuous flow of ideas and the means to move ideas along a design process with real feedback loops. We also must accept that nearly all new products that become significant start small.”

Danone North America takes such an approach, Debeche suggests, iterating and testing ideas throughout the product development phase. The company ensures any concept meshes with consumer needs and also helps fill a need or resolve a barrier to entry.

“Through multiple sensory and qualitative tests, as well as overall liking and purchase intent, we decide either to launch the product or continue to iterate,” she says. “As a purpose-driven organization, we believe that the food we eat and choose to nourish our families with becomes an extension of each of us and our values. If a new product meets an unmet consumer need, is backed by consumer insight, and ladders back up to an organization’s values, it’s likely one worth moving forward with.”

Avoid mistakes

Many mistakes can occur during the new product development process, so dairy companies will also want to pay attention to the most common — and take steps to avoid them. Rushing from ideation to actual product development is one of them, Debeche notes. Dairy brands need to spend sufficient time iterating in the concept phase to ensure the concept meets consumer needs.

“In order to develop an idea into a winning product, prototyping, iterating, and testing [are] essential,” she stresses. “One must consider all aspects of a product launch prior to developing the product, including packaging, communication, claims, manufacturing, pricing, retailer considerations, and more. Innovation is holistic work, and missing an aspect of the strategy or failing to consider all parts of the equation could be detrimental to the success of a new product.”

Not evaluating manufacturing feasibility early on and underestimating launch expenses for the relevant sales channels are two additional common missteps, Baird says.

“Develop strong relationships with appropriate manufacturing partners before you need them,” he advises. “In terms of launch and promotion expenses, I’d recommend gaining some experience in a smaller category before executing a major launch.”

Another common misstep is “launching too wide and too fast,” Thorpe says. She advises testing the product out at one independent retail store or even a farmer’s market.

“This is your proof of concept, and you will discover why your sales are growing or not growing; you will discover how your product holds up in distribution; you will discover if your product sells better in certain months or occasions,” she stresses. “This is all essential long-term information that will benefit your bigger launch down the road.”

Launching a new product concept that is overengineered also presents a problem, Graeter notes. It’s going to be expensive to buy and, therefore, unsustainable.

The assumption that the product will get distribution is a mistake, too, he notes.

“It’s easy to come up with an idea, concept, or formula, pay somebody to design a package for you, pay to get it contract manufactured and maybe sell it in local areas because you have connections and friends, but how do you get beyond that? That’s where most companies and products seem to fall down,” he says.

That’s why it’s necessary to work with retailers and other potential distribution partners early on in the process, Graeter says. They must believe it’s a good product, and the dairy company has to have enough data to show that consumers are willing to buy that product.

“You may have the best product in the world, but if you don’t have the right set of partners and strategy, it’s not going to work.”

Yet another misstep is not being open to true innovation, DeVito suggests, especially within the fluid milk category.

“In a declining market for cow’s milk, there is only so much efficiency you can squeeze out of an existing system,” he says. “At some point, we must innovate and diversify. Nondairy ‘milk’ sales have exploded and diversified across new states just as dairy milk has declined.”

DeVito advises dairy brands to be open to not just incremental innovation, but also to category innovation and disruptive innovation.

“If dairy processors don’t lead that innovation, others from the outside will fill the emerging demand,” he says.