System1 and Effie recently published "The Creative Dividend," a 122-page paper claiming to prove "how creativity and media work together to drive reliable, predictable, and repeatable" business results. Mark Ritson called it "the most important advertising thinking in 10 years" and "a book that cannot and should not be ignored." The former Global CMO of Diageo says it should be "mandatory reading for both clients and agencies." The CEO of Effie Worldwide calls it "the first analysis of its kind." It's been shared, cited, and celebrated across the industry. That's the only reason I'm writing about it. If some fringe consultancy published a paper with methodology this bad, you'd ignore it. When it captures mainstream marketing thinking and gets stamped as credible by the industry's most prominent voices, someone needs to say something. Because the methodology is an abomination, and the conclusions don't follow from the data. Let me be fair up front: the paper's general direction, that creativity matters and short-termism is a problem, is probably right. Most practitioners who've spent real money on advertising sense that already. The emphasis on consistency, distinctiveness, and emotional resonance as ingredients in effective work? Not new, but plausible. The problem isn't the conclusion. The problem is the methodology can't support the conclusion. And if the conclusion is "good creative and sufficient media drive business results," then I'd ask: did we really need 80 pages and 91 charts to arrive at something blindingly obvious? Richard Feynman had a phrase for fields that dress up the obvious in scientific clothing: "cargo cult science." The forms of rigorous inquiry without the substance. I found 21 errors in this paper. I doubt that's all of them.

## 1. The Winners-Only Database

The paper builds its "Creative Dividend" by analyzing 1,265 campaigns from the Effie Insights database. In plain English: "We wanted to know what makes marketing successful, so we only studied marketing campaigns that agencies submitted to win a success award." Imagine writing a book on "How to Survive a Plane Crash" by exclusively interviewing people who survived plane crashes. You notice they all braced for impact, so you write: "Bracing for impact guarantees survival." What you didn't measure were the hundreds of people who braced for impact and died anyway. The paper's dataset is Effie Insights' database. Every campaign in that database was submitted to the Effie Awards. Every single data point comes from a campaign that someone believed was successful enough to enter into an effectiveness competition. The thousands of campaigns that failed, that ran and did nothing, that spent the budget and moved no needle: they're not in here. That single problem is enough to bin the whole thing. Here's another 20.

## 2. Self-Reported Data Treated as Measurement

The "Business Results" that form the backbone of the analysis are self-reported by the marketers who submitted the Effie entries. Revenue growth, profit growth, market share gains, customer penetration, pricing power: all provided by the people who had a direct incentive to make their campaigns look as effective as possible. Anyone who has written an award submission knows the game. You pick the metrics that look best. You define the measurement window to capture the peak. You attribute outcomes to the campaign that might have been driven by distribution changes, pricing decisions, competitive withdrawals, or macro trends.

## 3. The 'Business Results' Everything-Bucket

The paper defines "Business Results" as a simple count of how many positive metrics a campaign reports from a list of six: Revenue, Loyalty/Retention, New Customer Penetration, Reduced Price Sensitivity, Profit, and Market Share Gain. More reported positives equals a higher Business Results score. A campaign that grew revenue but destroyed profit, lost customers, and saw market share collapse still scores "1 Business Result." The metric only counts wins and is blind to losses. There's no penalty, no netting off, no weighting.

## 4. The Cartoon Face Trap

System1 measures the "emotional response" of an ad by having a sample of viewers choose one of seven emotions after watching it, generating a Star Rating out of 5.9. That's it. 150 people clicking on cartoon faces is the foundation of the "creative quality" measurement that drives the entire paper. My emotional response to reading this methodology was despair. I'm not sure which cartoon face I'd click for that.

## 5. The Decimal Point Deception

System1 rates the emotional quality of a campaign using a "Star Rating" out of 5.9. They don't score it out of 5 or 10: they score it out of 5.9, because that specific, weird number gives the illusion of extreme scientific precision. The underlying data is 150 people selecting one of eight pictorial options on a screen. The resolution of the input cannot support the granularity of the output. It's like weighing something on a bathroom scale and recording the result to three decimal places.

## 6. The Sales Pitch Disguised as Science

System1 is a creative testing company. Its business model is selling tools that measure "creative quality." The core finding of the paper is that these exact metrics predict business outcomes. The practical recommendation: use creative measurement tools (like System1's) to improve your advertising. Research funded by the people who benefit from its finding. The paper even says on page 60: "We shared this chapter with John Kearon, who founded System1 over 25 years ago, to request a quote." The founder of the company whose tools are being validated is asked to contribute a poem celebrating the findings.

## 7. The R-Squared Smokescreen

"Creative quality and media support explain 60.1% of campaign Business Results" (R-squared = 60.1%, with 99.9% confidence). An R-squared of 60.1% means the model accounts for about 60% of the variation in the data. The "99.9% confidence" part means they're very sure the relationship isn't zero. That's all it means. With 1,265 data points, even an R-squared of 2% or 3% would clear the 99.9% confidence threshold. The confidence level just tells you the dataset is big. 40% of campaign outcomes are completely unexplained by the model. We don't know what's driving them.

## 8-13. More Methodological Failures

The paper uses hidden control variables that inflate R-squared (Kitchen Sink Math), confuses correlation with causation (Reverse Causality Trap), presents spurious correlations like Meta revenue vs. short-termism as causal (Ice Cream and Shark Attacks), discovers that award winners won awards (Gold Medal Paradox), applies CPG-specific insights to all categories (CPG Illusion), and uses absolute media spend without adjusting for company size or competitive context (Media Spend Without Relativity).

## 14-20. The Parade of Errors Continues

Campaigns don't become profitable because they run for three years—they run for three years because they're already profitable (Longevity Illusion). Observing what large brands do and treating it as mandatory for everyone is absurd (Right Foot Law). "Don't be boring" is entirely useless advice (Cost of Dull Revelation). The framework's unfalsifiable escape hatch—"you just didn't use enough Showmanship"—is the hallmark of pseudoscience. The multiplicative equation combining subjective feelings is numerology, not mathematics. And defining "Showmanship" as a checklist of features like melody and touching is like creating a formula for art.

## 21. The Grand Tautology

Even if I'm completely wrong about every single one of the previous 20 points, this one is enough on its own. Because the entire paper is a tautology. The entire definition of an advertisement is media designed to make people feel something strongly enough that they do something about it. All effective advertising is emotional. That is not a finding. That is the definition. Framing it as a research breakthrough is like publishing a paper concluding that restaurants that serve food people enjoy get more repeat customers. The interesting question—which emotions drive which actions in which categories for which audiences—the paper never touches. Instead, it spent 80 pages proving that advertising is advertising.

## The Bottom Line: Stop Reading Papers, Start Building Systems

The premise that somewhere out there, in the right dataset with the right analysis, there are general principles of advertising effectiveness that will tell you what to do is seductive but unrealistic. Booking.com runs 25,000 A/B tests a year across 43 languages and 75 countries. They don't need a paper from System1 to tell them whether their creative works. They know, because they test it, in their specific context, with their specific customers, measuring their specific outcomes. That's what "evidence-based marketing" actually looks like. Not retrospective analysis of award entries measured with the seller's own ruler. Prospective testing. In your business. With your data. As Feynman put it: "The first principle is that you must not fool yourself, and you are the easiest person to fool." Stop reading. Start testing.

The marketing effectiveness industry would like you to believe that their papers and frameworks and conferences are essential to doing better work. They're not. At best, they're interesting conjectures that might give you a hypothesis to test. At worst, they're expensive distractions from the actual work of figuring out what moves your business. Invest in your execution framework. How quickly can you test a conjecture? How reliably can you measure the result? How honest are you willing to be when the result contradicts what you expected? Build that system. Run it. Iterate. The speed at which you can test, learn, and correct is the only competitive advantage in marketing that compounds over time.