#192 | What CRO "best practice" completely let you down when you finally tested it?

Also: The Burger Menu Conversation Went Off the Rails

Follow us: Slack | LinkedIn | YouTube | Spotify | Substack

---

In this issue

• Community thread: Is the Future of CRO Fully AI-Driven?

• ICYMI - our latest episode: Your A/B Test Dropped Conversion 8% in 6 Days… Now What?

• Jobs: New job opportunities at Adobe and Google

• Save 50%: Off our Amazon Canada Bestseller, Prove It or Lose It

---

Community question of the week: What CRO “best practice” completely let you down when you finally tested it?

Experiment Nation’s Takeaways

• “Obvious wins” often aren’t. Tactics that are frequently cited as CRO fundamentals can produce flat or even negative results when applied without considering audience, product, or buying context.

• Test assumptions, not trends. No tactic is universally effective. Even personalization, a strategy many consider a CRO staple, can be a disappointing performer in some real-world situations.

The community discussion

• Rohit Rayala: Countdown clock during sale period. Always failed irrespective of the industry.

• Zonja: Sticky CTAs

• Juan Pablo: Hasn’t happened to me so far but I heard of product reviews dropping down CR

• Edd Saunders: Personalisation! (if that counts)

• Simon Clark: Adding “COMPLETELY FREE DELIVERY” near an ATB button.....Flat result, gutted

• Rommil Santiago: For me it’s the countdown, and ending prices with .99

• Gerda Thomas: but.. adding .99 at the end of a price tag is p.s.y.c.h.o.l.o.g.y

Join the conversation on our Slack

If you’re looking for a place where you can connect with other CROs either to talk shop, or simply shoot the sh*t - consider signing up to join our Slack.

Our Slack is a place to:

• Get answers in a judgement-free zone

• Share learnings

• Talk shop with other CROs

You can join by signing via this form and we will send you an invite within 48 hours.

---

ICYMI - our latest episode: The Burger Menu Conversation Went Off the Rails

In this episode, Koalatative’s Gerda Thomas and Experiment Nation’s Rommil Santiago break down CRO scenarios submitted by the community, including failed tests, executive pressure, guardrail metrics, knowledge management, developer friction, and why experimentation teams always seem to end up back in spreadsheets. Jump into the conversation below:

00:00 Burger Menu Debate: Left vs Right
06:18 Experiment Tanks Conversion by 8%
10:03 Building a CRO Analysis Tool
14:26 Managing Panic During Bad Experiments
15:21 Guardrail Metrics Explained
16:40 Using Control Limits in Experimentation
18:18 Building an Experimentation Knowledge Bank
20:05 Why You Don’t Need AI for Everything
24:31 Why Teams Always Go Back to Spreadsheets
30:41 Why Developers Hate “Throwaway Code”

Next week: Why CRO “Best Practices” Keep Failing (And AI Might Make It Worse)

---

Jobs of the Week

As a THANK YOU for being an active subscriber of Experiment Nation’s newsletter, you get access to Optimization Jobs, a CRO-focused job board with 2000+ open roles from the last 30 days alone.

From Optimization Jobs by Experiment Nation

1. Senior Web Engineer – Growth & Activation, Firefly Board
   📍 Adobe – 4 Locations

2. Product Manager – Growth
   📍 HowGood

3. CRO & Digital Experience Manager
   📍 Infinite Electronics International

4. Senior Growth Marketing Manager
   📍 Greenlight – Atlanta (Remote Friendly)

5. Sr. Office Experience Manager
   📍 Pendo – New York, New York

6. Senior AEO Growth Marketer, France
   📍 Hubspot jobs – Remote - France

To see more jobs, visit the Optimization Jobs board and explore 3000+ jobs from 400 companies from just the past 30 days that other boards may have missed.

---

Get your copy of Prove It or Lose It for $10 USD

The (mostly) no-nonsense guide to surviving experimentation program drama

Are you ready to prove the value of your Experimentation Program? In this actionable guide, Rommil Santiago distills over 15 years of experience and shares all you need to know to survive managing an Experimentation Program. Inside, you’ll learn:

• Frameworks for communicating value effectively

• Strategies to win over stakeholders

• How to build a brand for your program that lasts

Prove it or Lose it is laser-focused on delivering only the actionable advice you need. Your Experimentation Program is only as strong as the value you can demonstrate.

Get your PDF copy for 50% off today!