I’ve been thinking about what makes a world class conversion optimization organization for the past couple of days and have come up with what I think are the top 6 criteria.
I wasn’t shooting for 6 but it seems to cover all bases. I’d LOVE to hear your thoughts.
- Optimization is embedded in planning, process and corporate culture at all levels.
- Optimization efforts are prioritized based on maximum increase of revenue/goals.
- Optimization is executed for the entire end-to-end user experience across all lines of business.
- Optimization is based on analytical data, previous learnings and best practices.
- User experience is targeted to individual visitor or group.
- Optimization process itself is efficient (optimized).
In a bit more detail:
1 – Optimization is embedded in planning, process and corporate culture at all levels.
This means two things:
– There is full buy in from the executive team and every employee is on-board and understands that optimization is a commitment, not an add-on.
– All relevant internal processes take into account the opportunity to optimize. Testing is part of the standard process and budget.
2. Optimization efforts are prioritized based on maximum increase of revenue/goals.
What to test (both in terms of where on the site and which page elements) is based on where it makes the most business sense (based on numbers and research), NOT internal politics or personal opinion.
3. Optimization is executed for the entire end-to-end user experience across all site sections.
End-to-end means looking at both off-site (paid search, display, email, etc) and on-site opportunities as well as the making sure the “funnel” starts before they land on your site (ie. how does the messaging in your paid ad match the experience on the landing page).
Across all site sections applies to sites which have multiple competing goals or categories. For example product sales vs. consulting services. The goal is to maximize overall company revenue even if a large lift in one area causes a small decline in another. This also means cross section targeting.
4. Optimization is based on analytical data, non analytical user data (think personas), previous learnings and best practices.
This includes:
– Figuring out where and what to test (what the numbers are telling us)
– Visual site/page review (what is the user experience?)
– What do we know about our visitors (who are they? what makes them tick? what are they truly looking for?)
– What did we learn from previous tests? (Layout X performed better than layout Y on the shirts page).
– Are we just guessing to create challenger experiences or applying best practices (while still keeping an open mind).
5. User experience is targeted to individual visitor or group.
Serving up the same experience to all visitors will only get you so far (even if it’s optimized). I call this “lowest common denominator” optimization. Are you taking advantage of CRM type data (what did they buy in the page) and anonymous data (traffic source, search terms, geo-targeting, visit number, etc).
6. The optimization process itself is efficient (optimized).
It takes a while for the optimization process to go smoothly for all tests. Like anything new it takes a while for all of the parts to be in sync.
I can’t help but thinking a 7th bullet point would make a nicer headline (7 always sounds sexier than 6).
Any thoughts on what to add?
Thanks in advance,
Ophir
Analytics Impact 
Ophir, good article and like your way of thinking. I wonder if you adhere to each of these principles, what the longevity of this Could be? Let me provide you an example. If you were to constantly test and optimize, then segment traffic and customize the user experience for performance. Wouldn’t this eventually create even more tests for each of the new segments (testing and optimizing new user experiences) and ultimately fracture traffic volume which would create much longer tests to run their course?
Also, since consumer behavior is in constant change from outside influencers at what point do you retest to see if the same optimization in the past have the same effect in the present?
Hi Keven,
You make some excellent points!
In terms of segmentation and targeted user experiences, it really doesn’t make sense to target a single individual per-se, but use group behavior to provide a targeted experience.
How big or how small those groups are *does* depend on how much traffic you are getting.
Like any efforts to improve a business, you need to factor in the point of diminishing returns.
Also, I’m not saying every single decision should be split tested and optimized – just the important ones :)
A realistic process that somewhat addresses this issue would be to setup a testing schedule. For example, plan to launch one test every month.
This forces you to prioritize your efforts based on maximum ROI and also ensures your optimization efforts keep momentum.
I do have clients that are launching a test every week, but they didn’t start that way – they have been testing for over 3 years.
Hi Ophir, I really like your list, especially that your first point is a proper optimization culture! I know how important it is but also how difficult to establish.
I’d like to propose a seventh point: measurement of success doesn’t stop at the ‘final’ conversion. Say you’re testing an online store’s product page and the winning variation doesn’t display the store’s service hotline or any other way of contacting costumer support. Perhaps as a result this variations causes more returns and therefore reduces profit due to the cost of returns. Actually, the cost for returns might be even higher than the additional profit generated by the variation. If one doesn’t measure these ‘post-conversion activities’ one can’t say that the winning variation actually increases the store’s profit.
Obviously, merging data from the testing software with CRM data is challenging for a bunch of reasons (technical and political reasons, for example). If one wants to focus on a store’s profit it’s necessary though.
A related case is the tracking of conversions made via telephone. Say you’re testing a landing page: the control variation displays a telephone number where you’re able to order the product too, the variation doesn’t display this number. To fully judge whether the variation performed better, one has to compare the online conversions from the control variation + ‘telephone conversion’ on the one hand to the online conversions from the variation on the other hand. If one doesn’t measure the ‘telephone conversions’, one misses part of the picture.
Do you think this issue deserves its own point or should be under one of your point above?
Take a look through my new book – ‘Website Optimization: An Hour a Day’ – you will see I have a big section on this topic (http://bit.ly/bookbirth). I would add a few more bullets – having a dedicated team (critical), have weekly/quarterly reviews to review efforts and priortize, use tracking documents to learn from tests, and moving to an agile deployment process to launch tests quicker. And run contests for test ideas and frequent training sessions for employees!
Ophir, great article. Being British I always expect to see Optimisation spelt with an s and not a z.
Only thing I would add is that it should be a constant process, too many people do some work, get results, pat themselves on the back and then the results start to ebb and they revisit to try to regain the results they achieved. End-users habits evolve and so does the optimisation process.
Ophir, enjoyed your post, as I have past topics!
A few reactions that crept up while reading:
1a. part of a strong optimization culture should be to value learning-through-losing. i.e. team members should know it’s ok, even encouraged, to get a losing test result. so long as you, in Avinash’s words, “fail quickly.”
1b. the optimization culture needs to be extended to “partners” as well. trying to run tests is exceedingly difficult when the SEM/Design/Research/Technology partners aren’t on board or even aware of the efforts and goals of optimization
2. a bit troubled by your positioning of personas as “non-analytical.” I understand this is often the case, but to be fair, personas done well are based on tons of quantitative data!
Pingback: Marketing Day: May 17, 2012 | | IntoxicativeIntoxicative
Great article on CROO’s! I have one more point to add: Passionate communication of success, and scientific communication of failure to the organization. Celebrate the wins, learn from the losses, and get the organization excited about the process.
Pingback: 25 Must Read Conversion Rate Optimisation (CRO) Articles | Boom Online Marketing
CRO (Conversion Rate Optimization) saved our campaign on Adwords. Conversion tracking involves placing a cookie on a user’s computer when he/she clicks on an ad. Then, if the user clicks on your ad and reaches one of your conversion pages, the user’s browser sends the cookie to a Google server, and a small conversion tracking image is displayed on your site. When such a match is made, Google records a successful conversion for you. Over a period of 6 weeks, Simon’s team at RDM lowered my CPA (cost-per-lead) by 27.6% simply through creative split-testing of new ad copy. It’s often overlooked but is still a very important part of the overall success of your PPC marketing with Adwords.