Home > User Experience > The problem with the CRO is that it looks easy; but it is not? To succeed — You need a framework. | by Tushar Purohit | Sep, 2020

The problem with the CRO is that it looks easy; but it is not? To succeed — You need a framework. | by Tushar Purohit | Sep, 2020

The problem with the CRO is that it looks easy; but it is not? To succeed — You need a framework. | by Tushar Purohit | Sep, 2020


Many of us have some digital experience quickly find problems on site assuming them as a conversion blocker. But if test ideas are implemented randomly and not follow the prioritization framework, then it can impact the major customer journey.

post of the marketing team I have worked are structured to drive traffic on the website to convert this into sales & revenue.

Once this process starts to deliver the result, organizations adopt this understanding that online business is where they spend $1 to make $2 or $3 or even $10.

But despite their excellent marketing effort, sales remain stagnant (because if it’s the case, then everyone is doing the same.)

That’s where CRO comes into picture, as the name suggests it is ongoing, data-driven process of continually discovering and quantifying the most effective experience for your customers.

  • It’s not just a process of improving CR -its a process aimed towards optimizing different element of website in and out involved in the conversion funnel (including landing pages, UX design, on-site & off-site )
  • As mentioned above, CRO helps in optimizing the website for higher effectiveness and higher conversion.
    i.e., Improve website visitors’ experience, landing pages, defining the right content, reducing the exit and bound rate.
  • Lastly, the important it helps in improving conversion rates & drives more Average order value by retaining your existing customer. i.e personalizing homepage design, interesting categories, or preferred payment methods.

The below diagram represents why conversion optimization is essential and where you can apply in your customer journey.

The conversion framework mentioned in this post will help you remove the guesswork and quantifying the most effective experience for your website visitors

  • Need to test/measure everything
  • Testing can take time
  • Not every test is a winner — Failing fast is a good thing.
  • There is no fail or pass, but it’s a process to move temporary tests onto your website product plan as its continuous data collection process.
  • Analytics should be fully integrated with the process.
  • Need lots of ideas — based on research and data
  • The process should be developed and combined with an agile & quick, and also balances in place.
  • Customers hate popups
  • What are the rules for certain design patterns? Overlays; Toaster; Alerts etc
  • Define spots and pages that marketing can update
  • Promotions may conflict and overlap.
  • Ensure all ideas are well thought through with ROI

The conversion optimization framework deploys on a hundred websites, and many industry pioneers are strongly recommending it.

Conversion research is the first step in the CRO process.

The strategic approach focused on identifying/interpreting data to find possible points of friction in a conversion funnel (sales funnel) and ultimately allows you to actions against that problem statement.

Primarily there are two steps of data gathering and analysis- Quantitative/Qualitative way.

Followed by this, you can create a master sheet with all the issues that then turn into action items along with the expected uplift:-

  1. Qualitative analysis helps you understand or establish what users like or dislike about your website, mobile app, or services?
  2. How easy is it for users to perform the core task? Would they recommend your business to others? If no, then why? What is their recommendation for improving your website?
  3. It would help if you gathered all these insights from users through surveys, usability/customer interviews, heatmap tools that will keep your finger on your audience’s pulse.

Below are the primary ways which help you to establish these:-

  • Surveys: –Helps companies listen to customers’ voices and make informed, objective decisions based on what you know your users want.
  • Information Architecture: — Leverage different research methods within the same study for greater confidence in taxonomy like navigation structure
  • Heat maps: — Helps you to understand what users want, care about, and interact with on your site by visually representing their clicks, taps, and scrolling behavior.
  • Quantitative analysis is the practice that focuses on collecting data on user behavior, understanding these numbers, and interpreting with actionable metrics.
  • In a nutshell, quantitative research helps identify specific behavioral patterns and leaks in your funnel.
  • It refers to hard facts and measurable data that you can use to quantify and uncover your visitors’ online behavior patterns.
  • Tools like Google Analytics data/Adobe analytics helps you understand the big picture of what’s happening in “every room” of your website.
  • These data will show you the ‘WHAT’ — what’s happening and where and where the leak is.

Below are the examples which you can consider as the metrics:-

  • What visitors are doing on your site: — What activities they are performing on the site, which categories they are most engage in, where do they exist?
  • Type of visitors convert the most or the least: — for example, which is the best converting channel for your website and why?
  • What is the customer preferred journey?
  • What are the Leaks in the funnel:- You can find answers like HP to Cart conversion, HP.
  • Registration Success rate & basket completion rate:
  • Revenue participation of site section:- like (Homepage, category, product page, or cart pages)
  • Types of website users: — New visitors, returning visitor, loyal customer, Registered but not purchased, Lapsed Customer
  • High entry points and exit pages?

A hypothesis is nothing but a prediction that you can create before running a test. It clearly defines what is being changed, what you believe the outcome will, and why you think?

Because the experiment will either prove or disprove your hypothesis.

With the information you have found through Quant ITive or Qualitative data analysis, you can start brainstorming the test hypothesis.

Please don’t rely on competitor’s data — you always need to complement your data findings.

Then comes the prioritization with the uplift modeling

Prioritizing tests is one the strategic approach as a strong prioritization process helps conversion optimization research with the answer: “here’s what we’ll test, in what order, and here’s why.”

  • How big a difference is that we can expect to see with the proposed change compared to the status quo?
  • Product: — Different areas of your work performed?
  • Funnel: — How each part of your funnel converts, which will help you decide of an effect you’d need to see for the new change to be worth it.
  • Technical: — How much development work is required to graduate the test?
  • How strategically important is it?
  • Does this feature support plans?
  • What is the size of the audience or action are we optimizing for?
  • Consider a retest?

Uplift modeling, also is known as incremental modeling, true lift modeling, or net-lift modeling, is a predictive modeling technique that directly models the incremental impact of a treatment (such as a direct marketing action) on an individual’s behavior. To follow, we have to apply in two ways:-

  • Power Calculation:- We can do a power calculation to determine how big of a sample size we need for our test. The point of power calculations is to find out what sample size we need for our A/B test.
  • Statistical Confidence: — How many views/users/form submissions or other interactions we need in each group to achieve the necessary power for our test.

Then we can finally start our test! Time to wait for those events to roll in.

Idealist and scoring

  • Based on the test idea hypothesis, uplift, and scoring model –we built a scoring model based on the impact on each of its one.
  • Also, each item goes through a scoring approach to understand the potential impact of executing this activity.
  • Further, it will allow you to ‘edge case’ ideas instead of focusing on high potential activities backed by data.
  • Develop dedicated meetings each month with key stakeholders to review all of these ideas

Since conversion optimization is a continuous process of data collection and learns more about the customers, that’s the reason the success of ab test activity should not be measured based on pass or fail.

But to architect the AB test in such a way through which you can capture the customer journey and enhance it for better effectiveness.

That’s the reason for designing a proper process flow is essential in the AB test which can be achieved in the following ways:-

The concept behind journey mapping is to apply maximum data insights.

  • Successful journey mapping means mastering the moment and maintaining your customers’ attention at every step in their buying journey.
  • “With journey mapping, you can evaluate the factors contributing to whether people are getting through the process you outlined for them and then determine the reasons why they are not converting
  • Kindly Note:- Before you can start developing a useful customer journey map based on insights from your data, you have to access your data in an integrated fashion.

Customer journey analysis: Each step of a customer with a process flow helps to understand the visualization better.

Functional document: — You can create a functional document outlining the customer journey, why do you think you change is important, how you are tracking your test, how big is the sample size.

A functional document is nothing but a process document that defines the problem statement and what KPI you are measuring.

Kindly request me in the comments below if you are looking for a reference for a functional doc.

Now, since we have brainstorm through data collected from Qualitative and quanttitive analysis, which then converted into a good research hypothesis with quantified goals, it’s the time to execute these campaigns.

Type of CRO campaign you can consider for

  • A/B test: — A/B test allows you to compare two or more versions of your Web site content to see which best lifts your conversions, sales or registrations.
  • Multivariate test:- With Multivariate, you can Test many elements and variations. Multivariate less requires less traffic and fewer combinations than A/B tests require.
  • Auto target:- Auto-Target removes the guesswork and serves the most tailored experience to each visitor based on his or her customer journey and profile (for ex:- the behavior of previous visitors with similar profiles.)
  • Personalization:- Personalization you can track and recommend your users as they move through your site for the sake of optimization and personalization.
  • Recommendation: Display content/products that might interest your customers based on their previous activity or other algorithms.
    Recommendations help direct customers to relevant items they might otherwise not know about.

Once you’ve decided on the type of campaign you want to run, you also need to consider how much time you need to run your test to achieve statistical significance and how much size you need to consider.

There are multiple factors you need to consider while measuring AB test performance. Like a good research hypothesis, statistical significance above 90% of performance wouldn’t be conclusive to declare the winner and push to live.

Below mention, a few of the factors will help you to put a focus on these as the risk of deciding on bad data will be minimal.

Statistical Significance

What percentage of the time users are willing to be fooled into seeing an effect by random chance –this is called significance level (& more precisely null hypothesis)

Here are the great significance calculator tools you can use

  • Use this A/B Significance Test:-To compare two ads or landing pages and determine which performed better based on conversion rate.
  • This PCC Ad Testing Tool allows you to input more variables for a more concise analysis of click-through rate and conversion rate.
  • If you want to stick to Excel, I recommend checking out this video, which gives you more data surrounding I2C metrics for your campaigns!

Post-Test Segmentation

Post-test segmentation is vital to gain insights and maximize revenue because statistical significance will only tell us the difference between variations and control but not with the users.

As users are different and what resonates with one person, it doesn’t work with another one.

Types of Segmentation Strategy you need to consider

  • ‘’What segments have true value to our business?’ (Returning visitor or new users)
  • ‘What segments are actionable?’ (Desktop v/s Mobile)
  • ‘Which are most likely to be impacted by the test we are running?’ (Category X v/s Category Y)
  • ‘What level of correction you should do in the account for the treatment of multiple observations

Validity Threat

  • One version one truth: — Always integrate with your testing tool with analytics to see if the revenue numbers match up.
  • Conduct quality assurance for every operating system and every device.
  • If you have a mobile app and a website, it is recommended to run a test separately for each device type.
  • Stop your test only when you’ve sampled the corrected sample size.
  • Data fishing into account and adjust your significance level to compensate for the number of running tests.
  • Run the test as long as necessary until your sample size has reached.
  • It is also recommended to run a test for full incremented weeks, which will help you include the data from every day of the week and every day.
  • If you’ve run any campaign during the Holiday season or any specific festival campaign, then the data you have collected is only relevant to that season itself.
  • Look at your annual data and identify anomalies (in traffic and conversions). Account for this when running your tests.

What’s next?

There are many “best practices” out there, but ultimately, you need to find out what your customers respond to and what drives your business results.

Here are three follow-up actions to get started with CRO today:

  1. Use the three formulas to start the CRO conversation.
  2. Leverage the PIE framework to help prioritize your strategy.
  3. Make CRO someone’s responsibility.

What CRO strategies does your business leverage? Please share it with us in the comments below.

Now you’ve got everything you need to double your conversion rate: data, insights, and a prioritized testing map.

Good luck!



Source link