The Basics Of Conversion Rate Optimization & A/B Testing: Learn More, Earn More!

This post may contain affiliate links so I earn a commission. Please read my disclosure for more info.

This is a post by Marty, a student of Five Figure Niche Site. He’s crushing it with his own sites and has been making about $3,000 per month for a few months. 

Marty tells us about the basic of CRO and testing. That’s really the only way to understand and know if something will work for your site versus another site.

There’s a massive amount of things you’re “supposed” to do getting your affiliate site’s visitors to click and spend more money. It’s pretty much impossible to know which conventional advice really will work and what is complete hogwash.

To make matters worse, what you might not know is that some conventional advice can actually cause you to lose earnings! Sounds terrible, doesn’t it? How can you be sure you’re on the right track when making changes to your posts? What can you do?

That’s where the principles of conversion rate optimization (CRO) and A/B testing can help. Not only that, it’s a great way to get more earnings from the site traffic you’re already getting.

The great news is that you don’t have to have a huge site to take advantage of it. Even many small sites just like mine can put these simple ideas to work right away!

What is conversion rate optimization?

Conversion rate optimization (CRO) may sound super-complicated or like something you shouldn’t worry about. However, that couldn’t be further from the truth!

Let’s start with the basics:

CRO is a method of finding ways to test changes to your site & posts and discover ways to increase (optimize) earnings. That’s it in a nutshell.

We’re mainly worried about actual conversions, not just clicks. This is a very important distinction that I’ll explain later.

The scientific method approach

It’s the way that we go about doing so that’s so critical. Unlike guesswork, the principles of CRO require that we do the following, much like the “scientific method” you may remember from school or college:

  1. Create an idea (hypothesis) for a change we can test: This change (test variant) might increase affiliate clicks & earnings or it could perform worse
  2. Test the hypothesis via A/B testing
  3. Collect data
  4. Use the data to reach a statistically significant conclusion and find out if our hypothesis was good or bad: Did we improve clicks and earnings?

As Kurt Philip from the professional CRO agency Convertica taught me in his CRO Academy Pro course, what works on one site might not work on another: We have to test everything!


Anything that someone claims is “better” or “increases conversions” means almost nothing without cold, hard data. Without data-backed proof, you will never know if you’re missing out on clicks & earning improvements that could be easily gained or if you’re being misled.

A/B testing is where we find out what works and what doesn’t. In other words, just like your school science lab, it’s where we carry out testing our “science” (CRO) experiment.

How does A/B testing work? The basics explained

A diagram showing the basics of A/B testing. A testing application (one of several types: plugins, added code used with a service, or other) controls which version of a post under test is seen. Their click-through rate (CTR) or other test actions are recorded and statistical analysis is performed. For accurate CRO testing, the conversion rates are also tracked using unique test IDs.

A/B testing is actually fairly simple when it comes down to it. That’s because the real work is handled by a test application which can be one of several types:

  • A simple plugin
  • Service-provided plugin with a service provider’s data services (subscription, paid service)
  • Code added manually to posts then tracked using an external service provider.

In all cases, the principles are the same. For A/B testing we want to compare an original post (“A”) vs our test version (“B”) which has an item we want to track behavior for.

Note: When creating an A/B test setup, we duplicate the original post (A) and make ONE change. Having 2 or more changes under test introduces too many variables and they can affect each other.

Therefore, we only use one change we’d like to test at a time.

The application controls which visitors arrive at which post and also provides a way to create the test post. Visitors arriving at post version “A” aren’t aware of post version “B” and vice versa.

The “conversion” or “action” tracked by the user is chosen during test creation. Typically, this is an outbound affiliate link leading to a sales page.

Data tracking & conclusions

Data from visitors (most often the click-through rate [CTR] ) is recorded and evaluated by the application’s algorithm. Periodically, the results are updated and the site owner can see what’s happening up to that point.

After enough visitors have been tracked (usually around at least 1,000 in most cases) the algorithm will reach a fairly good confidence level (statistical confidence in the results) for the test.

Testing takes time & sufficient traffic

Testing does have a drawback: It’s definitely not a fast process.

Another thing I need to share is that it’s critical to have a sufficient amount of visitors to your test posts. More often than not you’ll need at least 1,000+ visitors before a reasonably good conclusion can be drawn. The more the better, too.

As Kurt Philip would advise, a statistical confidence level in your results is about 90%-95%, with 95% being ideal.

For a rough time estimate, divide 1,000 by the number of unique monthly visitors you have for a post you’d like to test to know how many months it will take to test that post. If a post has less than 500 per month it might be better to wait until your site has grown and has reached 10,000+ visitors per month total.

Be patient

In my test experience (with about 15K-20K visitors per month to each of my sites) it can easily take 1-2 months or more to get confident results for an A/B test. Anyone who starts a test then ends it too soon because they think their test change is getting great results is making a huge mistake!

That’s because as the data sample size increases (more visitors) the algorithm’s computations will “average out.” You’ll often find that it’s much more important to look at the big picture: The overall long term results are what matter and not just a few dips or peaks in CTR.

However, CTR tells only part of the picture.

Clicks do not always equal conversions!

As I mentioned earlier the click-through rate tells only part of the story. In fact, that’s one of the most common errors people make: They often assume that a higher CTR means more earnings.

As counter-intuitive as it may be, increased clicks do not always result in better earnings. The only way to correctly gauge a change’s impact is to track sales conversions.

Tracking conversions

Conversions can be tracked in many cases by creating additional “sub IDs” (test tracking IDs) within your affiliate account. For Amazon affiliates, this is easy as you can create quite a large number as needed.

I use the following method: For each site’s account, I create a series of tracking IDs just for this purpose with sequential numbers added to the end.

For example, if my example site’s main ID is “cathmk-02” I create the following: cathmk01-20, cathmk02-20, cathmk03-20, cathmk04-20, and so on.

To be consistent during testing I use the odd-numbered ID for the original post under test (A) and the corresponding even-numbered ID for test post B.

During the test setup, I replace the original post’s tracking IDs with the test ID and likewise for the test variant B as well.

The AAWP plugin makes it pretty easy to override the original (main) tracking ID for most affiliate links and elements. To do so, just add tracking_id=”testID-xx” within shortcodes.

Real world examples from my own sites

Now that you’ve got a better understanding of the what, whys, and hows, how about some real Amazon Affiliate website examples?

Here are 2 of my actual CRO tests and their results along with the actual conversion data also.

Example #1:

Test hypothesis: For a budget keyword buyer post, product tables with prices shown should convert better than the original without pricing.

Image #1: Test results based only on CTR rate. As you can see, the original (no pricing) actually, on the whole, resulted in more clicks to Amazon than the alternative (with pricing).

Image #2: Sales conversion data shows very little difference between the two test versions. We can conclude that the hypothesis is proven wrong.

From the images you can see here, I based test results not just on the CTR (shown in the Nelio A/B test dashboard) but actual Amazon conversion data & earnings. From the data we can see that on the whole, the test results are that the original product tables performed marginally better than tables with pricing included.

Result: The test hypothesis was not a winner. This is not a CRO optimization change I should use.

Example #2:

Test hypothesis: Using better optimized product tables in buyer keyword posts (test post B) will result in higher CTR and better conversions (earnings) vs the original tables (A).

Image #1: Implementing improved product tables resulted in a significant improvement in CTR. But what will the CRO results be? Will my test hypothesis be proven right?

Image #2: The numbers don’t lie! Great news – my hypothesis was true. The optimized tables (test variant B) proved to generate not just a higher CTR but earnings as well versus the original tables.

As you can see above, this was a great example of what’s possible with CRO methods. In this case, I used custom (optimized) product tables and wanted to prove for myself that they would help my sites generate increased clicks and most importantly, earnings.

After about 1,000 views during testing, there was a statistically significant conclusion, backed by my Amazon test tracking IDs: The optimized tables performed much better.

I got a great boost to both sites I’m still using to this day.

While an improvement like this doesn’t happen every day (often times improvements of 5-15% might be more common) it’s a great real-world example of what’s possible.

Final thoughts

To my surprise, many people completely ignore CRO and testing. If you do so, you’re leaving free money on the table and costing yourself a lot of great experience, too!

CRO isn’t hard and it’s a fantastic way to utilize the existing traffic you have to earn more money by trying new things backed by real data rather than guessing and actually losing earnings instead.

Guesswork doesn’t work. You have to test and get real data. Testing properly means following the right process and a few basic rules but it’s fairly straightforward. Its an excellent opportunity for anyone with a relatively good amount of site traffic – even small site owners like myself.

About the Author: Marty McLeod is a niche site owner who enjoys learning about affiliate marketing, SEO, and much more. He’s also a firm believer in contributing to others positively and lending encouragement where possible.

Get my exclusive Templates & Systems for free when you join my newsletter.


Featured or Mentioned on

The Doug Show is where I’ll talk to other marketers and experts, share success stories, talk about making the leap from corporate worker bee to entrepreneur.

Subscribe here: Apple PodcastsSpotify

Or go straight to the Website

Check out my YouTube Channel
  • Success Stories from readers like you.
  • Tutorials and How-Tos.
  • Real Life as an Entrepreneur after Corporate Life.

Comments on this entry are closed.

  • Excellent post Marty! I know I had some disagreements with you in FB groups regarding some of your A/B testing premises – but no disagreements with anything in this post.

    But c’mon Marty!!! Your 2nd example is nothing but a tease! You said, “In this case, I used custom (optimized) product tables…” – but you didn’t share any specifics on what exactly that entails. People who read this post will want to know.

    I will also note that it is also extremely easy to change tracking ID’s (in the shortcode or in the UI) in the AmaLinks Pro WordPress plugin. Just had to throw that in there… 😉

    • Marty McLeod

      Hi Matthew yes this similar to the information as in my Facebook post (For those not familiar, a specific test example I carried out [text links vs buttons] and posted the results & example short case study). But the principles are the same etc. 🙂

      In this article I wanted to clarify to readers and those new to CRO concepts that clicks DO NOT guarantee more conversions & earnings. That’s the important concept I learned from Kurt at Convertica, who deals with major sites & clients with tons of traffic.

      In fact in several test cases I actually got more clicks but began to see earnings drop – so it’s important to be sure by tracking to know what’s really happen.

      I used to only think the click-through rate (CTR) was the biggest think but found out from heatmaps & other stuff people are funny in how they click and/or buy (or don’t buy!)

      The answer is always there in the data.

      1. Thanks good note about the tables!

      My tables are a custom type and proprietary/I’m not able to share those publicly….however readers can see an EXTREMELY similar example with the same layout & style at Kurt’s post that got me started learning about CRO:

      (I hope Doug doesn’t mind me posting this link)

      Or they can see the tables your plugin offers, too, which has some of the same basic elements the examples above use:

      • Product thumbnail image
      • Text description
      • Button
      • Left-to-right layout on desktop/customizable
      • Mobile friendly (so important!)

      That might be a good start for beginners as well because this stuff can be overwhelming at first. An easy-to-use approach is always great to start with. 🙂

      2. Yep back when I wrote this article I didn’t know AmaLinks Pro had the tracking ID feature, so I posted a comment below already. 🙂 Just bad timing that I did so after you posted yours. Sorry!

      3. Also the test application being used is Nelio A/B, which is affordable for the average site owner. However it’s limited (it’s not an “agency” level tool) so you can’t get the high-end features or accuracy possible (like desktop vs mobile versions tracking) like with expensive tools such as VWO.

      But as they say, we all start somewhere, no? 🙂


  • Marty McLeod

    NOTE: Hey everyone here are some notes to add to/correct/supplement the article posted, as at the time of writing I had not done so:

    1. The AmaLinks Pro plugin (not just AAWP) also provides a way to override the Amazon tracking ID for test purposes if you’d like to. (Matthew Allen from AmaLinks Pro clarified this for me)

    It’s easy to do. 🙂

    2. For some cases, you may need a lot higher visitor traffic for testing. For example, if it’s a small difference in CTR/conversions it may take a long time to get a solid test conclusion and may need several thousand visitors (say up to 5K or more) to reach a confident test result.

    (Generally what I do is look for noticeable changes and not very tiny improvements like 5% etc, so I may not go to 4K-5K visitors and will move on to the next test idea.) For much bigger sites with say 100K+ visitors/month it’s a different situation.

    3. For tests that have much bigger results (let’s say for example you test a change and see an improvement like 50% or more) it’s possible to get confident results with as little as 500 visitors.

    Thanks to Doug for the opportunity to contribute! 🙂

  • Jayantabora.

    I m agree

  • Steve

    Hi Marty,

    What types of changes did you make to your tables in Example #2?

    • Marty McLeod

      Hi Steve. I think my comments above weren’t visible when you posted your question.

      In my comment above you can see a link showing specific examples. In my case I quit using my Amazon plugin’s table maker as well as another one I had used for slightly customized tables.

      I went to the same general format as in the CRO article you’ll see.

  • James

    Thanks for this Marty. 1. Please tell us more about your product tables and what do you use for your A/B testing? Do you use a simple plugin or Google Optimise?

    • Marty McLeod

      Hi James. I use Nelio A/B. It’s a plugin and connects to their backend to handle data & testing functionality needed.

      It’s very affordable ($29/month for a single-site license) but I believe you can try it for free too. Their support is very good as well.

      You can’t do much fancier stuff with it like the agencies use (and pay WAY much more for!) but it’s a great start and is easy to use.

  • Pete

    Any suggested plugins or tools to use for this?

    • Marty McLeod

      Hi Pete, yep see my comment above! 🙂

  • Evan

    Excellent post, Marty. I think one of the fascinating things is how there are very few “best practices” that work in every niche. Each audience is unique and responds to things so differently, testing is the only way to know for sure!

    Great stuff here, I’ll definitely be referring back to this.

    • Marty McLeod

      Hi Evan & thanks. Yeah, you’re definitely right there!

      That’s exactly what Kurt says as well. In fact I’ve noticed my 2nd site’s audience (more limited demographic) converts much less, so that’s a good example of how you can never make assumptions.

      Testing is the only way to know with confidence just as you said!

      Definitely learning these concepts also helps advance one’s perspective on affiliate marketing and trying to deliver what readers want in order to get more earnings.