Website personalization performance
Once you’ve added a campaign goal (or goals) to a website personalization campaign, RightMessage will automatically A/B test personalized content against the control, or default, content.
If you haven’t yet set up campaign goals, head over to our guide on it and get those set up. Without goals in place, there’s nothing to report on.
Understanding how we A/B test personalization campaigns
By default, new personalization campaigns with an attached campaign goal are tested as follows:
- 90% of traffic will have the campaign attempt to personalize content.
- 10% of traffic will be “held back” and treated as the control group.
Regardless of whether a campaign is personalizing content, RightMessage will capture segmentation and event conversion information. This allows us to compare and contrast not only how the control group converts relative to the personalized group, but also how each segment within these groups convert relative to each other.
In short, you’ll be able to see how each segment converts compared to other segments, and overall stats on how effective personalization is for your business.
When a visitor is placed into either the control (unpersonalized) or personalized cohort for a particular personalization campaign, they’ll remain in that cohort throughout the rest of their session.
This is done because some customers change radically different things – like offers and pricing – as part of their personalization, and we never want a visitor to be confused about seeing one thing promoted on one page view, and then upon returning a few minutes later seeing something entirely different.
Selecting a campaign and, optionally, a goal
On the Dashboard , click the top right card – Conversion Lift .
Ensure the time window for this campaign is set to reflect the window you want to report on. (By default, it’s the last 7 days.)
Your personalization vs. control conversion rate
This will view give you an overall understanding of how personalization is performing across all of your campaigns, along with which campaign is currently the front-runner.
A/B Testing & Conversion Tracking
RightMessage's A/B testing feature helps you measure the impact of personalization on your conversion rates. Here's how it works:
Control vs Personalized Groups
- Control Group (~x% of visitors)
- Sees default, non-personalized content
- Establishes baseline performance
- Randomly selected visitors
- Personalized Group (~y% of visitors)
- Sees personalized content
- Tests your personalization strategy
- Majority of your traffic
Key Metrics Tracked
- Impressions
- Number of times content is shown
- Tracked separately for each group
- Conversions
- Successful form submissions/clicks
- Measured per group
- Conversion Rate
- Percentage of impressions that convert
- Calculated automatically
- Conversion Lift
- Percentage improvement over control
- Shows personalization impact
- Includes statistical confidence
Understanding Your Results
- Observed Lift: Shows percentage improvement
- Statistical Confidence: Indicates reliability of results
- Confidence Interval: Range of likely true improvement
Pro Tips
- Let tests run until statistical confidence is high
- Monitor both groups for unusual patterns
- Consider seasonal or timing effects
- Test different personalization strategies
Remember: A/B testing is automatically enabled - no setup required. Just implement your personalization and RightMessage handles the testing.