So you discovered “AB testing” and realized statistics was useful after all.  The good news is Marketo does all the hard calculations. The bad news is you have to tell it what to calculate!

Marketo enables you to compare versions of your emails to see which ones result in higher Opens, Clicks, or whatever else you want people to do when they read your email. The key metrics in most cases are Delivered, Opens, and Clicks and their corresponding percentages vs. Delivered. (This is how Marketo shows your data in the Email Performance Reports).

While Marketo enables you to do email AB tests, doing so is much different than landing page AB testing. In this section, we will walk through your options and show you how to run an actual email AB comparison.

What I talk about when I talk about AB testing

While I hear much talk about performing AB testing by marketers, not as many of us are doing these tests back at the office. Let’s change that right now, with this chapter. First, let’s review what AB testing is

1. The comparison of two or more emails to the same group of people, randomly selected;
2. The comparison of responses to one email by two or more groups of people;
3. The system designed to build, send, and report on 1 or 2.

To keep this guide simple, we will discuss comparing two different emails or two different groups’ responses. We will not explore multivariate testing although you can try to do so on your own.

Key Concepts:

• Test the Null Hypothesis (H₀) which is always that there is no difference between Open Rate or CTR for the Control and the Experiment.
• A: Control: this is the original email. You may already have data on it too.
• B: Experiment or Test: this is the new design with a single change to make it different than the Control.

To setup a test properly, you need to have two things to compare. Your comparisons must be consistent, with one change at a time to be a valid test. Here is a table you can use for ideas. Note that you can only compare one cell at a time here. That is, “Compare open rates of different subject lines at one time of day,” or “Compare open rates of one subject line at two different times of day.”

For testing ideas and visuals, take a look at Anne Holland’s famous WhichTestWon.com or the Testing with the Stars (2012) Marketo Summit presentation.

I finished the 43 page chapter on How to do AB Email Testing in Marketo the other day. Instead of showing you all the goodies, let me point you to some useful resources already out there. When the Marketing Rockstar’s Guide to Marketo launches, you will see several great examples of AB testing options fully built inside Marketo.

Until then, sign up for the Marketing Rockstar Guides email list for the latest updates.

AB Testing in Marketo Resources

• Using Random Sample [Nation]
• Using Random Sample example of splitting lists. [Nation]
• AB Email Testing Options in Marketo [Official Docs]
• Using Champion Challenger AB Email Testing (for Triggers or Engagements only) – [Official]
• Forward to a Friend Link {{system.forwardToFriendLink}}
• Calculator Spreadsheet for Statistical Significance [Avinash Kaushik]
• AB Testing FAQ –  includes a calculator! [HubSpot]
• HubSpot’s AB Testing Intro [HubSpot]
• Increase Conversions Cheat Sheet [Mixergy]
• Whichtestwon.com –
  Case Studies – Tools Guide [WhichTestWon]
• Sirius Decisions’ Email Quality and AB Testing – about the number of inquiries related to target audience.

[Note: links updated on July 29, 2017]

Email is the foundation of marketing automation, thus keeping your list active and opted-in is a critical task. New Marketo users (and even experienced ones) need Subscription Management built in to their instance.

Building a subscription management system will increase your active and marketable database as well as help you stay compliant with local privacy laws.

References to legal situations are not intended as legal advice. Seek counsel in your jurisdiction.

One of Marketo’s key advantages for the marketer is the ability to integrate and manage contact preferences for all modes of communications, marketing channels, and more. Many of my clients request help creating subscription management workflows to graduate from the universal Unsubscribe to a system where the Lead can select the type, mode, and frequency of communication from their firms.

Developing a system from scratch can be fairly simple or fairly hard. More complicated workflows will require an SFDC Administrator and possibly your web programmer, depending on your needs.

There are 7 components to building a successful subscription management system.

1. Privacy and Communication Policy
2. Subscriptions and Communication Channels
3. Database Fields and Campaigns
4. Marketo Lists
5. Marketo Form
6. Marketo Page
7. Marketo Workflows

What is Subscription Management?

Subscription Management is a replacement for the standard “unsubscribe page” that is required by most jurisdictions, and in the United States under the CANSPAM rules. Instead of letting people opt-out entirely from your list, you invite them to subscribe to the communication types and channels they want. This nicely follows the Golden Rule while ensuring you comply with the law.

“Offer subscription management to your audience: it’s how they want to be treated.”

Now take the following steps to develop your working Subscription Management page in Marketo.

1. Privacy and Communication Policy

You probably have a privacy policy and one or more “subscriptions” such as a newsletter. Perhaps you have different types of emails you send to your opted-in list. You also communicate with your prospects and clients via Phone, Email, Post, and even SMS.

You should have a policy clearly explaining to your audience how you will communicate with them, when, and for what, while providing the option for your audience to change those methods.

2. Subscriptions and Communication Channels

Which channels do you currently use to reach your audience? Are you sure those are the right ones? Make sure you create a nice grid describing the Channels and the Content or subscriptions available through those channels. You will need this understanding to program the system correctly.

3. Database Fields and Campaigns

Once you have your grid, you can then update the fields, checkboxes, and SFDC Campaigns necessary to drive the entire system. If you don’t already keep track of certain channels or subscription content, here’s your chance to upgrade your system.

4. Marketo Lists

Now create the relevant Marketo Smart Lists and Static Lists corresponding to each channel+subscription for both opted-in and opted-out Leads.

5. Marketo Form

Now create a corresponding Subscription Management Form with the proper fields and data you need to collect. Remember to make Email Address mandatory. Also make sure that your default options are Blank. It is now against the law in Canada to “pre-check” the opted-in status, so I recommend letting the Lead choose what they want and avoid pre-seeding their mind.

6. Marketo Page

Of course, you will need a Marketo landing page to place the form on. If you can, you might consider pre-filling the form with the current preferences. Make sure the confirmation page shows the selected preferences if you can.

7. Marketo Workflows

Now you need to program the logic into Marketo using the proper Data Value Changes triggers to modify SFDC Campaign Member Statuses which ensure you have a clear opted-in and opted-out list which is updated instantaneously.

How Do You Build Subscription Management in Marketo?

You might check out my other post on building subscription management in Marketo. My upcoming Subscription Management chapter will provide exquisite detail on building a system as well.

Marketo often recommends a double opt-in process where a lead subscribes to your list, then receives a follow up email asking them to click a link to confirm their subscription. This process avoids improper prank sign ups as well as demonstrates the Lead’s engagement by having them take a positive action. Creating such a system in Marketo isn’t too hard, but it isn’t obvious either. That’s why I am sharing an excerpt from the Subscription Management chapter here, just on the double opt-in process.

Download the special double opt-in system example on Additional Tips & Tricks for Subscription Management in Marketo [PDF]

And remember to sign up for future alerts!

[Updated: Jan 6, 2018 typos and bad links]

After many hours of toil, I present to you the complete Marketo Filters, Flows, and Triggers Reference Guide, a chapter of the Marketing Rockstar’s Guide to Marketo.

This highly detailed reference guide covers every single item with screenshots, examples, and suggestions so you completely understand which combination of triggers, filters, and operators will get you the right Smart List. I show you the Flow actions and how they actually work.

I even tell you which triggers and filters to completely avoid. Marketo won’t tell you this, but I’ve learned it the hard way so you don’t spend time wondering why a trigger misfired.

This section of the Marketing Rockstar’s Guide to Marketo is the largest at 74 pages.

But this is too juicy to put out in the wild. Below you will find a link to a sample of this amazing reference work. I can only show you the first 11 pages. I am confident this is such a great tool that you will want to see more, even if you think you know a lot about Marketo already.

To see more email me: marketo.guide {at] marketingrockstarguides.com. If I get enough responses, I’ll consider a special early release of just this chapter for a very special price. But you need to email me to get on this early bird list.

Here it is:

Appendix I: Marketo Filters, Flows, and Triggers Reference Guide [Slideshare]