Last time we reviewed Nine Techniques For Testing in Marketing Automation platforms. The examples were a bit skewed to Marketo, just like today’s post. Remember, these techniques can likely be applied to any platform. Here is the process and a walkthrough of a real-life Lead Routing example from my friends at Beachhead.io.

Step 1: Create a spreadsheet of every lead combination that matters. This is your Test Plan

For example, if you are routing leads based on US State values, you’ll need 52 test leads (50 states, and 1 null, 1 that doesn’t match your list). If you route based on Fills Out Form and list import, you’ll need 104 test leads (52 per location * 2 entry points in your routing). In our case, we had 104 examples to test.

The columns:

• The columns for your CSV will look a lot like this.
• An ID Number for each lead. This is just for your needs, do not upload this column!
• A name describing the example, here State_Territory_Status. Which gives us Lead_CA_w_NotSCR for a lead from California who did not fill an Sales Contact Request.
• Same procedure with the email address: team+7_CA_w_NotSCR@beachhead.io so with only the email address you’re able to judge the accuracy of the results. (Note that you can use “+” sign to create a new unique email that goes to the same box).
• Country does not influence our routing, but is a part of the location and may be important in global routing.
• State is the key element to route leads by territory. You can see the first to leads as example of edge cases we want to monitor to make sure our routing is flawless.
• And finally your expected Lead Assignment. In your CSV, add a column to define who you expect your lead to route to. This will make it easier to review later.

Remember that when using this list of Leads to import or test, only import the Columns you need. Never import ID!

Step 2: Warn and Review Other Teams

Before going any further, you should:

Review your CSV with stakeholders. Your VP of Sales probably doesn’t think in flow charts and is more comfortable reviewing a list of examples. This is to make sure that you have covered all of the possibilities. Your inside reps are probably more aware of data flow errors than the Sales VP.

When ready to test, notify the sales team these test leads will be coming. They can safely ignore the leads marked as Company=Test – IGNORE or Email Address= X. This helps prevent salespeople from deleting or moving leads before your test cases can be complete.

Step 3: Add the Brake

Add a filter to each of your routing campaigns to require a specific email address. For example, our filter would say “contains beachhead.io”. Then every lead that enters without a beachhead.io email address will ignore this campaign.

Step 4: Run the Tests

For each case and lead, conduct the necessary tests, which could include:

• Importing the Lead
• Filling out a form with the Lead’s information
• Creating the Lead in Marketo or CRM.
• Modifying the lead’s details after creation.

Step 5: Record each Test Case’s Results against the Expected Result.

For any cases that the Expected and Actual Result are different, you will need to investigate the causes. Use Lead Detail View and Lead Field History as aides. This sheet will be called the Test Script.

Step 6: Clean Up

Once the tests are complete, you can clean up, which may include:

• Removing the Brakes
• Deleting Test Records in either or both systems
• Switching on the new campaigns.
• Turning off older or conflicting campaigns

Step 7: Migration of Older Systems or Turning Things On

Once you’ve validated your new lead routing works as expected, here are the general steps to replace an old routing program with your new routing program:

• Plan the change for the weekend, if possible. Even with testing, things can go wrong, and you’ll be better able to handle updates if the volume of new leads is lower. This also makes sure you can fix massive errors before anyone notices on Monday.
• Set aside 2 hours of uninterrupted time to make the changes, just in case.
• Create reports for potential bugs. For example, if you have five sales reps and ten total Salesforce users, your smart list might be all new leads assigned to someone who isn’t a sales rep. If you require a 2-digit state value for leads to route, then create a report of new leads this week by state. These reports will make it easier to spot issues as the routing goes live.

The migration will start on a Friday afternoon or holiday:

1. Deactivate current campaigns.
2. Activate the new campaigns.
3. Run a few additional tests.
4. Remind people that changes occurred and to send possible errors with the Lead’s Email Address directly to you and not NOT modify anything yet. Otherwise, valuable information can be lost.
5. Monitor the lead flow regularly for at least a week.

Final Thoughts on Testing in Marketing Automation Platforms

These examples and techniques are among the best practices for testing workflows in Marketo and in any other marketing automation platform (MAP). While I do encourage you to try things out in the Sandbox or a testing area of your instance, be sure that you take the time to test and test rigorously the more complex the system and the more you have at stake.

The final “Engineering Test Method” is the gold standard in testing any complex process and I urge you to do this when you build out lead lifecycles or more complex systems. What often separates the best consultants or marketing ops managers is their willingness to go all the way to avoid live errors.

More Resources on Software Testing

However much marketers like to believe they are not engineers of a sort, they are. If you are getting into some complex martech systems, it’s worth knowing more about software testing techniques. (Secret aside – I nearly became a tester at Microsoft as a first job). You may not need all of these for your daily marketing ops role, but you may find these helpful when discussing major integrations with IT folks.

• What is Software Testing?
• The “Box” Testing Methods
• History of Testing and Techniques
• Are we building it right, or the right product?
• More testing techniques.

User Acceptance Testing (UAT) is becoming more important to martech implementations. UATs are often seen in larger projects and multi-region roll outs where marketers are less familiar with the tools and were not part of the actual build. As you grow your stack, this process should be a part of your testing and finalization process.

Bonus – Join me and dozens of other amazing marketing automation nerds at the Marketo Summit in Vegas. Use this coupon by March 14: Hill200

As you gain experience with Marketo, you will build increasingly complex flows to manage leads as well as to nurture them.

Marketo, in essence, is a rules engine. You decide the rules for your system and your audience. As with all computers, the rules you decide on are executed faithfully and without question. Thus, if your rules are not properly setup, they will go ahead anyway (if they are logically correct). For example, if you set your Smart List to ANY instead of AND, you will likely bring in many more leads than you intended, possibly ruining data or worse, sending out 100,000 emails to the wrong people.

Fortunately, there are ways to build workflows and test processes to avoid disasters. If you follow these principles and any other policies your firm has, you can reduce the error rate greatly.

Technique 1: Pay Attention

When to Use: all the time

Time Involved: 1 minute

Level: All

Paying attention seems like an obvious way to avoid mistakes. It is also prone to many human biases such as “Glossing over work you just did,” and “I’ve done this a thousand times before.” Be careful and follow a few of my rules when I operate alone.

• Carefully create the smart lists. It is easy to drag, drop, and dash only to see the batch campaign send to 10 times the number you intended. When you think you are done, stop and look at the AND vs. ANY rules as well as the Counts.
• Watch your Flows – I always check these three times. Remember a Flow step will run once for every lead that goes through.
• Watch for red squiggly lines in Flow Steps and Filters. Even if it looks right, it means Marketo did not like something.
• Watch the Schedule Count – does this count match what you thought? Did you subtract the blocked email count from the total? If something seems off, STOP.
• Qualification Rules – Every vs. Once vs. something else. One of the most frequent questions on the Nation are related to this feature.
• Scheduled Time – I always schedule a run for 10 minutes in the future because it is very easy to realize that the Email Subject Line is missing 9 minutes after you press Run.

Technique 2: Review Thrice

When to Use: always

Time Involved: 1 minute

Level: Any 

I use this technique in combination with Technique 1, cycling through the steps three times…or maybe I’m a little OCD about sending emails to thousands of people.

1. Smart List x 3
2. Flow x 3
3. Schedule x 3

Technique 3: Paired Campaign Managers

When to Use: always

Time Involved: 1 minute to 1 day

Level: Any

A technique the Marketo marketing team uses is paired campaign managers. One person builds the Program, while the other prepares the creative. Then they switch to review each other’s work.

If you have the staff, I highly recommend setting up this system as it helps to avoid the human ability to ignore errors and typos after working on something for 4 hours.

You can go further and setup an entire approval process, even with just 3 to 4 team members:

1. Build Program
2. Add Creative
3. Review Creative
4. Review Program
5. Test Program
6. Approve by Director
7. Launch

The one challenge with a full blown approval process is Marketo does not have an “approval system.” It may be possible for you to break out Roles according to the process above. For example:

• Approver: can access all Marketing Activities
• Program Builder: Marketing activities, but cannot send or approve emails or Pages.
• Creative: Design Studio, Build Emails or Pages only. No Approval rights.

Technique 4: One Email, Multiple Leads

When to Use: Any system

Time Involved: 1 minute

Level: Any

This trick works on any email platform, although I tend to only use it on Gmail. You can create as many individual Leads in Marketo as you want and have them all go to the same email box.

your.email@gmail.com

your.email+test1@gmail.com

Replace “test1” with anything you want and even leverage the spreadsheet drag and populate to add more. In fact, on my email list, several people use this technique to filter my emails to a specific box: josh+marketoguides@company.com.

Send an email and you will get all of them in the same inbox.

A similar trick can be used if you happen to be the main box on a Google Apps Account – just set all emails to go to the one box and you can make up any email address you want.

Technique 5: Reset the Test Lead

When to Use: Any system

Time Involved: 1 hour or more

Level: Intermediate or above

This technique can be used on its own or as part of other techniques. If you have a Trigger or Batch that is listening for a particular value, you can test it continually using the same Test Lead.

The Test Lead should have the following components:

• An internal email address or one you own.
• Marked somewhere as a Test – IGNORE.
• Fields values as you want them to be for the Test.
• Qualification Rules set to Every Time. (otherwise, the lead goes through once regardless of test changes).

Once ready, be sure to copy the email address to a Form or notepad so you can keep using it. Then, make the change of value using any of the following steps:

• Direct Edit
• Edit in CRM
• Fill Out Form
• Change Data Value in Flow Action

Once your test is done, adjust your flow (if needed) and keep testing. To re-set your Lead, just undo the Data Value Change you made using the Direct Edit or another flow action. 

Technique 6: One More Time…

When to Use: Basic Trigger workflows, Drip campaigns, Engagement Nurture, Lifecycle Testing, Lead Routing

Time Involved: 1 minute

Level: Any

This technique is just to modify the Schedule to use a Qualification Rule of Every Time. This way you can continually run the same set of Test leads.

Technique 7: The Brake

When to Use: Basic Trigger workflows, Drip campaigns, Engagement Nurture, Lifecycle Testing, Lead Routing

Time Involved: 1 hour to 2 days

Level: Any

This is by far the best technique and it is the most simple. At the end of your Smart List, add one of these two filters:

Member of Smart List IN “Internal Test”

Email Address CONTAINS “@yourcompany.com”

Or better yet, restrict the Email Address filter to use specific test leads or a special test domain like mine.

Email Address IS “3930303@yourdomain.com”…

The Brake prevents the Trigger or Smart List from activating any other Lead. It retains the rest of the system so you can easily remove the Brake when you are done. This way you can test the full functionality and triggers without emailing or affecting real leads.

Technique 8: Brake and Clone and Wait

When to Use: Wait Steps are involved with Basic Trigger workflows, Drip campaigns, Engagement Nurture, Lifecycle Testing, Lead Routing

Time Involved: 1 hour to 2 days

Level: Any

In this version, you use the Brake and add some tweaks:

• Clone the Program or System – instead of editing the entire system you built. You can then edit the Brakes and modify as needed without losing previous work. You may end up having to duplicate your work in the original Program, or just use this one as the final.
• Edit Wait Steps to 1 second or 1 minute – when you Clone, Brake, and Wait, you need to reduce all Wait Steps to 1 minute Any Time. Otherwise, you will Wait 2 days until Tuesday for the next email to go out. It can be a bit time consuming, which is why Cloning and Testing work better.

Technique 9: Software Style Testing Process

When to Use: Lifecycle Testing, Lead Routing, Very Complex systems

Time Involved: 1-5 days

Level: Advanced

This process involves setting up leads that meet various criteria to flow through your workflow. Each time you run each lead, you should have an Expected Result and an Actual Result. Once complete, you will have a clear list of potential flaws in the workflow and possible ways to resolve them.

Do not let “software” intimidate you. The test cases you setup will likely be a bit short of what a full Engineer in Test might do, but it’s close. Here are some terms you may come across:

• Edge Case: used by engineers to discuss unlikely scenarios that could happen, but may not be worth the effort to test or fix. Be very careful that edge case leads do not bring the system to a halt.
• Test Case: this is a planned test and lead that meets certain criteria we expect to happen. For example, the Lead is entered in Form X with State=CA and Country=Canada. What do we expect will happen? Test Cases may be called “Use cases” if created before the build.
• Test Plan: The combination of Test Cases and materials to run through the system with Expected Results vs. Actual Result.

Next week, we’ll go through a walkthrough of a real-life Lead Routing example from my friends at Beachhead.io.

Bonus – Join me and dozens of other amazing marketing automation nerds at the Marketo Summit in Vegas. Use this coupon by March 14: Hill200