Testing, Testing 1-2-3
"If you keep doing what you've always done, you'll keep getting what you've always gotten."
Do you keep emailing the same subject line to supporters expecting a higher open rate? Do you mail a donor the same ask array every time expecting a higher average gift? Do you make the same social media post expecting a higher engagement? Test driving different strategies against a current approach can help you discover new ways to boost static or declining results.
So, what can you test?
Multiple pieces on a direct mail package can be tested – the outer envelope, copy, photos, colors, call to action, ask arrays, letter length, type of package and premiums are just a handful of testable items.
Just like direct mail packages, multiple items on an email campaign can be tested, including subject line, send time/day, "from" name, donation landing page, copy, photos, length and the location of buttons and links.
Segment donors and test sending different groups different packages. You can segment donors by location, recency, frequency, monetary or demographic information. Then, test different packages within a group. Or see if the same package performs better in one group than another.
Social media is a bit harder to accurately test, especially items like Facebook posts because Facebook’s algorithm shows your post to only a small percentage of your followers. And typically, those are the followers who already engage with your page regularly. Their engagement with your post determines whether or not more people will see it on their feed. You also can’t truly test against a control because you can’t segment your audience, and you can’t show different posts to the same exact group of people.
Nonetheless, you can get an idea of whether posting
during a certain time or day has better reach. Or if a type of photo or
story has higher engagement. Maybe adding a question to your post lifts your
followers’ response. If you pay for Facebook’s boost options, you can test
targeting different groups.
- Anything else you've ever wondered about or wanted to try
As long as you can create a test version, have a control and have an accurate way to measure the results, then try it. You never know if something new will outperform a long-standing control piece unless you test it.
Before you dive into testing, here are a few things to remember:
- On the test version, change ONLY one thing at a time. Shortening the subject line, sending the email at a different time and changing the donate button to green is not an accurate test. How will you know which modification on the test email caused the lift (or drop) in results?
- Compare overall results, not individual indicators. Maybe a control package resulted in a lower response rate than the test but generated a higher average gift. Or perhaps the test had a higher response rate and average gift but cost significantly more to produce and mail. Look at big picture indicators such as net income, cost per dollar raised and cost to acquire a donor to get a more accurate assessment of whether to keep a package or toss it.
- Make your tests as accurate as possible. Comparing a name label package you mailed this January to a certificate package you mailed two summers ago is not a test. Selectively choosing who receives the control and who receives the test is not accurate either. And mailing a test without a control won't work either. You can't compare apples to oranges and expect relevant results. Conducting accurate testing can be time consuming and sometimes expensive but you'll know the decisions you are making are based on correct and reliable information.
Testing can give you answers to some of your burning questions. Does shortening my subject line increase the open rate? Do I get more donations if I make the donate button bigger? Will a black and white mail package give me a higher response rate?
Creating, testing and analyzing new ideas and strategies helps you discover what works (and doesn't work) for your organization, your supporters and your community.