How Do We Know What To Test?

Some of the more curious questions we get from marketers involve testing.

I’ve found this strange, in part, because I’m not sure I’ve ever once wondered why anyone would need help figuring out what experiment to run next.

A quick glance through Google shows me that there are quite a few resources like Gabriel Weinberg’s32 Fast And Cheap Marketing Tests You Should Consider Running.” There are even collections of results like the delightful “I Spent $30,000” (Justin Brooke) and this great classic on meaningful tests.

I’m not sure I’ve noticed too many places where anyone explores how you decide what marketing experimentation means for your organization. The truth is that at the end of the day, you’ve got to have those goals in place before you can effectively conduct any sort of marketing experimentation.

It may be a cliche, but if you’re letting those goals guide your work, they’re going to guide your research, too. Try and be cognizant of how this may affect what you’re looking at. Sometimes it might mean you need to look at a new technique or tool. In other situations, it may mean that you need to reevaluate your approach.

Next, you’ve got to find the big picture questions you need to have answers to. Perhaps you are testing a new design or experience and are looking to collect feedback. You may need to answer a question about product cyclicality. You might want to know who your customers are.

For example, if I really knew the customers of a client, I would have a list of customer profiles and email addresses. I could talk to each person individually, or I could make a list and buy TV ads and send postcards out until I heard from each one. I could also place ads on websites and in smartphone apps my customers frequented.

Regardless, without a list like that, I don’t have a real 100% answer to “who is your customer,” and that’s why it’s a great example of a big picture question.

Once you’ve got your questions written down, you should figure out how important each one is.

You should already have some idea of how important each question as you made this decision once before (when planning out your company’s purpose, priorities and values.)

This should be enough structure to fight even the most stubborn cases of mind-block. If not, this is a good time to explore the wilds of delightful ideas, hacks and tips that get added to the internet every day.

Pick the strategies and tactics that will lead you towards an answer to your question. If you need to prioritize further you have a few options. Two of the most straightforward: you can sort your list of tactics by resources required (time or money involved are two common methods) or you can try and figure out the most efficient way to finish the full set.

Once you’ve assembled a collection of tactics to help answer your questions sorted by priority, you should take a little time to make sure you finish your work.

This is particularly important as you are likely to discover new questions while you are conducting your research.

One of my favorite ideas was recently very well explored by Srinivas Rao who shares a tip for using a calendar to conduct research in “Why Calendars Are More Effective Than To-Do Lists.

You can use this approach to make sure you keep everything moving ahead on schedule.

This won’t always work, and there really are some problems out there that are going to take a lot more to solve than this sort of simple arrangement can allow for. That doesn’t mean it isn’t worthwhile.

I’ve found that over the last few weeks a similar approach has reduced interdepartmental friction.

One problem I’ve noticed on a lot of teams is that while the folks who work on the marketing may understand what is going on, it’s very difficult to communicate that information to individuals who haven’t had the chance to look at it.

There’s just something that gets better when you’ve put together an experimentation that everyone can understand.

Working from the foundation of shared principles means you don’t have to worry about buy-in. When the whole team knows what the experiment is at the outset, they know what sort of result to expect at the conclusion.

Leave a Reply

Your email address will not be published. Required fields are marked *