With structured exploration comes initiative success
What is an experiment-driven approach? It’s combining your inner kid—driven by curiosity, a need for exploration and a knowing of possibility—with your business experience and proficiency. Children possess a thrill of learning, steadfast determination, vivid imagination and sense of possibility that leads to a burning need to explore, build and soar. Rekindle your connection with your inner kid to fire up your sense of wonder and possibility to achieve that seemingly elusive victory.
A sense of wonder and possibility are needed to actualize goals. Some call this the entrepreneurial mindset.
Remember your old knapsack—packed and primed at all times—to take on any new adventure you could wrangle up? Because, as spontaneity prevailed, who knew when you might just need that compass, flashlight, magnifying glass, cape, mini shovel, notebook, pencil, net, measuring tape and chocolate bar? There were undoubtedly adventures to be had, and you absolutely needed to be equipped for ingenuity.
Optimizing decision-making pairs together good judgement with a sense of curiosity and quest for knowledge and adventure. Grab ahold of that knapsack, lift the floodgates of exploration, get the escapades rolling, the gears cranking and reach those heights you’ve dreamt of—and then some. The tangible spark of childhood comes through in experimentation, and it’s this innovative approach that can be applied to businesses to boost initiative success.
The experimentation process
1. Visualize your target and set your objectives
The first step in the experimentation approach is to pull out your binoculars and set a target on that hazy mark you can half see and half imagine in the far off land that you one day hope to rule with the resources you have or can gain. These achievable outcomes that you can keep yourself accountable to are your objectives, or put more simply, where you’d like to go.
2. Embrace what-ifs and translate them into hypotheses
Let curiosity guide you as you consider the “what ifs” of your business. Where do your honed instincts and the challenges and opportunities within your company and industry lead you to explore? Grab your pencil and fill your notebook with your multitude of assumptions, which will become your hypotheses. Who says your hypotheses have to be right? Be creative with your list of ideas; start with what you know and embrace what you don’t. What you learn along the way becomes part of the picture of what is and isn’t possible (and at what cost and return).
3. Translate hypotheses into experiments
For each hypothesis, define a logical and measurable experiment for testing the outcome and efficacy with relative ease and agility. Ask yourself what your riskiest assumption is and what the smallest experiment is that can test that assumption. Each experiment should have a goal against which you could measure the actual results of the test. Check your compass, see which experiments would point you in the right direction of reaching your objectives, and prioritize your experiments based on impact and feasibility with the time, resources and abilities you have to work with.
4. Ready, set, explore!
With your experiments in front of you, your sense of possibility alight and your feet in ready position, pick a couple of experiments, start digging to discover hidden treasure! While exploring opportunities, get your boots muddy, crawl through some tunnels, cast your net, climb some trees and embrace the elements and surprises along the way. Let each experiment run for about two weeks so you can gather enough data to discover information that will put you closer to your ultimate objectives, and don’t be afraid to pivot from where you first started.
5. Measure
An experiment-driven approach is meant to measure uncertainty. How do you know if your experiment was successful? Shine your flashlight, pull out your magnifying glass and put your measuring tape to work. What was the result of each experiment against the goals that you had set at the beginning? Did the experiment drive impact? If so, how much? Toss the duds, keep the glittering pieces of information and go back to find even bigger jewels to make your treasure even more valuable.
6. Expand your reach
While you’re out there foraging and building, consider who else you could play with to make your adventures even more exciting and grand, and experiment around new playmates and territories! If your experiments are successful and maxed out, solidify them. Make your pacts, build your community and grow stronger with the help and reach of your compadres and comrades.
Four examples of structured curiosity and experimentation
The most rewarding aspect of applying the experiment-driven approach for clients at Gaussian has been seeing the sense of wonder and possibility—followed by realization of success—come alive across different types of companies and objectives. Here are four examples of experiments in practice:
A. Omnichannel software and service company seeks to increase discovery calls and proposals
Hypothesis: Sharing product updates can help drive sales and retention
Experiment: Send product update email to current and prospective customers for the newest software release
Goal: Click rate > 10%
Result: Click rate = 46%
Conclusion: Success
A client that provides an omnichannel software and service used by top companies in gaming, entertainment and hospitality used the experiment-driven approach with the objective of meaningfully increasing the number of discovery calls and proposals per month. The company focused their efforts across initiatives, and within each initiative, they ran multiple experiments simultaneously. One such experiment was to send a product update email to current and prospective customers for the newest version of their software with the goal of a click rate of greater than 10%. Generating a click rate of 46% with their experiment, the software company realized that product updates are a topic of interest to prospects and customers and that the emails are worth duplicating and building on to continue to engage with even larger audiences (the effects of which they can measure by segment).
B. Healthcare clinics seek improved provider satisfaction, utilization and quality of care
Hypothesis: Non-clinical staff double-booking leads to longer provider working hours and lower satisfaction
Experiment: Request that all non-clinical staff not double-book patients at one particular clinic for two weeks, despite a high no-show rate, and measure workload, utilization and feedback
Goal: Improved or same provider utilization (secondary goal: with less stress on providers due to scheduling demands)
Result: Increased utilization with less stress
Conclusion: Success
A healthcare client seeking to improve provider satisfaction while maintaining utilization and quality of care across locations has also benefitted from this experiment-driven approach. An example experiment that they conducted was to request that all non-clinical staff not double-book patients at one particular clinic for two weeks, despite a high no-show rate, and to measure workload, utilization and feedback at that location. With the learnings that led to the success of that experiment, the client was able to expand the scope and impact of the experiment.
C. eRetailer seeks to increase unique monthly visitors and sales conversion rate
Hypothesis: Doubling ad spend will not impact good cost per acquisition (CPA)
Experiment: Double the shopping search budget for 1 week
Goal: Increased revenue through additional orders
Result: Increase in orders but weaker revenue results
Conclusion: Fail
An established eRetailer client is seeking to meet its specific sales goals while increasing its number of unique monthly visitors and sales conversion rate with a CPA below average order value (AOV). To test the impact of an increased ad budget on revenue through additional orders, the company doubled their shopping search budget for one week. Results were inconclusive after that week: orders increased 22%, though revenue decreased 1% week-over-week. After running the experiment for an additional two weeks, more conclusive results showed that the experiment was unsuccessful, likely due to maxing out the audience for the keywords that were being bid on (which were primarily branded keywords, so limited audience made sense). Revenue increased 53% when reducing the campaign budget back to the original level.
D. Bookstore seeks to meet aggressive 30-day net sales and weekly upload targets
Hypothesis: Inbound books of high-quality currently sold to wholesale vendors can be more profitable if sold directly online
Experiment: Hire a temp to scan value of good condition donated books that would otherwise be sold to salvage and enable uploading and selling online
Goal: Increase in per-book profitability of inbound books through online sales vs. selling in bulk to wholesale vendor
Result: Stay tuned!
A bookstore with an online and physical presence has aggressive retail and online net sales targets each month while uploading a specific amount of books each week. For each experiment, the impact on sales and uploads are tracked, as an experiment is deemed successful if it is feasible to conduct and has a measurable impact. For each successful experiment, the next logical question is how best to expand the scope of the experiment to maximize impact or how to institutionalize it.
To manage quality of books sold and inventory at physical locations, each store sells boxes of inbound books to wholesale vendors. As a current experiment, the bookstore has hired a temp for four weeks at one of its physical locations to scan each donated book of good condition to determine the overall value of such books and enable books to be uploaded online for sale and stored at a larger location with other online books. After 30 days, the value of sold books will be compared to the cost of the temp and potential revenue from wholesale vendors to determine the profitability of selling these inbound books online. If the experiment is successful, the process will be solidified, optimized and expanded across locations.
Stay iterative and nimble
Track progress, incorporate learnings, refine hypotheses
To maximize time to impact, maintain multiple experiments with measurable results running at the same time. Continue to let your curiosity and quest for knowledge guide you as you push your limits. The process of uncovering surprises is very exciting! Especially when combined, experiment learnings are critical to your path to success.
If your experiment worked, go bigger this time! Does your ROI increase by the same magnitude or exponentially? Or does your ROI stay the same or decrease? Capture these insights to develop an understanding of your limits. Identifying buttons to push (another trait that children have and exercise often!) can reveal answers that can be applied more broadly and together for amplified affect.
If your experiment didn’t work, switch gears. An understanding of limitations is critical before allocating valuable time and resources toward a dead-end initiative.
Provide structure while uncovering opportunities
As you’ll be running dozens of experiments at once, having a robust tracking tool, such as Sapium (https://sapium.io/) or JIRA, will be key to providing structure while uncovering opportunities upon which you can continue to build.
Accumulating learnings result in significant impact
The thrills of childhood can seem out of reach to life as an adult, but there’s no reason to leave this joie de vivre behind. Take little steps, but take them boldly and with frequency. We were born to experiment, because experimenting is living and evolving and thriving and reaching one’s full potential. As you start making strides, strap on your cape: little steps pack a punch and can provide big answers; combined, learnings from experiments can lead to explosive results.
Oh, and that chocolate bar at the bottom of your knapsack? Toss it to the kid struggling through the same tasks, one day same as the next. With an experiment-driven approach, the reward’s much sweeter in the end.
Photo by Rene Bernal.
