Interaction
When the effect of one factor on a response is dependent upon the effects of other factors on that same response.
Billy’s Explanation:
I think of interactions as when you get unexpected results from two or more factors based on how they perform elsewhere. It’s difficult to correctly define interactions without explaining it, so let me use an example.
Say we have a test with 2 buttons and 2 headlines. Below is a table of experiments and their conversion rates (experiment 1 has the “Best MP3 Player!” headline and “I want it” button with a 4% conversion rate.)
What would you expect the conversion rate of experiment 2 to be? I’ve edited a diagram made by Widemile’s Chief Scientist, Vladimir Brayman, to help us out.
Let’s walk through this chart. The boxes show the conversion rates and each chart represents one of the levels of Factor 2 (”Buy Now” and “I want it.”) The chart essentially tells you the same information as the table. The top chart represents Experiments 3 and 4 with the “Buy Now” button paired with “Best MP3 Player!” on the left and “Listen to music anywhere” on the right. They have conversion rates of 5% and 6% respectively. The bottom chart is the same but for Experiment 1 and our mystery Experiment 2 in yellow.
Now that we have this lined up, can you try to figure out what conversion rate to expect for experiment 2?
Since the top chart increases 1% when switching from the left “Best MP3 Player!” to right “Listen to music anywhere,” it makes sense that “Listen to music anywhere” will have the same impact for the case below. It would make the 4% into 5%. This is what happens with no interactions and that case is shown in the chart below. Notice that Delta 1 and Delta 2 both equal 1% in this situation.
However, what if the Deltas weren’t equal? Let’s follow that same example. With our new table, Experiment 2 has a 7% instead of 5%.
Now take a look at the chart below, notice how the lines are no longer parallel, indicating that the differences are not consistent:
So something is happening between the factors that causes the conversion rate to be higher or lower than what we would expect, in this case it’s higher. This is an interaction. Anytime that the deltas are different, or the lines are not parallel, an interaction has occured.
Why does this happen? Continuity is a typical reason. If a headline makes no sense with a paragraph, but makes complete sense with another paragraph being tested then they will perform best when paired together. Or it could happen if the design aesthetics match for a button and a hero shot being tested (both green vs green and blue.)
In addition, interactions are not limited to 2 factors, they can appear for any number of factors. The names of these interactions are: main effects (the effects of the factors in isolation), 2-factor interaction effects (the effects of 2 factors interacting), 3-factor interaction effects (the effects of 3 factors interacting) and so on.
Finding interactions requires more data though, so there is a big trade-off in finding interactions versus ignoring them. For more about finding interactions during testing, look into fractional factorial versus full factorial test design.
