Noise is a tricky topic. If you’ve ever wrangled with terms such as standard deviation, variance, and statistical significance in the workplace, the discussion can get murky very quickly, and it’s a bit dicey to discuss metaphysics during a 9 to 5 workday.
Your friendly neighbourhood data analyst has the mathematical underpinnings covered, but it’s worth taking a step back and making sure we understand what Noise really is and why we care about it.
Say you have two game studios that have each released two games in the past three years. We look at the Metacritic scores of their two games.
Game Studio Boring’s two games scored 60 and 80 points.
Game Studio Weird’s two games scored 50 and 90 points.
The mean Metacritic score for both studios’ output is 70. Add the two numbers together and divide by 2 if you don’t believe me.
Just looking at the mean scores alone you would have to conclude that Boring and Weird are identical in terms of critical reception, but it’s clear that the studios exhibit differing levels of variability (or noise!) in their output.
So which studio is better? Take a second to think about it. It’s not obvious at all. Maybe pose this debate to your colleagues.
The answer is “it depends”. Depends on what? It depends on how much of a gambler you are. Would you take a gamble on Studio Weird, with its greater variability? Or would you stick with the more consistent Studio Boring?
Have a think about it, and we’ll talk again soon!