Categories
Uncategorized

Steam sales estimation methodology and accuracy

Transparancy of methodology

While reviews have been used as a proxy for sales on Steam for a long time, VG Insights was the first platform to start using more complex review based algorithms – accounting for year or release, price of the game, genre of the game and more. We’ve published articles on our findings in our articles “How to estimate video game sales” and “Further analysis into Steam review to sales ratio“.

In fact, SteamDB, a popular Steam stats website, has used our simple “review multiple based on the year of release” rules to publish their own owner estimate data

VG Insights has now moved on from just using review multiples to include a combination of 3 types of estimations:

  1. Review multiples – we continue to use review multiples as one source of estimations, but our algorithm includes more complexity behind the scenes, including things like year of release, genre and price of the game
  2. Steam top selling games lists – We also use the Steam weekly top selling games lists to adjust our data and make it more accurate, somewhat similarly to how the likes of SensorTower and App Annie estimate mobile games sales
  3. An algorithm based on our own proprietary data of actual units sold, collected from both public sources as well as through publisher and developer relationships

Testing VG Insights accuracy VS publicly available information

We have collected a sample of press releases and other public data on Steam game sales to compare our algorithm against the real world data. The results are summarised below

Number of games tested: 55
Total units sold tested: 306.7 million

We’ve included every datapoint that we could find through primary research. We’ll continue adding to this list of comparisons as new data becomes available.

Due to the nature of our methodology, we cannot guarantee that it’s a completely unbiased sample size (deciding to publish numbers publicly vs not is inherently a bias creating decision). However, the sample covers a wide range of games in terms of size, genre and release date

VG Insights is only 3% off the true reported numbers at an aggregate level

Firstly, we looked at how well VG Insights performed against the sum of all the reported datapoints

The results are better than anticipated! Even though at individual game level, the accuracy tends to fluctuate more, at an aggregate level, VG Insights’ methdology matches the true reported data very closely

At individual games level, there are outliers, but most games fall in the +-15% margin of error range

Details of individual games tested is highlighted below

GameDateReported Units SoldVGI Estimate% Difference
Townscraper19/05/20210.38m0.27m-30%
Loop Hero12/03/20210.50m0.41m-19%
Valheim03/03/20215.00m4.10m-18%
Soma16/03/20211.00m0.84m-16%
Black Desert Online31/07/20170.53m0.45m-14%
Inscryption28/10/20210.26m0.22m-14%
Escape simulator04/05/20221.00m0.86m-14%
Kingdom Come: Deliverance16/02/20180.30m0.26m-14%
ARK: Survival Evolved30/08/20164.50m3.89m-14%
Fall Guys11/08/20202.00m1.73m-13%
Total War: 3 Kingdoms29/05/20191.00m0.88m-12%
Days Gone13/01/20221.00m0.88m-12%
Stardew Valley13/04/20161.00m0.88m-12%
Core keeper10/03/20220.10m0.09m-10%
Inscryption05/01/20221.00m0.91m-9%
Valheim19/03/20216.00m5.47m-9%
PUBG08/04/20171.00m0.91m-9%
Golf Peaks11/12/20190.002m0.002m-6%
Horizon: Zero Dawn26/05/20222.40m2.27m-6%
PUBG05/09/201710.00m9.50m-5%
Stardew Valley01/03/202213.00m12.36m-5%
PUBG02/05/20172.00m1.92m-4%
Cogmind04/05/20180.02m0.02m-3%
Satisfactory05/07/20200.37m0.36m-3%
God of War26/05/20220.97m0.96m-1%
PUBG27/01/201942.00m41.59m-1%
Fall Guys13/11/202010.00m9.93m-1%
PUBG21/08/20178.00m8.00m0%
Loop Hero08/12/20211.00m1.01m1%
Forager12/05/20190.20m0.20m1%
Core keeper23/03/20220.50m0.51m1%
Core keeper14/03/20220.25m0.25m1%
Valheim25/04/202210.00m10.23m2%
The Last Spell29/01/20220.13m0.13m2%
Firewatch09/03/20160.50m0.51m3%
Witcher 308/04/202012.00m12.45m4%
Valheim20/05/20216.80m7.09m4%
Dota 221/11/202072.00m76.37m6%
Garry’s Mod01/09/202120.00m21.21m6%
Crusader Kings 317/03/20222.00m2.13m6%
Mortal Online 202/02/20220.11m0.12m8%
Euro Truck Simulator 201/03/20219.00m9.78m9%
Terraria31/12/202017.20m18.88m10%
Sea of Thieves23/12/20215.00m5.53m11%
Temtem21/02/20200.50m0.56m11%
CS: GO18/02/20155.60m6.22m11%
The Forest05/11/20185.30m5.91m12%
Rust10/01/202212.00m13.50m12%
Rust24/06/20175.00m5.72m14%
Aragami04/10/20190.32m0.37m15%
Prison Architect23/06/20172.00m2.39m19%
Stellaris12/05/20203.05m3.79m24%
Days Gone26/05/20220.85m1.07m26%
Yes, your grace09/03/20200.03m0.04m28%
Nightmare Reaper26/03/20220.02m0.03m62%

46 out of the 55 games tested (84% of games) fell into the +-15% margin of error range. The margin of error is fairly evenly distributed in terms of over- vs underestimation.

There are occasional outliers to out estimates, but these tend to be smaller games and games that have been only recenlty released.

Accuracy of trends

Given our sales estimates are partly dependent on number of reviews, an accurate data reported in real time by Steam, the VG Insights trends in units sold are highly accurate, even for games where the margin of error for point-in-time sales is higher.

Trends are also shown by looking at concurrent players and followers, actual data reported on the Steam platform, increasing the value of VG Insights as a trend monitoring platform.

Continued improvements and transparency

While we can never guarantee complete accuracy of our estimates, we strive to provide the best publicly available estimates

We will continue to refine both our estimation models as well as this data accuracy summary page in order to provide you with transparancey and confidence in our numbers