How Bad Did Analytics Bugs Fool Facebook Pages? Reach Looked Down 14% When It Wasn’t

Months of mysterious Insights bugs duped Facebook Page owners into thinking their reach was plummeting. The median Page’s analytics showed it reached 14.39 percent fewer people than it actually did, according to early data and graphs from EdgeRank Checker, a Facebook analytics startup that gave us the first look. The bugs may have caused people to mistakenly undervalue their Pages, change strategies or buy ads to make up for “lost” reach.

Facebook announced the bugs last week. During its efforts to speed up its iOS and Android apps in August and December, it accidentally stripped out the markers used by Page Insights to count impressions. This and other bugs led several metrics to be underreported. Fixes and bug-prevention systems are now in place, and accurate data began flowing into Page Insights yesterday, making today the earliest opportunity to get a concrete sense of the glitch’s impact.

Facebook Reach Total Organic ViralEdgeRank Checker analyzed 1,000 Pages who posted both last week when Insights was still wrong, and yesterday with reach reported accurately. How much reach was underreported depends a bit on the Page’s fan count, as those with 500,000 to 1 million Likes were led farther astray.

A Page with 750,000 fans would have been reaching 47,000 fans per post (6.24 percent), according to the broken analytics, when in reality they were reaching 78,000 (10.35 percent). That means Insights was underreporting reach by a massive 66 percent. The impact was more subtle for a Page with 100,000 fans. The buggy Insights would have shown them reaching 10,000 of their fans per post when they were actually reaching 11,000, with reach underreported by 10.2 percent.

The most stunning part of the data was that the median Page’s viral reach according to Insights went up 275 percent after the fix. EdgeRank Checker‘s CEO Chad Wittman believes Facebook also fixed a bug that caused viral reach to exclude the impressions from posts re-shared by a Page’s fans.

Facebook Page Total Reach Change

Once there’s a whole month of accurate data, we’ll get a more precise look at the effect of these bugs. Facebook tells me it doesn’t calculate the aggregate impact, but didn’t dispute EdgeRank Checker’s findings. However, it did note the day-over-day data may look a little more extreme, and the data may even out a bit with time.

While in some ways Page admins should be relieved, some are certainly mad. Though Facebook snapped into action squashing the bugs as soon as they were discovered, it still didn’t disclose the issues for the three weeks while it fixed them.

Some admins have complained about wanting refunds for ads they bought during the bug period, because they wouldn’t have paid for them if they knew their real reach was higher than shown. If Facebook wants these Pages to keep filling it with content and buying its ads, it can’t have any more slip-ups like this.