40% of BBM’s 5-star reviews could have been bought
At first, I was inclined to dismiss it as coincidence. However, since she had linked to the app, I decided to maybe have a look, and the pattern was instantly noticeable. I quickly wrote up a blog post with some additional data which pointed out that as many as 40% of BlackBerry’s 5+ star ratings could have come from nefarious means.
The post was ready to go live, but I wanted to wait and talk it out with Rob. The accusations are huge, and I wanted his opinion. We talked it out, and decided to hold on until I could get some more damning evidence.
More than just a meme
Unfortunately, other blogs caught whiff of it and hit publish. Unfortunate not because we weren’t the ones to break the news (well, that too) but because I had written a bit of Javascript code that I had injected via the Chrome browser’s inspector to look at as large a number of comments as possible, and the reason I had delayed the post was to improve the script a bit more to get an even larger data set.
The problem is most blogs seem to be focusing on one particular comment that is being repeated as the proof of possible fake downloads and reviews. A lot of other reviews have come in from users filling up the Google Play page, making it impossible to separate the sarcastic comments and the potentially fake ones.
The issue that I noticed was that there were a multitude of other questionable reviews in there, that mentioned the “smoothness” and “user friendliness” of the app, and thanked BlackBerry. From my experience as a developer, I found it difficult to fathom that so many users actually wrote such detailed praise. Clearly someone was sending out a bunch of mails to people telling them to write something that contained these buzzwords, and probably had a larger order than they ever dealt with hence leading to a lot of people writing the exact same thing. That’s why I built the script to find reviews containing the words “user friendly”, “smooth”, and “thank” (for either thank you BlackBerry or thanks BlackBerry), the most repeated phrases.
That’s when things started becoming even more suspicious: out of 213 4+ star written reviews, 5 mentioned all the three keywords, an additional four mentioning both “user friendly” and “smooth”, 43 mentioning “smooth”, 6 saying “user friendly” and 37 saying “thanks”. That is about 45% of BBM’s 4+ star reviews that I checked. For 5 star reviews from the sample set, that percentage is even higher at 53%.
It still could be a coincidence, so I decided to have a look at one of the most popular apps in the category: Whatsapp. For a 167 4+ star reviews of Whatsapp that I had a look at, less than 13% mentioned the key phrases, with no review mentioning them together.
The difference is just staggering. My belief was that potentially 40% of BBM’s 5-star reviews (which numbered over 60,000 at the time) were questionable. I would have loved to have gotten a lot more data to back up my claim, but it’s not possible right now. You can, however, see the raw data of the reviews I scanned (for both Whatsapp and BBM), visit this Google Drive document.
Who stands to benefit?
When I reached out for comment, all I got was the statement they’re sharing with everyone (despite me explicitly meaning I’m writing about my suspicions about that 40% number):
Thank you for reaching out. We are aware of a number of potentially fake five-star reviews of BBM for Android on Google Play. We do not approve of or condone such activities and are committed to working with Google to resolve this. There are also many genuinely great and useful reviews from our new BBM users on Google Play. We would like to encourage our passionate fans and users to continue to provide true assessments of the BBM experience through the proper channels.
That’s expected since it’s just the PR-team working to handle the shit-storm. Obviously, they’re not going to come out and say “yup, we did it”. However, it’s difficult to imagine who would be behind such a large and coordinated effort if not them, or at least someone working on their behalf.
- Buying reviews in such huge amounts can cost serious money, so I believe that rules out crazy fanboys coming together (which would probably have left a trail, too).
- Employees could have potentially pooled some money if they felt it was really important for the app to succeed for the company’s future and their own employment. That, however, would raise questions as to how high up within their hierarchy was this known.
- A PR agency hired by them seems the most likely in an effort to publicize the app online. A high volume of downloads and ratings gives them more stuff to pitch to bloggers and write about on forums. However, this raises questions whether the move was authorized by BlackBerry.
- BlackBerry itself would almost certainly not take the risk of getting caught themselves. Information about the payments could have easily become public, and would be the easiest of all the options to track down.
The business of buying and selling reviews and downloads
Coincidentally, I received a mail from someone offering me to purchase downloads and reviews for an app I have uploaded to the Play Store. Here is the offer I received (emphasis is mine):
You will get 20 or more 5 Star Ratings delivered in less than 5 days for just $69. My friends will write reviews and we will also share it with Google+ so your app page will look good. This is has been enough to get some apps on the front page in some categories! Or You can order 50+ 5 Star Ratings and 50 positive Comments delivered in less than 5 days for $169! as a bonus we will do 50+ google shares too.
With this package, you are guaranteed to see a HUGE improvement on your overall ranking in Google Play store and other android markets… and you stand the chance of getting noticed by the editors and have your app be recommended by the Editors (which would mean 1000s if not 100s of thousands of downloads)!!!!
After the incident, I tried to reach back to the sender trying to get a quote on a larger volume and, more importantly, a few examples of the Google+ profiles that would be used to share the app. I haven’t received a response, but if I have to make a guess whoever performed this for BlackBerry, would probably have paid a much smaller amount per review considering most seemed to be coming from Asian countries.
This isn’t the first time large corporations are potentially employing such questionable tactics: back in 2010 there was something very suspicious about a poll for Wall Street Journal readers asking who makes the best mobile operating system. And just this past week, the Taiwanese Fair Trade Commission fined Samsung and two marketing companies for employing people that “highlighted the shortcoming’s of competitors” and performed a “disinfection of negative news surrounding Samsung” on various forums. Unfortunately, with a petty fine for the three companies adding up to just a little over half a million dollars, it hardly does scares off future infringements.
Gaming the system, and why it needs to be stopped
Let’s get this very clear to anyone out there thinking this is a Google Play Store problem: developers have been looking at ways to cheat their way to the top of any app distribution channel from the very beginning. Here’s a post from TechCrunch talking about a similar issue back in 2009 for iOS apps. That issue was much easier to digest that the “download bots” on the App Store that helped free apps reach the top of the charts. I know somebody personally who used such a tactic, and while it cost them quite a bit while starting out, they made a killing over the longer term.
The issue I have with such practices is that it’s already incredibly difficult for an indie-developer to get noticed and succeed. When these developers fail to earn what they would liked to from their work, they might shy away from future projects and that, in turn, hurts the entire ecosystem.
However, it gets worse when larger companies behave like this, and I would rate it close to anti-competitive behavior. While BBM has a sizeable contingent of loyal fans, a higher rating for the app could be the deciding factor for a lot of people over other competitors. It’s not easy to predict how much the like of WhatsApp, Line, WeChat, etc would lose as a result of this (unlikely to be a large amount individually due to the competitiveness of the segment) it is clear that, if the allegations are true, they BlackBerry has benefited. After all, they’re claiming to have seen the app downloaded 10 million times in 24 hours.
The punishments that could be dished out
This is probably what I have been thinking about the most: how could Google punish perpetrators, once confirmed, that it makes a company think twice about such strategies? The extreme answer is probably banning them from the Play Store entirely. However, this might not be an easy step for them to take, with potential legal ramifications.
A more likely one is that they could bring to the Play Store from their bread-and-butter: penalties in search results. Search is a very important source of new users for apps, and also where developers benefit the most from inflated download counts, reviews and ratings. By penalizing their ranking for a significant portion of time, the loss suffered by the developer could be significantly more than the gains.
Google might also need to work on a spam filter for reviews. Yelp, for example, only publishes about three-quarters of the reviews in an effort to combat phoney-reviews. I know Google Play has a “mark as spam” option, but at this scale, reviews need to be blocked right from the beginning.
At a larger scale, the FTC would need to step in and impose fines, like the ones Samsung faced. These are rare particularly since they’re often done through other marketing companies, with even more complications when comments are coming from users on the other side of the globe. This is what has allowed most companies to astroturf without any consequences.
If this starts becoming standard practice (and it very well could) then matters could become really bad. For instance, there were reports that 20% of Yelp reviews were fraudulent, and New York attorney general Eric Schneiderman began an investigation that saw 19 companies fined $350,000 for astroturfing. Honestly, at about $20,000 per company, I don’t think it’s going to stop anyone.
Conclusion and Poll
I am not a fan of BlackBerry, and was particularly not happy with the way they went about releasing BBM, though I have stated before that I respect what they have done for the industry in the early-to-mid 2000s. If BlackBerry did indeed use such tactics (which would help increase their visibility on Play Store as well as convert more users who are on the fence about installing the app), then I would personally say that BlackBerry has reached its lowest. And considering everything they’ve gone through the last couple of years, that’s saying something.
In the entire mess, it’s difficult to put a number as to what BBM’s true rating would have been (my approximation was in the mid-3-point-somethings). Even without what happened, I believe they would have done a decent enough job to be called a success, and the response would have been nowhere near the negative one that Facebook Home got.
If you did download the app, do let us know the rating that you gave (or would give).
[polldaddy poll=7509689]