Recommendation engines have had an enormous, and paradoxically invisible, influence on the typical consumer. Today’s consumers buy products recommended by Amazon, watch movies recommended by Netflix, and listen to music recommended by Pandora. They read Facebook posts organized according to its internal recommendation engine, and they visit restaurants recommended by Foursquare. Even the ads they see are recommended based on their previous online behavior.

Without question, recommendations can boost profits for some companies. After heavily incorporating its recommendation engine, Amazon saw sales increase by 29 percent.

But recommendation engines typically require massive computing resources, and they don’t necessarily pay for themselves. Worse, many of them don’t actually work very well, and can even leave consumers baffled or amused… in a bad way.

The reason for this is that recommendation engines are trying to solve a very complicated problem with limited data. The reality of the situation is that the best way to resolve this probably isn’t with better algorithms. Instead, it might be better to simplify the problem itself.

The Absurdity of the Perfect Prediction Engine


Image Credit: Flickr User Mel Silvers

Recommendation engines exist to connect users with options better than something designed for a generic audience. That’s it. Unfortunately, it’s becoming increasingly common to expect recommendation engines to act as a go-to source of information, as though an app could really recommend a better restaurant than a chef.

This problem is virtually difficult to solve. To put a user in touch with the ideal solution to their problem based on dozens of data points or fewer is a statistical impossibility.

Every data scientist knows this. And yet apps, e-commerce sites, and start-ups are launching, betting on the promise that a recommendation engine will act as their selling point. A few companies have the right idea. The rest are missing the mark entirely, because they:

  • Have a relatively small number of recommendations to choose from anyway, rendering the engine effectively pointless.
  • Have a relatively small user base, making it difficult to draw any correlations between their behavior.
  • Have a relatively small number of interactions with each user, making it difficult to segment their users in any meaningful sense.

The idea that these businesses can simply insert their database into a machine learning algorithm, crunch it, and output good recommendations for their users, regardless of the variables, is out of touch with reality.

Is there a better way?

Embracing the Simple Recommendation

The core problem with most recommendation engines is twofold:

  • They make too many recommendations, and they do it too often
  • They attempt to make recommendations that are simply too specific based on the      available data

Let’s start with that first problem.

Too Many Recommendations, Too Often

Image Credit: Flickr User Chris Devers

In 1995, Sheena Iyengar, a professor of business at Columbia University, conducted a landmark study into the effect of options on choice. The researchers set up a jam booth at a gourmet store. Every few hours, they switched between a booth with 24 jams and a booth with only 6 jams.

They study discovered that while more people were drawn to the bigger assortment of jams, more people actually bought jam if they saw the smaller assortment.

If your recommendation engine offers too many recommendations, it might paralyze your users into indecision.

Don’t get me wrong: users might like having a large number of options when they visit your site and shop around. Variety might be the thing that draws them to your site. But when you make a recommendation, offering too many options is counterproductive. If a customer asks a waiter for their recommendation, they don’t want to hear twelve options. That’s what the menu is for. A customer who wants a recommendation wants to hear one or two options, maybe 3.

Offering too many recommendations has another downside. It reduces the perceived quality of the recommendations.

In 2005, researcher Kim weaver investigated exactly this problem [PDF link]. A hundred participants were split into “evaluators” and “presenters.” The presenters were supposed to compile a resume of their best work by sharing 1, 2, or 3 publications.

While only 10 percent of the presenters opted to share only their single best work, the evaluators rated these portfolios highest.

In other words, while we tend to think that we will give ourselves the best chance if we make the most recommendations, the opposite is the truth. Consumers will tend to be more pleased with the recommendations if we share only the best option.

Likewise, most recommendation engines tend to offer suggestions too often, when they don’t make sense, and when they aren’t particularly useful.

Put simply, there isn’t always enough data to make a good recommendation. If our sites are designed to offer recommendations regardless of the quality of the data, users will become accustomed to receiving lackluster recommendations. This leads to ad blindness. Users will simply ignore the recommendations and fail to see the good ones when the right data is available.

A smarter engine will avoid sharing any suggestions until a statistically significant result is available.

Jumping the Gun: Specificity without Data

Image Credit: Flickr User Lena

Most recommendations are simply too personalized.

That statement might sound heretical, but it’s true. Amazon often jumps the gun and makes recommendations based on products that you’ve seen, even if you didn’t make a purchase. Facebook tends to show you posts only from people that you’ve interacted with, despite the fact that you only interact with the people that you see to begin with.

All algorithms have faults, and we routinely forgive Amazon, Netflix, and Facebook for their imperfect suggestions. But you have to remember that these companies also have massive datasets to work with. If you used the same algorithm on your very limited dataset, your recommendations would almost certainly be terrible.

The reality of the situation is that most of these algorithms place too much emphasis on personalization, and not enough on conversion rates. For larger companies, the approach almost certainly boosts overall conversions. But for small companies, with insufficient data, the predictions overreach and underperform.

With limited data, it’s almost certainly a better idea to go with a less sophisticated algorithm.

Here are a few ideas worth considering when your datasets are limited:

  • If you have a limited number of products to begin with, a manually coded, flowchart-based algorithm is probably going to outperform anything a machine learning algorithm can come up with, especially if your number of customers and actions is limited. It might even be less expensive to freelance it out, and manage the project with a tool like WorkZone.
  • Even with relatively large datasets, an algorithm based on carefully designed regression analysis is probably going to do better than a machine learning algorithm. Most machine learning algorithms need to make mistakes in order to improve, and there’s no reliable way to train them on anything other than a real set of customers. Small businesses will likely hurt their      reputation more than they will help their sales if they use machine learning early on.
  • Recommendations based on direct input from the user will almost certainly be better than recommendations based on their behavior, especially if you don’t have sales data to work with. By recommending items with high conversion rates in      categories that users have self-selected, you are not only likely to get a sale, you’re also less likely to creep customers out by monitoring their behavior. Of course, asking customers to fill out preferences before a sale is probably a bad  idea, since it can make the purchase process more painful.

To Wrap Up

Advanced recommendation engines are great for big companies with product diversity and large datasets, but these same engines will almost always overreach when used by smaller businesses. If you think a recommendation engine will enhance your unique selling proposition, it’s probably best to keep things simple by:

  • Not overwhelming the user, and offering only a few high quality suggestions
  • Offering  suggestions only when statistically significant results are available
  • Using simpler algorithms, possibly manually coded, with tried and true systems like regression analysis
  • Considering the use of direct user input

Thanks for reading. What are your thoughts on recommendation engines, and how small businesses should approach them?

Rohan Ayyar works for E2M, a premium digital marketing firm specializing in content strategy, web analytics and conversion rate optimization for startups. His posts are featured on major online marketing blogs such as Moz, Search Engine Journal and Social Media Today. Rohan hangs out round the clock on Twitter @searchrook – hit him up any time for a quick chat.