Rating system scores are built on an unstable foundation
Institute for Operations Research and the Management Sciences
INFORMS Journal Marketing Science New Study Key Takeaways:
- Reputation systems, which play a critical role in online marketplaces, might be built on an unstable foundation.
- Research suggests reputation system scores become less useful over time.
- Ratings are prone to inflation because raters feel pressure to leave “above average” ratings.
BALTIMORE, MD, June 1, 2022 – Five stars means a perfect rating, right? Consumers use online ratings systems to decide which products to purchase, but are they reliable sources of information? New research in the INFORMS journal Marketing Science finds these ratings systems might be built on an unstable foundation, with the systems becoming less useful over time.
The study, “Reputation Inflation,” conducted by Apostolos Filippas of Fordham University, John Horton of the Massachusetts Institute of Technology and Joe Golden, CEO and co-founder of PerfectRec, looks at data that spans more than 10 years and five online marketplaces.
“Our research shows across a variety of online marketplaces that five-star feedback increases over time, and it’s not because people are more satisfied with what they receive. This likely makes the system less informative over time, as everything looks nearly perfect, which potentially has bad consequences for the marketplace,” says Filippas, an assistant professor in the Gabelli School of Business at Fordham.
The authors say that ratings are prone to inflation because raters feel pressure to leave above-average ratings, which in turn pushes the average higher. This pressure stems from the raters’ desire to not harm the rated seller.
“Reputation systems, as currently designed, sow the seeds of their own irrelevance,” says Horton, the Richard S. Leghorn Career Development Professor at MIT Sloan School of Management, and a Faculty Research Fellow at the National Bureau of Economic Research.
“Our results suggest that reputation systems, which play a critical role in online marketplaces, might be built on an unstable foundation, with the systems becoming less useful over time,” concludes Golden.
About INFORMS and Marketing Science
Marketing Science is a premier peer-reviewed scholarly marketing journal focused on research using quantitative approaches to study all aspects of the interface between consumers and firms. It is published by INFORMS, the leading international association for the decision and data sciences. More information is available at www.informs.org or @informs.
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.