How harnessing curiosity can improve online reviews

BY: GINA MANTICA

Reliable reviews on digital marketplaces are hard to come by. Many reviews are biased, and honest ones are difficult to incentivize. These “reputation systems” on digital marketplaces provide customers with information that help guide their future purchases, and therefore need to reflect transactions accurately.

During graduate school, Andrey Fradkin became a part-time employee at Airbnb and started researching Airbnb’s review system. Fradkin and coauthors tested a new reputation system for Airbnb and discovered that it leads to not only more reviews by guests and hosts, but also faster and more accurate reviews. Now a Hariri Institute Junior Faculty Fellow and Assistant Professor of Marketing at BU’s Questrom School of Business, Fradkin’s findings were recently accepted for publication in Marketing Science, and the review system is still used today by Airbnb.

Prior to 2014, reviews were revealed as soon as they were submitted. That means that the second reviewer, whether it be the guest or the host, could reciprocate or retaliate in response to the first review. “Everyone was aware of this retaliation issue, but no one had run an experiment about changing the review system,” said Fradkin. So, Airbnb experimented with changing their reputation system to only reveal reviews when both parties submitted their assessments, or when two weeks passed since the checkout date.

The simultaneous reveal of reviews entices both guests and hosts to submit more reviews. Fradkin and his coauthors argue that this increase is caused by the desire of users to see what the other person wrote about them. Consistent with this idea, the researchers found that the first review triggers a much faster second review, with the time between the two decreasing by 35% compared to the old reputation system.

Revealing reviews to both parties involved in a transaction at the same time could also lead to more accurate ratings. Sometimes, digital marketplace reviews are written with a sense of reciprocity, or the idea of kind actions being rewarded in kind. This may lead to inflated ratings. But reciprocity is not possible in a system where reviews are kept hidden.

In line with these ideas, the researchers found that reviews using the new system were more negative on average, if only by 0.25%, and reviews from guests and hosts were less similar to each other. The more negative reviews were not from new groups of people, but rather from people that would have written reviews regardless of whether they used the old or new systems.  Additionally, the lack of similarity between reviews suggests that ratings were not influenced by the other party. Taken together, these findings support that the new system was able to capture more accurate reviews.

If reviews are indeed more accurate, Fradkin and his coauthors predicted that lower quality hosts should see less demand from guests. But the researchers did not find any effect on hosts. This might be because there was such a small impact on ratings with the new system.

Seven years later, and this reputation system is still used by Airbnb. “I’m interested in studying whether and how platforms could encourage informative reviews and how to measure if that creates better matches on the platform,” said Fradkin. In the case of the Airbnb platform, curiosity and a desire to unveil reviews drove quicker, more accurate reviews more often.

Interested in learning more about the transformational science happening at the Hariri Institute? Sign up for our newsletter here.