Reputation management: Building trust among virtual strangers

Copyright ©1999-2007 IBM Corporation. All rights reserved.
IBM and the IBM logo are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Other company, product and service names may be trademarks or service marks of others. References in this publication to IBM products and services do not imply that IBM intends to make them available in all countries in which IBM operates. G510-6327-00
Reproduced with permission by the author
http://www-935.ibm.com/services/us/gbs/bus/pdf/g510-6327-00_etr_reputation_management.pdf
Executive Technology Report is written by Peter Andrews, Consulting Faculty, IBM Business Institute, and is published as a service of IBM Corporation. Visit  ibm.com/ibm/palisades
ibm.com/bcs Executive technology report September 2006  
Author: Peter Andrews
pja@us.ibm.com
Innovation Strategist, IBM Executive Business Institute
17 November 2007  

Executive summary
Trust is essential to efficient business activities, but it doesn’t come naturally in an online environment. By setting up the right systems of feedback and ratings, it’s possible to create environments where participants feel safe to buy, sell and work together.

In this Executive Technology Report, Peter Andrews, Consulting Faculty Member at the IBM Executive Business Institute, interviews Claudia Keser. Claudia is a Research Staff Member in the workforce optimization group of the IBM Mathematical Sciences Department. She collaborates with IBM Global Business Services, doing workforce analytics for IBM clients and bringing more rigor into the statistical data analysis. Her research background is in economics, specifically in experimental game theory/behavioral economics, with a focus on issues of trust, cooperation and incentives.

Peter Andrews Could you explain what “reputation management” is and provide an example?

Claudia Keser Reputation management, or a reputation system, is a way to maintain trust in online communities, where we anonymously interact with people that we might have never met, not even heard of, and that we might never meet again. This is achieved by the provision of information about past performance. To be somewhat more precise, [with] a definition given by Paul Resnick et al., it is a system that collects, distributes and aggregates feedback about past behavior. 

A famous example is eBay's Feedback Forum, where after each transaction the parties involved may evaluate each other. Such a reputation system plays a double role. The first is to enable trade by making trade safer and increasing participants' trust. The second role is in promoting satisfactory trade and increasing participants' trustworthiness.

Do you want me to elaborate on eBay's Feedback Forum?

Peter Andrews Since it's a familiar example, that would be a great way to illustrate some of the tools and methods of reputation management.

Claudia Keser The participants in a transaction are allowed to rate each other by submitting a comment and a rating. The rating takes one of three values: "+1" for a positive comment, "-1" for a negative comment and "0" for a neutral comment. All ratings that an eBay user receives from distinct other users are summed up into a Feedback Rating Number. This number is attached to each user ID. A user who accumulates 388 positive and no negative comments has a Feedback Rating of 388. But a user with 759 positive and 371 negative comments has the same Feedback Rating. The Feedback Rating is part of the user's Feedback Profile, which can be obtained by clicking on his Feedback Rating. It provides the full list of textual comments (received from buyers and sellers, and given to others), the distribution of all previous ratings received from distinct users, the percentage of positive ratings, as well as the distribution of recently received ratings over the past seven days, past month and past six months.

Peter Andrews What are some of the benefits of a reputation management system like eBay's?

Claudia Keser If we had no reputation management in an online market like eBay, we would most likely have a "market for lemons" problem. Since typically there is no opportunity for inspection, we could not distinguish between sellers offering good or bad quality. Thus, buyers would be reluctant to pay high prices so that more and more of the high-quality sellers would leave the market. In the end, there would remain only the "lemons" on the market, and nobody would want to buy there. With reputation management, as has been shown in many studies, sellers with high reputation obtain higher prices. Reputation management mitigates the lemons problem.

In my own research, I've shown in a very controlled laboratory environment, using the method of experimental economics, that reputation management significantly increases both buyers' trust and sellers' trustworthiness and, thus, increases the volume of trade. Reputation management provides information to buyers and encourages the recipients of positive information to trust more. Also, the attribution of negative reputation may work as a sanctioning mechanism to punish dishonest behavior by sellers, which makes the owner of reputation behave in a more trustworthy way.

Peter Andrews Can this be applied beyond buying and selling? For instance, can it be used to facilitate the formation of teams? Making the Internet safer for kids?

Claudia Keser Yes, this is actually applied in environments such as Epinions.com a Web site which provides opinions/reviews about goods. Other examples include Slashdot.org, a technology-related news Web site with user-submitted summaries of news and comments, and Advogato.org, an online community for free software development. Interestingly, in those environments, much more complex reputation systems are used. A lot of research is going on in designing such systems. 

The formation of teams could certainly be facilitated by reputation information. [But]... I might not be happy about evaluating my colleagues at IBM Research. We are talking about virtual environments where I might have no other way to gather information about anonymous users. A company is not like the Internet. I can easily get information, for example, about my colleagues in the China Research Lab. As a human resources person, I would be concerned. I believe that reputation management among people who know each other in person and frequently interact can do more harm than good for teamwork.

Peter Andrews There are, of course, privacy and other concerns. Which brings up another question: what ARE some of the issues around reputation management? The challenges to designing a good system?

Claudia Keser Going back to eBay, there are several issues: 

• Feedback is a public good; to speak like an economist, users have an incentive to take a free ride on the evaluations given by others. I believe I've read somewhere that about 50 percent of the traders provide feedback.

• We observe hardly any negative comments, less than 1 percent, I think. The major reason might be fear of retaliation. A seller might punish a buyer who gave him a negative evaluation by doing the same, whether justified or not. To date, eBay uses a unique feedback rating number, aggregating information received both as buyer and seller, although the verbal comments are split up.

• Strategic behavior is hard to detect. For example, since the ratings are not weighted by monetary importance of the transaction, it is easy to build up a positive reputation by buying or selling a large number of cheap items, maybe even trade with friends, then boom, come up with very expensive items to sell ... and not deliver.

Peter Andrews What are the elements of a good reputation management system?

Claudia Keser Let me use here Paul Resnick et al. again. They identify three basic principles:

• Entities in the system must be long-lived enough to establish an expectation of future interaction.

• Feedback concerning current interaction is elicited and distributed and must be visible in the future.

• Feedback must have influence over the actions/trust of entities in the future.

The elicitation and distribution of feedback provide challenges. Research still needs to be done in this area. This is where my paper in the IBM Systems Journal comes in, where I suggest using the experimental economics lab... Want to hear about this?

Peter Andrews Absolutely. Please say a bit about how you use the lab to explore the possibilities of reputation management.

Claudia Keser I used a very simple abstract game that allows us to measure the trust of one player (the buyer) and the trustworthiness of another player (the seller). I applied this trust game to measure the effect of an eBay-like reputation system in an environment where "strangers" interact with each other in one-time encounters. These “strangers” were student participants who came to the experimental economics laboratory, where they sat in isolated cubicles to make decisions at a computer, not knowing the other participants with whom they interacted. They knew that they would not interact more than once in a row with the same person. 

I could not only show that reputation management significantly increases trust and trustworthiness compared to a situation without reputation management, but also that the levels of trust and trustworthiness are as high as those obtained in environments where "partners" – always the same two anonymous players– interact with each other. Interestingly, with reputation management, the high levels can be maintained up to the very last round of interaction, when they significantly drop. (This is a very typical effect observed in experimental economics, cooperation breaks down when people know that the game ends.) In contrast, the trust level tends to continuously decline over time in the partners’ interaction.

I also started to examine design issues of reputation management systems, examining different information situations. Providing buyers only with the most recent feedback given to the seller they are currently interacting with is enough to significantly increase trust and trustworthiness. However, providing the entire history, that is the distribution of all previous ratings, turns out to be more efficient. 

The experimental economics method could be used to examine many more design issues. For example, would we observe a larger number of bad evaluations if we use a 1-to-5 scale rather than a negative-neutral-positive scale? ... and many others.

Peter Andrews Now, I noticed that, in this framework, buyers’ benefits seem asymmetric. Is this a source of concern? If so, do you have any ideas on how to get things into balance?

Claudia Keser When we look at the trust game that I used, if there is no reputation management, trust by the buyers does not pay at all, the sellers get all of the surplus. (By the way, according to economic theory, there should be no trade at all in this game.) The reputation system makes trust pay off for the buyers. This makes them trust more, [with] more transactions taking place, and both are better off than without reputation management. Thus, I do not see a concern. 

The payoffs become more equitable. (I could talk now for hours how people like equitable outcomes, even in situations where it is obviously bad for growth.)

Peter Andrews Could you talk a bit about the implications for low-trust societies and their participation in the global economy? Would reputation management help?

Claudia Keser Individuals in high-trust societies spend less money and effort to protect themselves from being exploited in economic transactions. In other words, trust economizes on transaction costs. It has been empirically shown that differences in trust across countries help explain differences in investment and economic growth. In a study with Hai Huang, Jonathan Leland and Jason Shachat, we have shown that the level of trust is likely to have an additional indirect impact on economic growth through an impact on the Internet adoption. Assuming that the Internet will positively affect productivity, low-trust countries, most of which tend to be of low and middle income, are doubly penalized in terms of economic growth. First, they are penalized for low trust in terms of investment and growth impact, and then again through lower adoption of growth-enhancing technologies.

Reputation management can probably alleviate the problem to some extent, although we first need to get the people to use the Internet at all. The issue of trust in a society is complex. Francis Fukuyama argues, based on historical analysis, that individuals in some societies are more naturally led to trust each other due to their cultural, institutional and religious heritage.

Peter Andrews It's often said that trust is gained over time and lost in an instant. From your studies, do you have any insights on this old adage?

Claudia Keser In our studies, we do not find evidence for this. Buyers tend to look more at long-run reputation than at short-run reputation when the former is available. Thus, a single misstep would not ruin the trust in a person...

Trust needs certainly a longer time to be built up than to be lost, but in general it is not lost instantaneously. If this were the case, I would not, for example, use my credit card any more.

I would say that we still know relatively little about the design of efficient reputation management. We observe on some sites the design of very complex systems that deal with some of the evident issues, but they seem pretty difficult to understand. I have doubts about their impact on behavior if they are too complex, just a black box. I think we should keep them simple, but there are still many open questions, such as how best to present the information to the users, the effect of using different scales or multidimensional ratings, etc. And, since more and more market places are created, there are potentially many more applications for reputation management.

Related resources
Keser, Claudia. “Experimental games for the design of reputation management systems.” IBM Systems Journal. Vol. 42, No. 3, 2003.
Resnick, Paul, Richard Zeckhauser, Eric Friedman and Ko Kuwabara. “Reputation systems.” Communications of the ACM, 2000, Vol. 43, No. 12, 45-48.
Huang, Hai, Claudia Keser, Jonathan Leland and Jason Shachat. “Trust, the Internet, and the digital divide.” IBM Systems Journal. Vol. 42, No. 3, 2003.
Fukuyama, Francis. Trust: The Social Virtues and the Creation of Prosperity, New York: Free Press, 1995.

About this publication
Executive Technology Report is a monthly publication intended as a heads-up on emerging technologies and business ideas. All the technological initiatives covered in Executive Technology Report have been extensively analyzed using a proprietary IBM methodology. This involves not only rating the technologies based on their functions and maturity, but also doing quantitative analysis of the social, user and business factors that are just as important to its ultimate adoption. From these data, the timing and importance of emerging technologies are determined. Barriers to adoption and hidden value are often revealed, and what is learned is viewed within the context of five technical themes that are driving change:
Knowledge Management: Capturing a company's collective expertise wherever it resides – databases, on paper, in people's minds – and distributing it to where it can yield big payoffs
Pervasive Computing: Combining communications technologies and an array of computing devices (including PDAs, laptops, pagers and servers) to allow users continual access to the data, communications and information services
Realtime: "A sense of ultra-compressed time and foreshortened horizons, [a result of technology] compressing to zero the time it takes to get and use information, to learn, to make decisions, to initiate action, to deploy resources, to innovate" (Regis McKenna, Real Time, Harvard Business School Publishing, 1997.)
Ease-of-Use: Using user-centric design to make the experience with IT intuitive, less painful and possibly fun
Deep Computing: Using unprecedented processing power, advanced software and sophisticated algorithms to solve problems and derive knowledge from vast amounts of data.
This analysis is used to form the explanations, projections and discussions in each Executive Technology Report issue so that you not only find out what technologies are emerging, but how and why they'll make a difference to your business. If you would like to explore how IBM can help you take advantage of these new concepts and ideas, please contact us at insights@us.ibm.com. To browse through other resources for business executives, please visit ibm.com/services 

Peter Andrews is an innovation strategist and consulting faculty member at IBM's Executive Business Institute. He has spent a career bridging the gap between the technical potential and the bottom line. He is the author of over 100 articles on innovation, emerging technology and leadership, and his Executive Tech Reports are featured monthly on the IBM services Web site. Andrews consults and holds workshops both within IBM and externally. He uses a variety of techniques to probe, extend and validate the opportunities presented by new technologies. He has helped banks, insurance companies, manufacturers and retailers develop their own capabilities to take a fresh look at emerging technologies, come to a common understanding of their value and take practical steps to exploit them. Notably, he has held innovation workshops with over 100 IBM Researchers worldwide that have helped them to determine the business implications of their inventions, recognize possible sponsors and create value propositions. Andrews has been actively involved in research and working at the leading edge for his entire career. His participation is always in demand for IBM Academy studies, and he is a popular presenter on the future, most recently as the closing keynote speaker for KMWorld 2006. He can be contacted at pja@us.ibm.com, New York (845) 732-6095 and http://www.ibm.com/ibm/palisades 

Short summary
Managing reputation amongst individuals who interact infrequently or for only a few times can greatly enhance an organisation's reputation and bottom line, but would be used with caution amongst more regular team members..

Keywords and relevant phrases
cooperation, feedback, incentive, online environment, participation, rating, reputation management, reputation system, teamwork, trust,