Why a Kickstarter Reputation System Could Reduce Prejudice

7 April 2016 | 32 Comments

belo-200x200-4d851c5b28f61931bf1df28dd15e60efAirbnb and Kickstarter have something big in common: They both rely heavily on trust.

Airbnb hosts trust that their guests are going to respect their space, and their guests trust that their hosts aren’t going to endanger them.

Similarly, Kickstarter backers trust–by paying for something months before it will be created–that creators are going to deliver on their promises (or at least communicate when things aren’t going as planned), and creators trust that backers are going to be constructive, not destructive.

What does this have to do with prejudice? I recently watched a fantastic TED talk about trust by Airbnb co-founder Joe Gebbia. I’ve linked to a pivotal moment in the talk at which Gebbia starts to talk about a joint study Airbnb did with Stanford University.

The study measured trust levels between two complete strangers, with tests like asking someone to hold your baby for a minute or to take care of and return your unlocked phone. The study revealed two key findings. Here’s the first:

  1. We have a natural social bias to trust people who are similar to us. The more different someone is, the less we’re inclined to trust them.

This is the root of many of our biases. It’s an unfortunate side of human nature that we’re more likely to trust people who are the same age, race, nationality, gender, religion, sexuality, etc. Most of us fight against these biases all the time so that we can base our trust on evidence, not our innate fear of the unknown. But the study reveals it’s an uphill climb.

This bias is in effect all the time, including when we’re browsing Kickstarter. Our human nature dictates that we’re more likely to trust creators who are similar to us.

I don’t know about you, but that scares me. It scares me that I might have some kind of built-in predilection to backing projects by white, male, American creators, because those attributes have no correlation to the trustworthiness of those creators.

Fortunately, there’s a solution. This is the second finding in the Stanford/Airbnb study:

2. High reputation beats high similarity.When it comes to trusting someone, if we’re aware of someone’s reputation, our similarity bias goes away.

When first heard this, I had to pause the TED talk. The implications of this were huge. I’ve always thought it would be nice for Kickstarter to have a rating system, but it had never occurred to me that it could be an effective tool for fighting prejudice.

Imagine a Kickstarter platform where we trust creators based on what they’ve done, not who they are. A platform where our deeply rooted biases are trumped by hard data.

I think this is a pretty big deal. For me, it elevates the KS rating system idea from “nice to have” to “must have.” And I’m not talking about implementing it on an ancillary site like Kicktraq or Smarter Backer. I think it should be fully integrated into the Kickstarter platform.

Here’s my proposal:

  1. After a project has been fulfilled, the creator can check a single box on their dashboard to signal to Kickstarter that all rewards have been delivered (not the current system where you have to check each backer individually–who does that?!)
  2. Kickstarter then sends a message to all backers asking them to rate the creator’s performance on a 1-5 scale for that specific project.
  3. The rating is displayed at the top of the project page and as a composite rating on the creator’s profile and on all of their future project pages.

One criticism I’ve heard about this idea is that it could create a disadvantage for new creators. That may be true, but it’s not much different than seeing “0 projects created” on the current version of Kickstarter. Plus, unlike the similarity bias, it’s fair to expect that someone needs to earn our trust, and they can start to do that on the project page itself.

Secondarily–less about prejudice and more about creating a more welcoming ecosystem–would be a creator’s ability to rate backers. This information would probably only be seen by Kickstarter to identify backers who are consistently destructive and disruptive. I don’t think the current spam system accounts for this.

Now that you’re aware of the Stanford/Airbnb study, what do you think about Kickstarter implementing a reputation system? Feel free to comment and/or respond to the poll below, or even tweet to @kickstarter if you think they should know about this.

Here’s the video, with the study on trust discussed at the 8:00 minute mark:

Also read: Do You Trust Me? and 10 Ways I Would Update the Kickstarter Platform

32 Comments on “Why a Kickstarter Reputation System Could Reduce Prejudice

  1. It’s a similar system that Uber uses. Rate the driver, rate the passenger. It’s a near must in our fear-driven modern society when it comes to trust-based face-to-face encounters (live in my house, sit in my car).

    For Kickstarter I see the benefit, but I don’t think an Ebay rating of the seller effects my choice to buy. Unless it’s amazingly low. But there-in lies the point, I guess.

    I like the idea of secretly rating backers for Kickstarter’s sake, so they can be more aware of trolling, etc. But on either side, a person is only 1 new account away from resetting their rating.

    I think it’s a very neat idea, but it’s going to need an ingenuitive quirk to make it truly effective.

    On TRUST, I just think this is huge. It takes a village to raise a family, and without one not much gets done effectively. Kickstarter creates a village for your project and the inherent trust required is exciting to be a part of. The anonymity and forced trust of it all really cultivates a positive environment. I wonder if lasting ratings would take that away and make it “just another online marketplace”.

    John

    1. Thanks for your comment, John! I think the key here–particularly in regards to this post–is that it’s not like we’re making a conscious choice to trust creators who are like us. You might feel like you’re in control of it, but the survey indicates that’s not true. It’s a deeply rooted bias. So a rating/reputation system would help us fight that bias and determine our trust based on data instead of race, gender, nationality, etc.

  2. I can see it as problematic for the Creator rating the backer, except in fringe cases. It’s not really one on one similar to Uber, Ebay, Air BnB, for that kind of feedback. However for the backer evaluating the creator, it might help in the long term.

      1. You know, I haven’t really been in the creator seat yet, so it was pure speculation based on the idea that you have thousands of backers. From the Air BnB, ebay, Uber perspective, what value would you find from seeing backer ratings from other creators?

      2. I think you’ve just illustrated the problem, Jamey. You have a *lot* of interactions with your backers (as do we all). Rating all but the outliers would be time-prohibitive, and in general I think creators would only have time to down-check people.

        Perhaps more useful would be to allow everyone to rate everyone, so that backers could rate other backers. That would distribute the troll-detection workload.

        1. Sean: Well, that’s the thing–I don’t think backer ratings would be visible to other creators. They would only be for Kickstarter to see.

          Robert: I wouldn’t rate all backers. Mostly, I think, it would be used for me to identify destructive backers so Kickstarter can keep an eye on them. I think sometimes it’s just misplaced passion, but on rare occasions there are some really nasty backers that fall outside of Kickstarter’s “spam” rules.

          1. As much as I would like to never deal with certain backers ever again, I more strongly like the idea that all backers can voice their questions and concerns about a project without fear of repercussion. They are the customer, and we are obligated to listen to them and professionally address their issues as best we can. I don’t know, maybe I just haven’t encountered any of the truly destructive ones – just the kind-of destructive ones.

            If you’re just talking about flagging the truly destructive backers, then maybe something other than a rating system might be better, though I can’t think of anything that still couldn’t be used as a punitive measure against people speaking out their legitimate concerns.

          2. Isaac: That’s a great point, with one small caveat: Not every backer is there to be a customer. There are a very small number of backers who make a pledge just to make a quick, nasty comment, and then they cancel their pledge and leave. That’s the category of “backers” I worry about–I’d like to help identify them to Kickstarter. Perhaps this could be solves simply by expanding the bounds of the “spam” flag.

            Because overall, you’re right–I wouldn’t want a rating system to stifle backers from being a vocal part of the process and community.

        2. I was thinking the same thing about outliers. Personally, as far as a creator goes, I would think that there could be two options: 1) helpful contributor – someone who is always positive and answers questions, as long as they are mostly correct; and 2) troll/destructive/whatever…

          Otherwise, you don’t rate all the people who are just kind of there for the community.

          While I think that a rating system for the creators is a good idea in theory, could be difficult to implement well. Some people will rate a creator poorly for the smallest things, or for things completely outside their realm of control. Maybe one of those up/down-vote systems could be helpful. I always go and look at a creator’s past projects to see how they went. If this sort of rating system was well-done, it could save me a lot of time… ;)

          1. Michael: That’s a good point about outliers. BGG has an interesting solution for that–essential, in terms of the idea I presented, a creator’s rating would be diluted with 2.5 (out of 5) stars so that a few super low or super high ratings don’t skew the results. I think that could help, though I also think a few outliers in general probably wouldn’t ruin the dataset in this case.

  3. In principle I like the idea of rating; it could well reduce bias (by backers) and improve quality (by producers).

    I’m afraid however that the system would get mis-used, in that backers would use the rating to indicate whether they like what they got (e.g. is the game fun), instead of whether the producer kept their promises (was everything deliver on time and on spec).

    1. Wouldn’t part of a rating process be if a creator can not only deliver on time, but create a product you want and like?

      1. Yeah, I think the rating is representative of your overall experience with the creator. Kickstarter could nudge backers in that direction by telling them the rating is indicative of the whole experience–communication, quality of reward, timeliness, etc.

        1. Or you could have a rating type system with those (and possibly a few others) as different metrics that the backer could fill out like a standard satisfaction survey. That could be agrrigated into a single rating but available in a more granular form on the creator page or something. Right along the lines of that standard ‘rate these items 1-10’ satisfaction surveys you get all over the place.

  4. Allowing comments in addition to ratings would help the negative votes of nitpicky people not mean so much. It would also help as to how “real” the positive feedback is. When you’re blindly trusting someone, it tends to attract those types that always see the good in people anyways. As such, multiple 5/5 star ratings may not be accurate to determine a quality such as “was the project as promised?” rather than just “I was happy.”
    For negative reviews, when you see “shipped one month late” or “shipping cost too high to XX country” it helps you to decide for yourself whether or not the less than perfect rating is worth the risk.

    I don’t remember the game now, but there was one project on kickstarter that was well done as far as everything advertising went… but once the game was in the hands of players, it was nothing like advertised earning it a very low score on fun/playability. Even if you can trust the designer to deliver on time and the items listed, is it what you actually wanted? A rating system could make this easier than having to research each previous project to see what the results were.

    1. Vicki: That’s an interesting idea about the comments. I think I like that.

      I consider a product’s utility/fun to be a part of the creator’s promise, so I think that would be part of the rating (you’re rating your overall experience with the creator, the project, and the product).

  5. My concern about a rating system in Kickstarter is that unlike Uber, ebay or airbnb, there’s not a huge repeat environment.

    There aren’t a lot of kickstarter creators who are creating 10, 12 or 20 kickstarter projects…so there’s not a lot of history available to build a reliable rating system. Whereas, on ebay, a part time seller may put up 60+ listings a month…and airbnb hosts can lists dozens of times a year…

    ebay, airbnb, etc, I don’t trust ratings unless there’s a statistically significant number of ratings. Same with Apple App Store, amazon.com, etc.

    For example, Jamey, you have 8 projects created. That’s getting to “significance” levels that I could trust your rating as a creator. (For comparison CMON has put up 22 kickstarter campaigns, but I’m convinced they’re an outlier)

    Also, your “backer” rating doesn’t have a lot of relation to your “creator” rating. They’re two entirely different mechanics. A good backer doesn’t necessarily make a good creator. I’ve backed 200+ projects, but I doubt I could run a kickstarter campaign effectively. I’m just not setup that way.

    So, while I think ratings are an interesting idea, I don’t know that it would be that effective trying to setup a “web of trust” on kickstarter…

  6. kenm: That’s a great point about the lack of repeat projects. The system is only useful if a creator makes more than one project.

    But I think it’s more like Uber, ebay, and Airbnb than you indicated. The “repeat” nature of those ratings comes from the number of customers. That translates well to backers. Even though you’re just making and delivering one product, there are many backers that may have different experiences with that same project. Thus there’s tons of repetition within each project.

    That’s true about a backer rating not correlating to the creator rating–that’s not the purpose of why creators could rate backers. That would be a private rating seen only by Kickstarter to help them weed out destructive backers.

  7. Hi Jamey. Nice articulation of the issue, as always. Unsurprisingly a lot of strong opinions on this topic. Perhaps easy enough to sum up as “if the system in place works, it’s great for everyone…if it’s broken/abused, it’s bad for everyone”. Reviews of sellers/vendors/providers almost always shape my buying decisions, and I try to act in a transaction as if I would be reviewed to maintain a positive experience. I’d just hate to see Kickstarter end up with similar problems to BGG, with people rating a game a 1 out of 10 when they’ve never purchased it or played it, or they bought a worker placement game and they’re upset because they “don’t like worker placement games”. There are of course thoughtful solutions to prevent or mitigate all these problems.

    I think a potentially larger issue about prejudice relates to your first example…only trusting people “like you”. If the Cardboard Edison industry report (https://cardboardedison.com/reports/) is truly reflective of our current environment, most designers making games right now are OVERWHELMINGLY from North America, are male, are white, are 30-39, and have not published before. So yeah…that’s me :)

    1. John: Very well said. I think one of the benefits Kickstarter has is that you would only be sent the ability to rate the project if you were actually a backer and if the project is fulfilled. Conversely, on BGG, anyone can rate anything for any reason at any time.

      I’m glad the prejudice element stood out to you as well–that was my main motivation for writing this entry.

  8. Great post, Jamey. I really appreciate when the conversation can get to some of the “roots” of what it means to be human and how we interact with each other. Meaningful stuff.

    I think a rating system has both pros and cons, but there really is no way around that basic fact. Certainly your point is well made that “0 projects created” is already a sub-par rating system and could really benefit from some fuller articulation. Besides, even “10 projects created” does not automatically mean “10 projects happily and successfully fulfilled.” I think this is a great proposal.

  9. Geoffrey: Do you have any thoughts about how a creator with 0 projects created could have some sort of reputation indicator? I was thinking they wouldn’t have a rating until after they’ve completed a project.

  10. No, I really don’t. I think no matter what it’s going to be challenging for anyone who’s just starting out. Just the nature of the beast. All I was saying in the previous comment was that I agreed with you on the point that the phrase “0 projects created” can already create a negative predisposition on the part of the viewer, but I think it’s unavoidable. Everybody has to “prove themselves” to a certain degree, which takes time. :)

  11. Jamey, thanks for the thoughtful post. I appreciate what you’re trying to say, but I think there’s a component of these studies you may not be aware of.

    Yes, we trust people more when they are more like us. But in the case of Kickstarter that does not necessarily imply age, gender and race. It can also refer to having the same tastes or preferences as someone else. For example, I might take a liking to someone because they’re the only other knucklehead in the bar willing to drink Sambuca. (I like anise flavor. Deal with it. ;))

    In the same way you will find my Kickstarter history full of things that I like. Make a space-based board game? I’ll literally be climbing over myself to back it. Unless perhaps it takes 6 hours to play, which is why I really love backing small and/or short games like Mint Tin or Tiny Epic. (Tiny Epic Galaxies is a scary-awesome convergence of my preferences, BTW.)

    Games about the old west or Rome tend to be less appealing to me. Unless of course I already like and trust the developer. Which is why I’m all in on Tiny Epic Western.

    As scary as the research might be, don’t let it get you down. Kickstarter is well laid out to put the most important attributes first. In many cases I would argue that this works a little too well. For example, a large number of people backed the Unique Box without picking up on the (very obvious) fact that it was a scam. I myself almost fell for it in the excitement of how cheap x86 computing was getting. I thought I was supporting a real push to make computing that much more accessible.

    In case you’re still not convinced, I’ve backed LattePanda by Chinese developers, Dice Driving by Spanish developers, Small Star Empires by Macedonian developers, and I really wanted to back that geek dress Kickstarter. But I’m not a woman and my wife wouldn’t wear it. :-( I also bought an Ouya from a very public female CEO.

    In the end I have to agree with others that a rating system could have a very negative effect on Kickstarter. Sure you might up trust between strangers, but you might also escalate arguments and prevent strangers from having an incredible opportunity to know each other!

  12. Jamey,

    Great piece, as usual. I’m quite interested in the latter part of the conversation with Geoffrey as there’s a fairly large community on BGG which participates and reads the comments on the Kickstarter pages. By and large there’s a negative bias for those individuals who have “0 projects Created” coupled with “0 projects Backed” ¬ as that’s a strong indicator (not perfect, but verifiable) that the project may not go swimmingly.

    Unfortunately, this may be much ado about nothing…insofar as does the number of trolls evident on Kickstarter warrant a robust system for rating everyone? I don’t know, but it seems as though creators with cred (like yourself) should be able to identify these individuals to Kickstarter and have them police their site…just a thought.

    Cheers,
    Joe

    1. And it does raise the question of how it would affect community. I see your name and know that you are “Ambassador” Joe, designer of the Tau Ceti game. That gives me a warm and fuzzy because I know who you are and I really liked that campaign. Would that sense of community suffer under a ratings system?

      No one talks about the amazing eBay or Amazon community…

        1. I’m primarily thinking of backers rating creators (since that would be public), though I can think of creators who would create problems in the other direction.

          Once you introduce the concept of ratings you created a mechanism individuals can use to exercise leverage over others. No matter how well done it is, someone will threaten lower ratings if they don’t get what they want. And those types of threats are toxic to communities.

          (P.S. I’m the same guy who posted as EngineeringThings. The name is left over from a long time ago and I kept forgetting to change it. Unfortunately, WordPress isn’t propagating the change. Just wanted to reduce confusion. :))

    2. Joe: Thanks for your comment. That’s a fair point about the minuscule number of trolls. Having thought about it in these comments, I think that problem could be solved simply by expanding the definition of the spam/flag feature. Just a little bit. Because I think Isaac had a great point above about not wanting to scare people off from speaking their mind.

  13. Personally I think that a rating from 1-5 allows for too much personal bias to creep in and too much of a chance to rate things differently from each backer. If someone backs a project, never visits the comment page and gets their reward on time and as expected is that a 5? They got everything they wanted perfectly delivered by that creator could have been a monster on the comments page. Conversely if I post a lot of comments on a busy creator’s board and they don’t personally reply to all of them do I down grade them to a 4 or worse? And when does the system end, when I receive my reward? When I ask questions about a rules issue on my fourth play through and get a response or not? When I back the creator’s next project and change my feelings about them?

    Don’t get me wrong, I think a clear rating would be a good idea, but I think it should be more clean cut, did you fulfill your last project to the majority of your backers yes or no, and then you should get a trust check, just a little tick in the corner. Also one way of helping first time creators is to assume trustworthiness, you get the check until you prove you don’t deserve it.

    The thing about a complex rating system that allows you to dip in and read backers comments etc, is that its already there, you can click on all of a creator’s previous projects, you can check the comments board, you can even contact backers of their previous projects. If a backer is willing to really look into a complex rating system then looking at your previous project pages comments sections gives a far better picture than developing a new and complex rating system could ever do.

Leave a Comment

© 2019 Stonemaier Games