I am incredibly grateful for anyone who has ever playtested a Stonemaier game or expansion. Whether it’s a local playtest with me or a blind playtest somewhere in the world, every playtest makes a positive difference in developing the best possible version of a product.
Every few years I send a test to Stonemaier Ambassadors who are interested in becoming lead playtesters for our products. We’re blessed with around 130 lead playtesters (all of whom are paid and credited) and their playtest groups around the world, but I’m always looking for an array of perspectives about our games.
I sent this test recently, and now that I’ve processed the results, I thought I’d share some tips to consider if you’re looking to become a better playtester. Many of these are just as much the responsibility of the designer, publisher, and lead playtester as they are the various playtesters themselves.
- Share what happened, why it happened, and how it made you feel. Do not try to solve the problem or redesign the game–that’s the job of the designer and developer. The designer may request potential solutions, but they still need the context of what/why/how.
- Listen attentively during the teach. This may seem like standard gaming etiquette, but I think it’s especially important for a playtest, as playing something incorrectly could invalidate the data. If you do play something incorrectly, always mention it in the final report, as the developer may identify a pattern among playtest reports that will inspire a change in the rules.
- If you successfully break something, point it out and then stop using the broken element. A key part of playtesting is discovering broken elements, but continuing to use the broken element after discovering it can invalidate the rest of the playtest. You could, however, try the same broken element in different circumstances.
- Understand that opinions are subjective. Your opinions matter, but they are not facts, so present opinions as such. Rather than saying “This action is bad,” say, “I was never compelled to take this action because only receiving $3 didn’t seem as efficient as pursuing passive income. However, one of the other playtesters took that action several times because their character increased the benefit to $5.”
- Be specific, especially with frustrations and confusions. Even from experienced playtesters I still get some reports along the lines of, “There were a few cards that felt too expensive” instead of what I need, which is, “Cards 2, 8, 19, and 54 felt too expensive.”
- Offer both positive and negative feedback. It is just as helpful to know what to keep as it is to know what you would prefer to change or lose. I want to stress that BOTH of these are important, as I’ve had to stop working with some playtesters who only said nice things about games that I knew for sure were not very good at that stage of development.
- Use your intuition. One of the reasons blind playtesting is so important is that it’s a chance for people to learn the game from the rules–not the designer–just as everyone else in the world will do when the game is released. A great blind playtest is one that identifies gaps and unclear elements in the rules. However, in the moment–when you’re in the middle of the game and you can’t find an answer to a rule–the best lead playtesters will use their intuition to allow the playtest to continue (and make a note to share with the developer later).
- Follow the instructions and meet the deadline. Every publisher has different instructions–including confidentiality levels–so if this is the first time you’re playtesting for a publisher, make sure to read the instructions (and the playtest survey, if there is one) in advance.
Here are a few additions if the designer is present:
- Before you begin, ask the designer what they’re trying to get out of the playtest (balance, outliers, general feel/flow, etc). This is hopefully something the designer will say without being prompted, but if not, it’s helpful to ask.
- Wait until the end of the playtest to offer feedback (take notes). I notice at our Design Day event that eager playtesters–including myself–tend to offer feedback during the game. I think it’s perfectly fine and helpful to ask questions while you play to understand how to play and even why something is the way it is (maybe you’re missing something important), but a big part of playtesting is actually going through the motions of playing the game. There will be plenty of time for feedback when the game ends.
- Be aware of your mental state and how your demeanor is impacting a designer. The designer, is being vulnerable in showing you their game, so while they’re opting into receiving constructive feedback, they’re also still a human being.
One of our top lead playtesters pairs, Garrett and Allie, also added some feedback I really like (specifically for lead blind playtesters). In their words:
- Before asking the publisher for clarifications, first print, cut, set up, and thoroughly read the game rules. Often, questions raised while reading rules are answered explicitly or implicitly during setup and gameplay.
- Always take notes during the playtest. They help capture real-time reactions and observations.
- Before writing reports, discuss your notes and experiences with all playtesters. This ensures a comprehensive view of the game experience and offers a great opportunity to debate or compare experiences. Understand that everyone’s gaming experience might be different. Use feedback from all players to gain a holistic view, don’t just focus on YOUR feelings.
- Provide candid feedback, be respectful, and don’t worry about hurting feelings. Negative observations are more likely to lead to game improvements which is what the designer is looking for. Positive observations lets the designer know what’s working.
- Focus on detailing the experience, not the solutions. Describe events, why they happened, and the resulting feelings. Avoid offering design solutions unless explicitly requested by the publisher.
- Stay engaged! Approach each playtest with enthusiasm even if the game isn’t polished or fun. You agreed to do the playtest, try to do it as best as you can so the designer has actionable feedback.
- Always respect the confidentiality of the game. Ensure all playtesters are aware of this unless it’s an open playtest environment.
This is a living list; I’d love to hear your additions and thoughts in the comments.
Also read:
I also posted a related video just yesterday:
***
If you gain value from the 100 articles Jamey publishes on this blog each year, please consider championing this content! You can also listen to posts like this in the audio version of the blog.
31 Comments on “How to Be a Better Playtester”
Leave a Comment
If you ask a question about a specific card or ability, please type the exact text in your comment to help facilitate a speedy and precise answer.
Your comment may take a few minutes to publish. Antagonistic, rude, or degrading comments will be removed. Thank you.
How many times should the same group of people play-test before giving feedback? In my limited play-test experience, the first time through, I’m just learning about the game, not even trying to win, but rather to try things out. After a second or third play-test, I’ll probably have better feedback to give. I suppose it depends on what stage of development the game is at and what the designer says they are trying to focus on for the play-test.
We ask for data after every playtest, then a final report after the final playtest.
This blog post and a few of the ones that precede it, got me to thinking about the incredible level of two-way communication and engagement that Stonemaier has cultivated. I was excited and I so I reached out to the company to ask that Jamey discuss this in an upcoming blog post.
In doing so, I did something I know better than to do: I emailed the contact address and explained my idea. This is an idea that, to be honest, should be shared as a part of the public discourse, not thrown into an email where only a handful of people see it. This is something that was brought to my attention a long time ago, and I forgot. Very sorry about that — will not happen again.
That said, here is (a slightly edited version of) what I wrote:
=====
In the most recent blogs that Jamey has posted, there is an element that has come out in some of my comments that got me thinking.
Communication.
I have been active in the gaming industry for many, many years. In all that time, there has never been a game company that has as robust a set of tools and channels to facilitate communication between the company and the customer.
Recently, I have had difficulty getting in contact with no less than a half-dozen game companies. They have email addresses and online communications forms that when you write elicit no response. They have web pages that obfuscate or completely omit anything that might allow for communication to take place. They have pages or a presence on social media that does not engage with (and sometimes does not allow) two-way communication. These companies do not talk *to* their customers, they talk *at* them.
So I was wondering — could you write up a future blog post discussing the various ways Stonemaier Games interacts with its customers; potentially to provide some helpful guidance for other companies to help them facilitate this part of their business?
Thanks.
=====
The language I originally used in the end of this email was not the best. I have tried, here, to be more clear about what I am hoping to see come from this.
Anyway — thanks always to Jamey and Joe and the rest of the team for listening. You guys are amazing.
Thanks David! I appreciate you sharing this, and I love the idea for this blog post. I still feel like I have a lot to learn about creating and encouraging two-way conversations between me and customers, but I would be happy to work on a blog post on this topic to share what we do (and hopefully learn some things we can do better).
Jamey,
It’s great to see an article about playtesters, as I often feel as though they are often the unsung heroes of well made games. Having served as a developer for the past 12 years, one strong pillar of my work is playtesting and I’m happy to have served as a Stonemaier Playtester on a number of occasions and by extension for the Automa Factory.
I concur with all of the bullet points and would weigh particular elements more than others, such as focused playtesting; the good, the bad, and the ugly; and refrain from fixing. Focused Playtesting speaks to setting expectations and ensuring that you enter the playtest with the Designer’s needs (not yours) in mind. The “good” often includes identifying those elements that you and/or your fellow playtesters really enjoyed, “bad” identifies those areas that could use streamlining as the moves or actions are not fully developed, and the “ugly” are those things that, for instance, take you out of the theme, seemed tacked on, or other issue. As always, treat these issues with tact. Finally, take great notes, but leave the fixing to a developer or designer (sometimes, even a designer is too close to their work to remain objective).
If you haven’t served as a playtester, I certainly encourage you to do so.
Cheers,
Joe
We’ve playtested a number of Stonemaier games, and this will be a very useful rubric for me moving forward. I’ve often wondered if the manner of feed back, as well as the way I lay it out will be useful to the designer(s). Thanks for writing this article!
Thanks Jeff! I really appreciate you as a lead playtester.
I help run a monthly game development night with an amazing community. One thing I have found helpful is setting up a foundation for what we are looking for in the community and letting people know that early on. For example, we remind people that they are in a space to share their ideas, and that they can take their time doing so because we are there to experience other people’s creative thoughts. This in turn helps to cement that they should listen to others’ ideas when they are trying other people’s games and this has helped to create a more cohesive and creative atmosphere.
I think it’s important for a designer to be clear, from the outset, with what they are looking for from their play-testers. Some people want ideas all the way through because they are at the start of their development, whereas others just want to see how it plays. From participating in these I have now learnt to ask at the start what the designer is looking for and everything seems to go much smoother.
Thanks for sharing, Kerry! I like that you set the stage for a safe space for ideas, listening, and sharing.
Having been a playtester for numerous companies, for both board games and RPGs, I think it’s also important for lead playtesters to gather feedback thoughtfully. If there are four people discussing a play of a game, and someone gives their feedback, I make a note. But then I also ask the others a question like “What do you think?” And I make notes on all their feedback to give a thorough comment on real issues. I recommend not asking things like “do you agree with that last comment?” because that forecloses getting input from everyone. Sometimes the quiet members of a group make excellent points that might get overlooked.
I really like this! Finding ways to include all playtesters in the feedback process is a huge asset for a lead playtester.
A very interesting article. I think getting accurate feedback from play testers can be difficult to achieve. If you can build up a team of play testers who have worked for you before , Is a very valuable asset to have. Also by having a list of questions that you would like the play testers to answer will give you important feed back as to where your game might need a bit of tweaking. Every game will have unique characteristics of play and strategy so will need a unique questionnaire.
The first tip about not trying to solve problems is one I have heard before, but it’s one that puzzles me. I will offer a lot of suggestions when I’m playtesting. A lot of it is probably bad, but my view has been that it’s the designer’s job to filter out the good from the bad. And all the games I have playtested have had some of my suggestions in the finished game.
I should say that I have never done any blind playtests, but have mostly worked with designers I know, often very early in the process (sometimes the very first prototype that have not been played yet). I have also worked with designers I don’t know, and have been a little careful about what to say. But my experience has been that they have been very open and glad for suggestions.
I’m on the other side of the table now, and I don’t want any of my playtesters to be afraid of what they can say. So I don’t think this is a universal rule, but something that varies from designer to designer.
As a game developer, I actually do like to get recommendations from playtesters, predominantly because that helps me identify the scope of the change they think would be appropriate.
There’s a world of difference between adding 1 point of health and adding 20 points of health. “This character dies too easily,” is a valid critique, but a specific recommendation will allow me to get a better feel for the scope of change required.
Kjetil: If the designer invites potential solutions, they can be quite helpful (if the context of what/why/how is still given). However, from my experience with playtesters, what I’ve found is that if I don’t steer people towards focusing on what/why/how, they do not give that context–they’re thinking about solutions instead of describing the problem. So if you ever find yourself presenting an idea or solution while playtesting, first ask yourself, “Have I adequately shared what happened, why it happened, and how it made me feel?”
DC: I appreciate you sharing the example of “This character dies too easily,” because I don’t think that’s anywhere close to the level of detail I would ask of a playtester, especially before they start to propose solutions. I would want a detailed description of how that character died, exactly how many life points they lost from each monster, when in the game they died, etc. I think this is a great example of why I really want playtester to focus on what/why/how; otherwise I don’t get enough context and have less of a chance to solve the problem. Like, what if the problem isn’t the character at all–what if the game is just lasting too long or if that character happened to encounter some overpowered monsters?
Hi Jamey,
If you consistently read/listen to your content, like I do, it was pretty clear what the “correct” answers to the test were. Although I (fully) agree and understand your success criteria for being a good and valuable playtester, during the test, I also started to feel like I didn’t want to be the test taker that only filled in the desirable answers, so I considered purposefully answering one question “wrong.” I don’t remember if I actually ended up doing that, but I was wondering whether that was a system you built into your evaluation of the test.
Tonke: It might be a little less clear than you might think, as there were only 5 people out of the 140 ambassadors who recently filled out the form who answered all answers “correctly” (I say that in quotes because there’s at least 1 question that is correct for us but may not be correct for other publishers).
Oo, interesting! Self-overestimation is probably not a great quality for a playtester either, haha.
I am retiring from a career in medicine and would enjoy a challenge…how does a Uk resident get involved in this process
If there’s a publisher for whom you’re interested in playtesting, typically it’s linked to their volunteer system (or mentioned elsewhere on their website). If you’re looking to playtest in general, here’s a list of groups to get involved with: https://cardboardedison.com/playtest-groups
One thing I wish publishers did a better job of in blind play testing is briefing the play testers on what they’re looking for. Too often they provide the materials and a link to the survey where they request feedback,w/put instructions for what you should be attending to while playing. I shouldn’t have to click through tot he form to figure that out. Give me a bulleted list of what questions you want us to address during the play.
Seth: Thanks for your input! I’m not sure I agree, though. I think it’s reasonable to ask a playtester to skim through the form they’re going to fill out before beginning the playtest to see the types of questions the publisher has.
I have been in many playtests — mostly for role-playing games and supplements — but I have participated and lead playtests for board games (even for some games of my own design). All of these are things I wish I had known and had integrated into my practices way back when I started doing that stuff.
I just wanted to add (for the publisher): if possible, acknowledge the playtesters. Some recent games have put their names on the box, or in the rulebook, or on the games page on the publisher’s website. It doesn’t matter to most playtesters (that I have known; myself included) if money or merch changes hands. The most important thing is to feel appreciated. All we need is a pat on the back and an acknowledgement of our contribution to the final product.
The big thing here is to acknowledge all of the playtesters (if you can). Far too often I have seen a publisher cap the recognition to a fixed number of people, and so dozens get left feeling like only the first and loudest voices counted in anyone mind. As those leads are keeping notes, keep notes on who was there and who was actively participating in the spirit of the test.
David: I 100% agree that playtesters should be credited publicly. We typically do that on the sides of our box bottoms, but sometimes it’s in the rulebook.
Yep — you are one of the good guys in this regard! Thanks for all you do. I was certainly not suggesting that you were deficient in this area. It is just something that bugs me.
I have been in several playtests for a company where my comments and contributions would make it into the final book uncredited. Even fundamental areas of the rules and what-have-you. In one playtest, I was asked by the lead playtester to compile a whole bunch of data. It was a lot, so it took several days to compile, check, and double-check that data. I presented the data to the lead playtester, they used all of it, resulting in a whole new 16-page section in that book. In the end, I was not considered worthy of being in the top 10 contributors; so my contribution was left uncredited, yet again. It hurt.
After the final book came out, I made a comment on the company message boards. I was not complaining or anything; I just said that I was a bit surprised because I really thought I was going to be in the credits that time. The pile on that came after resulted in that being he last playtest I participated in for that company, and me leaving those message boards not long after.
So yea… this was not meant to reflect upon you. Just a reminder to publishers that when people take the time to read your material, actually test that material in multiple sittings, take the time to provide you with valuable feedback that contributes to the quality of your game or book or what-have-you… have the common decency to thank them. It will go a long way to maintaining a good stable of playtesters.
Absolutely! I didn’t interpret your comment as commentary about how we credit playtesters at Stonemaier Games–I was just sharing so others could see how we do it.
I’m really sorry to hear that you were treated that way. Mistakes do happen when compiling playtesters lists, but I really don’t see any reason to arbitrarily credit some playtesters and not others.
When it comes to thinking I’ve found something broken, I do sometimes like to retest that theory with someone else trying that combination. This way we see if it was more a matter of a more experienced player or if things just worked out the previous game. When something happens once, it could just be a coincidence. I do think there is value in adding a second piece of evidence to show a combination might be broken, much like I tell my students that they need more than one piece of evidence to prove a point.
At least, that has been my thought when play testing.
I think that type of evidence is really helpful, Carol. Though from a design/development perspective, I’d still want to know even though single circumstance in which an ability is broken.
Definitely! Not reproducing it doesn’t negate the narrative of the original play. Just interesting to try to see if it happens consistently or is an occasional fluke of combinations.
Very interesting! I manage a designing/testing team in the federal government and many of these applies. How many times we get a design done but then it’s not being test properly and the tester cannot explain exactly the situation and how they think it might have happened. I often tell them, out yourself in the shoes of the designer. What do you think you’d need to fix it.
Thanks for sharing!
Maybe one day I’ll become a play tester for my fav company
Thanks Valerie! I like the idea of trying to put yourself in the shoes of the designer.