[0:00–0:48] Introduction:

Hi, I'm Chris Wilson. Veteran game designer Mark Rosewater once said "Your audience is good at recognising problems and bad at solving them." Today I want to give some advice to the game developers in my audience: don't survey your players to help you decide your game's design direction. Now, that probably sounds strange, because asking your community how to improve your game seems completely reasonable. And to be clear, I'm not saying you shouldn't listen to your players. Of course you should. You should solicit feedback about specific topics, watch how people play, pay attention to complaints, and take reported problems seriously. But there's a big difference between listening to feedback and asking players to help you decide what your vision for the game should be. In this video, I want to talk through four reasons why surveying players can be dangerous. The first problem is that it makes you look like you just don't know what you're doing.

[0:48–1:31] Lacking Vision:

As the game's developer, you should have a clear vision of where the game is heading. And if you don't, you should fix that while at least projecting the impression that you have one. When you ask players to choose between possible game features in a survey, it can make it seem like you just don't have that vision. It gives the impression that you're unsure about what matters, or that you don't understand your own game well enough to assess what's most important. Now, sometimes your team does have a plan, but someone elsewhere in the company, or at your publisher, decides to be really helpful and to do some market research for you via a survey. But if that survey leaks, players aren't going to know who wrote the questions. They'll assume those questions reflect what the dev team is seriously considering. And if the questions seem out of touch, then your team seems out of touch. The second problem with surveys is that they create expectations.

[1:31–2:32] Creating Expectations:

The moment you put an idea in front of your players and ask them to weigh in on it, you imply that it's on the table. You imply that it might happen. Why else would you include it in the survey? Once players believe that the only thing stopping an idea from becoming reality is persuading you to go ahead with it, they'll campaign for it and expect it. This is very different from players independently asking you for a feature. Banding together and pushing for an idea on their own is normal feedback. But if you asked them about it first, then not following through will feel like a broken promise. They'll feel ignored and misled. Even worse, some players may answer based on what benefits them personally rather than what's best for the game overall. They may vote for whatever helps their build, their playstyle, or their short-term enjoyment, even if that would damage the long-term health of the game. If you make a habit of surveying players, you can train your community to expect consultation. Then the next time you make a decision on your own, people may ask why they weren't consulted first. You'll gradually give up your freedom to act decisively without backlash.

[2:32–3:37] Useless Results:

The third problem is that surveys often don't give you information that's actually useful. Players are often very good at telling you that something feels frustrating, unrewarding, unfair, or confusing. And that kind of feedback is incredibly valuable. But the solution they propose is often not the right one, because unlike you, they don't have to balance the entire game. This is the trap with surveys: they often ask players to choose solutions when what you really need to discover is the problem. As any statistician can tell you, it's actually extremely hard to design a good survey. Wording affects outcomes. Answer choices affect outcomes. The kinds of people who respond affect the outcomes. The players who fill out surveys may skew towards the most engaged, upset or opinionated players, missing out the many people who really just don't feel strongly enough to fill out the survey on that given day. And this skews the results also. Your survey tells you that 50% of players reached maximum level in your game, but your own database tells you it's less than 1%. Your survey gives you a pretty spreadsheet of data that looks clean and insightful, but it may tell you far less than your own game data, direct player observation and common sense already can.

[3:37–4:27] Making Worse Decisions:

The fourth problem is that bad survey results can push you into worse decisions. This is the part that worries me the most. If the sample is biased, the answers are self-interested, the questions are focused on solutions instead of problems, and if you've already started giving up on the perception of having a strong vision, then the survey not only fails to help you, but may actively pull you in the wrong direction. You start optimising for the loudest players instead of the whole community. You start reacting to short-term discomfort instead of thinking about long-term health. You start drifting toward safer, more obvious ideas instead of making the stronger creative decisions that your game may actually need. A lot of players naturally argue for immediate comfort: more rewards, less friction, less punishment, more power, more convenience. And sometimes, of course, that is the right answer. But sometimes the things players push back on in the short term are exactly the things that give a game depth, meaning and memorability over time. That's why the design direction has to come from you, the developer.

[4:27–5:30] Conclusion:

I'm not trying to say "don't listen to players." Obviously you should. But there's real danger in asking players to tell you how to change your game. So how should you get feedback from players instead? Well, start with finding out about the game's problems, not the proposed fixes. Watch people play. Read what frustrates them. Ask targeted questions about specific pain points. Your players are excellent at telling you when something feels bad, confusing, unfair, or unrewarding. What they are not responsible for is balancing the whole game or protecting its long-term health. That part is your job. Don't ask players to choose your direction for you. Ask them whether they're having the experience that you intended. Because when you ask players how to improve the game's design in broad, solution-seeking surveys, you risk looking like you lack vision, you create expectations, you gather distorted information, and you can end up making worse decisions for the game as a result. Players are very good at telling you where the pain is. It is your job to decide what the cure is.