Sorting 63 Player Comments Into Fixes: A Feedback Devlog

Two weeks after we shared the demo of Lantern Run, our little one-button platformer, the project page had 63 comments. That sounds like a nice problem to have, and it was, right up until I sat down to decide what to change. Some comments said the game was too hard. Some said it was too easy. One person wrote four separate comments asking for a dog. Reading them top to bottom left me with a vague feeling that everything needed fixing and no idea where to start.

This devlog is about how we turned that pile into a short list of changes, and which comments turned out to matter. If you have ever shared a Scratch project and then frozen in front of the replies, the method here is small enough to copy in an evening.

Why reading comments in order misleads you

Comments on a Scratch project page show newest first, and every comment caps out at 500 characters. Both details shape what you take away. The newest comments feel most urgent because they are on top. The short limit means people write reactions, not explanations. "too hard lol" tells you a feeling. It does not tell you where.

The bigger trap is volume. A single player who writes four times sounds like a crowd. Eight players who each mention the same thing once, in different words, barely register because no two of them look alike. When I went back and counted, the loudest request in the thread (the dog) came from one account. The quietest real problem came from eleven different people, and none of them used the same words to describe it.

So the first rule we set: never decide anything from scrolling. Copy everything out first.

The spreadsheet, and the five tags

I pasted every comment into a plain spreadsheet with three columns: the username, the comment text, and a tag. Usernames matter because they let you collapse repeat posters into one voice. Replies from us were left out.

Then each comment got exactly one tag. We tried more tags at first and it turned into a mess, so we settled on five:

• Bug: something that behaves in a way we did not intend.
• Confused: the player did not understand what to do or what something meant.
• Difficulty: too hard or too easy, with or without a location.
• Wish: a feature or content request.
• Praise: positive with no actionable detail.

Anything off-topic (remix requests, "follow me", a question about a different project) got its own pile and was ignored for this exercise. Mira and I tagged the list separately and then compared. We disagreed on nine comments, almost all of them between Confused and Difficulty, which turned out to be the most useful disagreement of the whole process.

What the numbers looked like

The second column is what you would feel from scrolling. The third is what actually happened. The wishes collapse from seven comments to three people, the dog fan accounting for four of them. The confused column does not shrink at all: eleven separate people, each saying it once.

The confusion that looked like difficulty

When we read the eleven Confused comments side by side, nine of them were about the same thing: the lantern. In Lantern Run, the lantern on your back slowly dims, and when it goes out, you restart the level. We thought that was obvious. It was not. Players read the dimming as a lighting effect, not a timer, and then felt cheated when the level reset with no warning.

Here is the interesting part. When we went back to the Difficulty pile, five of the "too hard" comments mentioned level 3 specifically, and level 3 is the first level long enough for the lantern to run out if you dawdle. Those players were not failing jumps. They were running out of light and thinking they had died for some other reason. One of them wrote "i keep dying for no reason on 3", which, from their point of view, was exactly right.

So a single fix addressed what looked like two separate problems. We added a flicker and a short crackle sound when the lantern drops below a quarter, and on level 1 the lantern now dims fast enough that every new player sees it run out once, safely, before it can cost them anything.

The most useful comments were the ones that described a feeling in the wrong words. "Too hard" sometimes means "I did not know the rule."

The real bugs, and the one we almost missed

Six players reported bugs. Four of those were the same one: at high speed off the long slope in level 4, the player sprite could pass straight through a thin wall. This is a classic Scratch problem. If a sprite moves 12 steps in one frame and the wall is only 6 pixels thick, the sprite can be on one side of the wall in one frame and on the other side in the next, and the touching check never fires.

The fix was to split movement into smaller steps. Instead of changing x by the full speed once per frame, the script now repeats the speed value as a loop of one-pixel moves and checks for the wall after each one. It costs a little performance, but Lantern Run is small enough that we could not measure a difference on the school Chromebook we test on.

The bug we almost missed was reported once, by one player, in a comment that read mostly like praise: "love it!! the music stops after you restart a few times tho". It got tagged Praise on the first pass. Mira caught it during the comparison. It turned out the music loop was starting a new copy of itself each restart and eventually hit a sound limit. One player, one sentence, and it would have affected everyone who played for more than about ten minutes.

What we did with the wishes

Nothing, yet, and I want to be honest about that. Three people asked for things. One wanted a dog companion, one wanted a level editor, and one wanted a two-player mode. All three are reasonable. None of them fix anything that is broken, and a level editor alone would take longer than the whole demo did.

We replied to each of them, thanked them, and said the idea was on our list. We did not promise anything. I have seen small studios promise features in comment replies and then spend months either building something they did not want or apologising for not building it.

For more on putting an early build in front of people in the first place, we wrote about playtesting with friends, and the comments thread is really the same exercise at a larger scale. Our guide on sharing your game so people play it covers how we got the 63 comments to begin with.

If you have a comment thread waiting

Pick your most-commented project and copy every comment into a spreadsheet tonight, with the username in its own column. Tag each one with only one of the five tags above, then count how many different players sit behind each tag, not how many comments. Read the smallest real pile slowly, because that is where we found our worst bug. If you want to see where Lantern Run goes next, the rest of our build notes live in the devlogs section.