My latest game, Sun Stones, is out for iOS and Android/Amazon/etc. I've spent a few exciting days tracking my analytics, viewing order history, etc. I also now have tens of thousands of play-session data points, which allows me to look at user behavior in some very nice specificity.
I want to talk about one specific feature path in Sun Stones - they way you play/win/finish a puzzle. Sun Stones is a puzzle game, which means that you solve a sequence of puzzles. I followed my "Two Curves of Engagement" strategy, and built a nice aesthetic layer on top of the sequencing, so players who start the game can be pulled in by the aesthetics long enough to start to enjoy the puzzles on an abstract level. In Sun Stones I use a Hopi Glyph series, which plays like this:
1 - Upon starting the game, players are presented with a glowing stone, and a sun.
2 - Players move the stone by touching and dragging on the screen - they place it on the sun.
3 - Intro animation starts, logo, sound effects - nice reward for figuring out the first puzzle (which was "move the stone onto the sun to start the game.") This sets players up for the presentation style - no words, no explanation, but a simple, logical building upon past knowledge.
4 - Players are now presented with a "Glyph Wall" - which acts like a level select screen. There is a cave-painting-style sketch on the screen - faded and incomplete. One glyph is pulsing. Touching that glyph enters that first puzzle. Eventually all of the glyphs on the wall will appear, and the picture will be completed - allowing players to scroll the wall to find more glyphs to play.
5 - Players are brought to a closed eye, a puzzle name "Sunrise" and the first bit of text they have seen: "Open your eyes, your journey begins when you see the rising sun."
6 - Players must touch the screen, and drag open the eyelid. This starts the first puzzle. The top of the eyelid remains on screen at all times - players may drag the eye closed again to check hint text, and other useful data.
7 - In the puzzle proper, players must move stones onto black squares. This is mechanically trivial, but it might take them a few seconds to understand what to do.
8 - We now play the victory sequence - which closes the eye and brings up some brief information - how many moves did the player use to match all of the stones to the pattern? How did that number of moves compare to the bronze, silver, and gold rankings?
9 - Players are now returned to the glyph wall - and a new glyph is pulsing.
Now, that was a rather verbose description of something which generally takes about 30-40 seconds for players to experience. And in my face-to-face playtests players often have questions or comments to the effect of "That was pretty tricky" or "I think a lot of people would have trouble with that" or "It wasn't clear what I was supposed to do."
So what does the data say? I can't actually track "player understood how to play the game" but I can track the number of new users each day, and the number of unique players who get a gold rank on at least one puzzle. I think it's safe to say that a player who gets a gold rank on at least one puzzle has successfully navigated into a puzzle. With 4 days of data reported, here is what I can say:
Day 1 - 91%
Day 2 - 87%
Day 3 - 93%
Day 4 - 92%
So at the very worst 90%+ of users are earning gold stars on puzzles, which lets me say with some authority that people are absolutely getting into the game without significant difficulty.
This actually sounds just about perfect to me - I WANT the game to seem tricky, to give people a sense of accomplishment. An ideal game is one where a player is able to beat a section just barely - and feels as if most other people could not have replicated the accomplishment. (Of course, if everyone feels that way it's obviously not true. But it's still nice for players to feel that they have made a significant leap by performing the task.)
Falling into the Canyon
Imagine that you are giving tours of the Grand Canyon. People are lining up and paying you to walk them along a path at the bottom of the canyon to an exit point. (Perhaps a raft is better, since the Grand Canyon ends in a river - but let's just run with it.)
But there is a problem - a significant number of people are getting to the bottom of the canyon, then realizing that they've left their camera, or medicine, or some other critical thing back at the top. It's very inconvenient for everyone to wait for them to walk all the way back up, get their item, and then come back down. Folks suggest that the park pay to build an escalator from the bottom of the canyon, to speed up these incidents.
This is a terrible idea.
It's terrible because the problem is NOT the fact that it takes forever for someone to climb back out of the canyon. The difficulty of access is a huge part of appeal of the canyon! The problem which needs to be addressed is the fact that some significant number of people are entering the canyon unprepared. Building an escalator does nothing to help make people more prepared - in fact it will quite likely increase the number of unprepared passengers, because the penalty for forgetting something has now been reduced!
The proper fix to the situation is to help people double-check their supplies before they come down the canyon in the first place. This might just be a sign which reads "Do you have everything you need?" or perhaps better would be a warning sign like "WARNING: Once you go down, you will not be able to come back up for lost or misplaced items."
Build a Railing, not an Escalator
I call this principle "Build a Railing, not an Escalator", because it is almost always the case that preventing an error is better than facilitating people who have made an error. In early version of Sun Stones our testers complained that when they entered the wrong puzzle by accident (touching the wrong glyph on the wall) there was no easy way to leave the puzzle. Our solution? When you touch a puzzle you have already played, a box pops up showing your record, earned stars, and asks if you want to re-play that puzzle. We tried an "exit" button within the game, but it make puzzles seem not very important - like we were always pushing you to just quit the puzzle without solving it. It reduced engagement for players when they entered a puzzle, and led to shorter overall game sessions (3-5 puzzles, instead of 8-10).