The Santa Tracker app for Android is a Google holiday tradition. Every year, millions of people around the world use the app to play games with elves and reindeer and, of course, track Santa, as he flies around the world on December 24th. While the app is live for a few months each year, about 90% of our usage occurs in the last two weeks of December. In order to turn around improvements to Santa Tracker quickly over this time, it's critical that we can monitor and adjust the Santa Tracker app remotely. This year, we decided to go all-in with Firebase as our monitoring solution. In this blog post, I'll talk about how we use a combination Analytics, Crash Reporting, and Remote Config to maintain a high level of quality, without ever having to republish the app.
As users navigate through the app we use Firebase Analytics events to record their behavior. Most of the mini-games in the app live in their own Activity classes, so we can use Firebase Analytics' automatic screen tracking feature to record these events without writing any code.
For events within games we use custom events to record important user actions. For example after the user finishes playing the "Penguin Swim" game, we record the event swimming_game_end with custom parameters score and num_stars. In the first week of December we noticed that 85% of users were getting zero stars when playing the Penguin Swim game. Clearly, the game is too hard, we were hoping that only 60-70% of users would get a score this low! We were able to correct this using Remote Config, which I'll talk about later.
swimming_game_end
score
num_stars
The other feature of Analytics that we put to use is user properties. At the start of each Santa Tracker session, we use user properties to record some information about the user's device. These properties are then attached to every analytics event. Since Santa Tracker is used all over the world, we get a lot of diversity in the devices people use. These user properties help us to make sense of our analytics data. Some examples are:
API_LEVEL
DEVICE_BRAND
DEVICE_BOARD
The combination of our custom events and user properties with Firebase Analytics' automatically tracked events enables us to get a good understanding of what our users are doing in the app by looking at the Firebase console.
Despite our best efforts, the Santa Tracker app is not perfect. With millions of users on hundreds of device types in dozens of countries we are constantly discovering new bugs in the wild. Firebase Crash Reporting lets us see all of the fatal errors in our app within a minute of their occurrence. Since Firebase Analytics events show up in Firebase Crash Reporting logs we can see the progression of events before the crash which was very helpful in diagnosing some issues.
For example there's an OutOfMemoryError crash which seems to happen during the "Penguin Swim" game on some low-RAM devices. We did not see this error during our testing, but the Firebase Analytics data in Crash Reporting tells us that this occurs when playing the game repeatedly.
OutOfMemoryError
This integration is invaluable in helping us to reproduce issues that our normal QA setup does not find. We can get the exact device model and then use the analytics log to recreate the crash conditions.
Once we have analyzed the data from Analytics and Crash Reporting, we need to make changes in the app to improve the user experience. Due to the short active life span of this app there's no time to go through the full development lifecycle of the app to publish changes, and we don't get a second chance at Santa's big day!
Santa Tracker uses Firebase Remote Config to gate access to various features, and to provide remote fine-tuning for experiences in the mini game. For example, in the "Penguin Swim" game, there are two key variables we store in Remote Config:
SwimmingObstacleDensity
DisableSwimmingGame
As mentioned earlier, users were having a hard time getting a score higher than zero stars in the game. In order to make the game more fun, we changed SwimmingObstacleDensity from 1.5 to 1.1, which made it much easier for users to dodge obstacles. By making the game easier in this way, the percentage of users getting 0 stars went down from about 85% to 70%. This change took place instantly over the air, with no need to publish a new version of the app!
Right now the OutOfMemoryError in the swimming game happens for <1% of users. But if this issue became rampant, we could use the DisableSwimmingGame flag to immediately hide the game from affected users whilst we resolve the issue. By taking advantage of the fact that Analytics user properties can be referenced in Remote Config, we can even disable the game only for certain device types! For example, let's say the Penguin Swim stopped working on all KitKat devices (API level 19).
First, we add a condition based on user properties:
Next, we disable the game only for users who match the condition:
Now the game will only appear for users who will have a stable experience, which will lead to fewer crashes for our users and more positive app ratings for us.
Adding deep Firebase integration to Santa Tracker gives us the ability to monitor and fine-tune the app over time without releasing app updates. As developers, it's invaluable to have a clear picture of what our users are really doing and how we can improve the app. Throughout December we knew we could rely on Firebase to give Santa Tracker users a magical holiday experience.
