Whoa! Is it the end of the year already? It looks like 2018 will be upon us before you know it; but I'm sure like most of you, it's going to take me three months before I remember to stop writing 1513684800 on all of my timestamps.
Looking back at 2017, I can say it's been an exciting year for all of us here in Firebase land. We launched a whole boatload of new features, attended dozens of conferences, welcomed several dozen new team members, met hundreds and hundreds of developers from all over the world, and had one crazy dance party.
I don't know if any year could have topped everything we accomplished in 2016 in terms of new features, but 2017 certainly gave it a run for its money. Here's a quick overview of just some of the great new products we released over the past year.
We released the beta version of Cloud Functions for Firebase, which lets you run server-side code in response to changes that happen within your app. I know the team who worked on this was very excited to see it launch, and this really helps us realize the whole "serverless computing" vision that Firebase has been working towards for the last several years.
For all you game developers out there, the C++ and Unity SDKs graduated from Beta to official General Availability status. Woo hoo!
We also announced support for games for Firebase Test Lab for Android, so you can help squash a few more bugs before your game hits the Play Store.
Over in the world of Analytics, we released StreamView and DebugView to all of our developers, along with the ability to create reports for custom parameters, and worked closely with the AdMob team to help you see important data about your ad revenue directly within the Analytics console.
We made several dozen new best friends in San Francisco and Cambridge by joining forces with the Fabric team.
This has resulted in some nice improvements to Firebase, including adding Crashlytics to the Firebase Console, and creating some new reports for the Analytics dashboard. We were also able to add Phone Authentication for Firebase, which allows your users to sign in to your app through a simple text message.
We began the process of open sourcing many of our client and server SDKs. We also released the Admin SDK for Go, for all of you Go fans out there (who, if they're not calling themselves Goficionados, are missing out on a golden opportunity here1).
We announced Firebase Performance Monitoring, so you can make sure your app is performing well out in the real world, not just at work with your fancy office wifi.
We introduced Cloud Firestore to the world, our highly scalable document database in the cloud. Cloud Firestore contains many of the same features that made the original Realtime Database successful, along with a number of improvements -- including a more sophisticated querying language, shallow queries, and a number of scalability improvements on the backend.
We added platform specific overrides to Firebase Cloud Messaging, so you can customize messages for your iOS, Android, or web users while still writing one single call to the FCM service.
To help you make smarter and more data-driven decisions around your app, we announced the beta version of Firebase A/B testing. This feature, in conjunction with Remote Config and Notifications, allows you to A/B test both app features and push notifications, so you can drive growth in the metrics that matter to your team.
And what Google product would be complete without machine learning? Just a few months ago, we announced Firebase Predictions. Predictions allows you to create dynamic groups of users who are predicted to churn, spend money in your app, or trigger special conversion events, so you can deliver unique experiences just to them, either through Firebase Remote Config or Notifications.
Phew! That was a lot! And while all of these new features and product announcements are great, 2017 wasn't just about new launching new products; it was about meeting all of you, too!
Some of our favorite moments came from attending developer events around the world. For a lot of our product team, getting the chance to meet with real developers, hear about how they were using Firebase, and all of their experiences, both good and bad, was thrilling and educational.
The Firebase team went to over 136 events in over 120 countries all around the world. And this included everywhere from Kuala Lumpur to New York City...
...to Manilla…
...to Google I/O down here in Mountain View...
….to Japan…
...to Washington DC2, to, of course, our big Developer Summit in Amsterdam. Fun fact: One attendee decided to bike all the way from Poland to Amsterdam just to see us. I think he earned an extra stroopwafel for that.
How were we able to accomplish this amazing task while still getting our work done? Mostly because we had Casey stay home and do all the work for us while we were gallivanting around the world. Thanks, Casey!
And some of our favorite moments didn't have anything to do with official work duties at all! Here's a few other highlights from Firebasers all around the world.
The San Francisco team continued its traditional annual observation of Take Your Pineapple to Work Day, because we believe it's important for all pineapples to discover their true potential.
Firebase hosted its second annual hack week, where Firebasers from around the company worked on feature improvements or fun side projects using the Firebase platform. We saw a number of fun hacks, including a coffee shop kiosk, a video doorbell, and some nifty improvements that might make their way into Firebase security rules someday soon.
Sam Stern dressed as an elf. And somehow managed to still look intimidating.
Amber from the Cambridge office had the privilege of hosting the Cambridge Women Techmakers conference and emceeing a panel with some of the area's top female entrepreneurs.
We all made it through the eclipse with minimal eye damage.
Alex -- our Cloud Firestore Product Manager -- had a baby! Also, Alex -- our Cloud Firestore Technical Writer -- is expecting a baby! Basically, if you're named Alex and working on the Cloud Firestore team… you might want to invest in some onesies.
Speaking of which, Jake on the Fabric team also had a baby! So cute3!
And Jake isn't alone. The folks in the Cambridge office had 7 babies total in 2017!
The Crashlytics and Performance Monitoring teams went to an offsite where they learned about street art and made a kick-ass Crashlytics logo. Because when you're building a crash reporting tool, the most important thing is to have street cred.
In cute animal news, the team in Cambridge got a new puppy, allowing us to meet our ambitious belly-rubs-per-employee goal for Q4.
Abe rescued a stray kitten he found in an alley behind his house. He's hosting the kitten right now at his apartment, and while he claims this is only temporary, it's been that way for three months. So, to summarize: Abe adopted a new kitten and it's his forever.
And Annum in Cambridge adopted Juno! Juno is a Maine Coon cat, which is generally known as the largest domesticated cat breed.
Anit on the Crashlytics team, worked the Firebase tent at I/O and got married! He won't tell me which event was more exciting, but let's be honest; you don't get a cool "Staff" T-shirt at a wedding, amirite?
The Cloud Functions team built a fireplace in the office. Although after some discussions with the fire marshall, who's a total wet blanket4 when it comes to things like open flames, they had to settle with a video fire.
Rich shaved his beard for charity. I'm pretty sure he raised a dollar for each whisker shaved.
Firebase hosted its very own Gingerbread House contest. Along the way, we learned that gingerbread baked specifically for constructing houses is only slightly less tasty than actual drywall.
With that, most of us are taking several days off this holiday season to spend some quality time with family and loved ones. We look forward to starting up again in 2018 with renewed vigor and more ways for you to build successful apps.
Until then, we hope you take some time off as well to relax and recharge, and have a Happy New Year, from all of us on the Firebase and Fabric teams!
I'll also accept Go-urmands ↩
At our Washington DC event, we had both an 11-year-old and a professional opera singer write their very first web apps on top of Firebase. ↩
I'm referring specifically to the baby here. Jake's cuteness is a private matter between him and his spouse. ↩
Which is pretty apropos, if you think about it ↩
Hey, welcome to part 3 in this series about using lifecycle-aware Android Architecture Components with Firebase Realtime Database. In part 1, we started with a simple Activity that uses database listeners to keep its UI fresh as data changes in the database. We converted that to use LiveData and ViewModel to remove the boilerplate of dealing with the listeners during the Activity lifecycle. Then, in part 2, we completely refactored away all mention of Realtime Database from the Activity, and implemented a performance enhancement. This optimization uses MediatorLiveData and some threading, for the case where data manipulation might be too expensive operation to perform on the main thread.
MediatorLiveData
There's one more optimization that can be applied in the code. It could have a large impact on performance, depending on how much data your database listeners are receiving. It has to do with how our FirebaseQueryLiveData implementation deals with the database listener during its onActive() and onInactive() methods. Here it is again:
FirebaseQueryLiveData
onActive()
onInactive()
public class FirebaseQueryLiveData extends LiveData<DataSnapshot> { private static final String LOG_TAG = "FirebaseQueryLiveData"; private final Query query; private final MyValueEventListener listener = new MyValueEventListener(); public FirebaseQueryLiveData(Query query) { this.query = query; } public FirebaseQueryLiveData(DatabaseReference ref) { this.query = ref; } @Override protected void onActive() { query.addValueEventListener(listener); } @Override protected void onInactive() { query.removeEventListener(listener); } private class MyValueEventListener implements ValueEventListener { @Override public void onDataChange(DataSnapshot dataSnapshot) { setValue(dataSnapshot); } @Override public void onCancelled(DatabaseError databaseError) { Log.e(LOG_TAG, "Can't listen to query " + query, databaseError.toException()); } } }
The key detail to note here is that a database listener is added during onActive() and removed during onInactive(). The Activity that makes use of FirebaseQueryLiveData executes this code during its onCreate():
onCreate()
HotStockViewModel viewModel = ViewModelProviders.of(this).get(HotStockViewModel.class); LiveData<DataSnapshot> liveData = viewModel.getDataSnapshotLiveData(); liveData.observe(this, new Observer<DataSnapshot>() { @Override public void onChanged(@Nullable DataSnapshot dataSnapshot) { if (dataSnapshot != null) { // update the UI here with values in the snapshot } } });
The observer here follows the lifecycle of the Activity. LiveData considers an observer to be in an active state if its lifecycle is in the STARTED or RESUMED state. The observer transitions to an inactive state if its lifecycle is in the DESTROYED state. The onActive() method is called when the LiveData object has at least one active observer, and the onInactive() method is called when the LiveData object doesn't have any active observers. So, what happens here when the Activity is launched, then goes through a configuration change (such as a device reorientation)? The sequence of events (when there is a single UI controller observing a FirebaseQueryLiveData) is like this:
Activity
LiveData
I've bolded the steps that deal with the database listener. You can see here the Activity configuration change caused the listener to be removed and added again. These steps spell out the cost of a second round trip to and from the Realtime Database server to pull down all the data for the second query, even if the results didn't change. I definitely don't want that to happen, because LiveData already retains the latest snapshot of data! This extra query is wasteful, both of the end user's data plan, and and counts against the quota or the bill of your Firebase project.
There's no easy way to change the way that the LiveData object becomes active or inactive. But we can make some guesses about how quickly that state could change when the Activity is going through a configuration change. Let's make the assumption that a configuration change will take no more than two seconds (it's normally much faster). With that, one strategy could add a delay before FirebaseQueryLiveData removes the database listener after the call to onInactive(). Here's an implementation of that, with a few changes and additions to FirebaseQueryLiveData:
private boolean listenerRemovePending = false; private final Handler handler = new Handler(); private final Runnable removeListener = new Runnable() { @Override public void run() { query.removeEventListener(listener); listenerRemovePending = false; } }; @Override protected void onActive() { if (listenerRemovePending) { handler.removeCallbacks(removeListener); } else { query.addValueEventListener(listener); } listenerRemovePending = false; } @Override protected void onInactive() { // Listener removal is schedule on a two second delay handler.postDelayed(removeListener, 2000); listenerRemovePending = true; }
Here, I'm using a Handler to schedule the removal of the database listener (by posting a Runnable callback that performs the removal) on a two second delay after the LiveData becomes inactive. If it becomes active again before those two seconds have elapsed, we simply eliminate that scheduled work from the Handler, and allow the listener to keep listening. This is great for both our users and our wallets!
Handler
Runnable
Are you using lifecycle-aware Android Architecture components along with Firebase in your app? How's it going? Join the discussion of all things Firebase on our Google group firebase-talk.
We're happy to announce that the Realtime Database has integrated with Google Stackdriver! This integration allows you to monitor your Realtime Database in powerful new ways. Here are some of the highlights:
Several metrics are already available in your Firebase Console under simpler names. For example io/utilization is "Load", storage/total_bytes is "Storage", network/sent_bytes_count is "Downloads", and network/active_connections is "Connections". These metrics form a great base, but now we you can go further so can can closely monitor your application as it scales with a whole collection of new, in-depth insights.
io/utilization
storage/total_bytes
network/sent_bytes_count
network/active_connections
To get started, check out https://console.cloud.google.com/monitoring to create a Stackdriver account for your Firebase projects.
To set up a graph of a new Realtime Database metric, go to Dashboards > Create Dashboard in Stackdriver, then click on the Add Chart button in the toolbar.
This example is replicating the "Load" graph in your Firebase Console, except we've improved it by also breaking down the data by "operation type". With this detailed view, you can see how long it takes your database to respond to REST "get" requests, versus how much REST "put" or realtime "set" operations are taking up your database's capacity.
We also have a few other key metrics we think you'll love, such as network/https_requests_count which tells you how many requests to the database require a full SSL handshake, network/sent_payload_and_protocol_bytes_count which is a measure of the raw bandwidth from the database (excluding encryption and SSL handshakes), and many others. Check out our list of all metrics for a more in-depth explanation and stay tuned for follow up blog posts where we'll dive into more complex examples of alerts and charts in Stackdriver.
network/https_requests_count
network/sent_payload_and_protocol_bytes_count
Welcome to part 2 of this blog series on using lifecycle-aware Android Architecture Components (LiveData and ViewModel) along with Firebase Realtime Database to implement more robust and testable apps! In the first part, we saw how you can use LiveData and ViewModel to simplify your Activity code, by refactoring away most of the implementation details of Realtime Database from an Activity. However, one detail remained: the Activity was still reaching into the DataSnapshot containing the stock price. I'd like to remove all traces of the Realtime Database SDK from my Activity so that it's easier to read and test. And, ultimately, if I change the app to use Firestore instead of Realtime Database, I won't even have to change the Activity code at all.
ViewModel
DataSnapshot
Here's a view of the data in the database:
and here's the code that reads it out of DataSnapshot and copies into a couple TextViews:
// update the UI with values from the snapshot String ticker = dataSnapshot.child("ticker").getValue(String.class); tvTicker.setText(ticker); Float price = dataSnapshot.child("price").getValue(Float.class); tvPrice.setText(String.format(Locale.getDefault(), "%.2f", price));
The Realtime Database SDK makes it really easy to convert a DataSnapshot into a JavaBean style object. The first thing to do is define a bean class whose getters and setters match the names of the fields in the snapshot:
public class HotStock { private String ticker; private float price; public String getTicker() { return ticker; } public void setTicker(String ticker) { this.ticker = ticker; } public float getPrice() { return price; } public void setPrice(float price) { this.price = price; } public String toString() { return "{HotStock ticker=" + ticker + " price=" + price + "}"; } }
Then I can tell the SDK to automatically perform the mapping like this:
HotStock stock = dataSnapshot.getValue(HotStock.class)
After that line executes, the new instance of HotStock will contain the values for ticker and price. Using this handy line of code, I can update my HotStockViewModel implementation to perform this conversion by using a transformation. This allows me to create a LiveData object that automatically converts the incoming DataSnapshot into a HotStock. The conversion happens in a Function object, and I can assemble it like this in my ViewModel:
HotStock
ticker
price
HotStockViewModel
Function
// This is a LiveData<DataSnapshot> from part 1 private final FirebaseQueryLiveData liveData = new FirebaseQueryLiveData(HOT_STOCK_REF); private final LiveData<HotStock> hotStockLiveData = Transformations.map(liveData, new Deserializer()); private class Deserializer implements Function<DataSnapshot, HotStock> { @Override public HotStock apply(DataSnapshot dataSnapshot) { return dataSnapshot.getValue(HotStock.class); } } @NonNull public LiveData<HotStock> getHotStockLiveData() { return hotStockLiveData; }
The utility class Transformations provides a static method map() that returns a new LiveData object given a source LiveData object and a Function implementation. This new LiveData applies the Function to every object emitted by the source, then turns around and emits the output of the Function. The Deserializer function here is parameterized by the input type DataSnapshot and the output type HotStock, and it has one simple job - deserialize a DataSnapshot into a HotStock. Lastly, we'll add a getter for this new LiveData that emits the transformed HotStock objects.
Transformations
map()
Deserializer
With these additions, the application code can now choose to receive updates to either DataSnapshot or HotStock objects. As a best practice, ViewModel objects should emit objects that are fully ready to be consumed by UI components, so that those components are only responsible for displaying data, not processing data. This means that HotStockViewModel should be doing all the preprocessing required by the UI layer. This is definitely the case here, as HotStock is fully ready to consume by the Activity that's populating the UI. Here's what the entire Activity looks like now:
public class MainActivity extends AppCompatActivity { private TextView tvTicker; private TextView tvPrice; @Override protected void onCreate(@Nullable Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); tvTicker = findViewById(R.id.ticker); tvPrice = findViewById(R.id.price); HotStockViewModel hotStockViewModel = ViewModelProviders.of(this).get(HotStockViewModel.class); LiveData<HotStock> hotStockLiveData = hotStockViewModel.getHotStockLiveData(); hotStockLiveData.observe(this, new Observer() { @Override public void onChanged(@Nullable HotStock hotStock) { if (hotStock != null) { // update the UI here with values in the snapshot tvTicker.setText(hotStock.getTicker()); tvPrice.setText(String.format(Locale.getDefault(), "%.2f", hotStock.getPrice())); } } }); } }
public class MainActivity extends AppCompatActivity { private TextView tvTicker; private TextView tvPrice; @Override protected void onCreate(@Nullable Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); tvTicker = findViewById(R.id.ticker); tvPrice = findViewById(R.id.price); HotStockViewModel hotStockViewModel = ViewModelProviders.of(this).get(HotStockViewModel.class);
hotStockLiveData.observe(this, new Observer() { @Override public void onChanged(@Nullable HotStock hotStock) { if (hotStock != null) { // update the UI here with values in the snapshot tvTicker.setText(hotStock.getTicker()); tvPrice.setText(String.format(Locale.getDefault(), "%.2f", hotStock.getPrice())); } } }); } }
All the references to Realtime Database objects are gone now, abstracted away behind HotStockViewModel and LiveData! But there's still one potential problem here.
All LiveData callbacks to onChanged() run on the main thread, as well as any transformations. The example I've given here is very small and straightforward, and I wouldn't expect there to be performance problems. But when the Realtime Database SDK deserializes a DataSnapshot to a JavaBean type object, it uses reflection to dynamically find and invoke the setter methods that populate the bean. This can become computationally taxing as the quantity and size of the objects increase. If the total time it takes to perform this conversion is over 16ms (your budget for a unit of work on the main thread), Android starts dropping frames. When frames are dropped, it no longer renders at a buttery-smooth 60fps, and the UI becomes choppy. That's called "jank", and jank makes your app look poor. Even worse, if your data transformation performs any kind of I/O, your app could lock up and cause an ANR.
onChanged()
If you have concerns that your transformation can be expensive, you should move its computation to another thread. That can't be done in a transformation (since they run synchronously), but we can use something called MediatorLiveData instead. MediatorLiveData is built on top of a map transform, and allows us to observe changes other LiveData sources, deciding what to do with each event. So I'll replace the existing transformation with one that gets initialized in the no-arg constructor for HotStockViewModel from part 1 of this series:
private final FirebaseQueryLiveData liveData = new FirebaseQueryLiveData(HOT_STOCK_REF); private final MediatorLiveData<HotStock> hotStockLiveData = new MediatorLiveData<>(); public HotStockViewModel() { // Set up the MediatorLiveData to convert DataSnapshot objects into HotStock objects hotStockLiveData.addSource(liveData, new Observer<DataSnapshot>() { @Override public void onChanged(@Nullable final DataSnapshot dataSnapshot) { if (dataSnapshot != null) { new Thread(new Runnable() { @Override public void run() { hotStockLiveData.postValue(dataSnapshot.getValue(HotStock.class)); } }).start(); } else { hotStockLiveData.setValue(null); } } }); }
Here, we see that addSource() is being called on the MediatorLiveData instance with a source LiveData object and an Observer that gets invoked whenever that source publishes a change. During onChanged(), it offloads the work of deserialization to a new thread. This threaded work is using postValue() to update the MediatorLiveData object, whereas the non-threaded work when (dataSnapshot is null) is using setValue(). This is an important distinction to make, because postValue() is the thread-safe way of performing the update, whereas setValue() may only be called on the main thread.
addSource()
Observer
postValue()
setValue()
NOTE: I don't recommend starting up a new thread like this in your production app. This is not an example of "best practice" threading behavior. Optimally, you might want to use an Executor with a pool of reusable threads (for example) for a job like this.
Executor
Now that we've removed Realtime Database objects from the Activity and accounted for the performance of the transformation from DataSnapshot to HotStock, there's still another performance improvement to make here. When the Activity goes through a configuration change (such as a device reorientation), the FirebaseQueryLiveData object will remove its database listener during onInactive(), then add it back during onActive(). While that doesn't seem like a problem, it's important to realize that this will cause another (unnecessary) round trip of all data under /hotstock. I'd rather leave the listener added and save the user's data plan in case of a reorientation. So, in the next part of this series, I'll look at a way to make that happen.
/hotstock
I hope to see you next time, and be sure to follow @Firebase on Twitter to get updates on this series! You can click through to part 3 right here.
MightySignal, a mobile intelligence startup based out of San Francisco, just published a new report examining the fastest growing Android SDKs of 2017. This fascinating report sheds light on which app development tools are taking off and what trends they signal for the year ahead. We're humbled and excited to see that 8 out of the top 20 fastest growing SDKs are part of Firebase!
This positive reception from the community validates and fuels our commitment to helping developers succeed. It also motivates us to continue making Firebase even better. Over the past year, we've made numerous improvements to our SDKs, including the ones highlighted in MightySignal's report.
Source: MightySignal's 2017 report on the fastest growing SDKs
For example, Firebase Realtime Database is number three on MightySignal's list, and it continues to be one of our most used and trusted products. We understand how important storing and syncing data is for your mobile business, and to further help you with this, we introduced another database product this year. If you're a Realtime Database customer, we think you'll love Cloud Firestore, our latest realtime, scalable NoSQL database that we built in collaboration with the Google Cloud Platform team. It allows you to sync and store data like Realtime Database, while also addressing its key limitations like data structuring, querying, and scaling. Cloud Firestore is available in beta today!
Another notable mention is Firebase Remote Config. Remote Config gives you the power to customize your app's interface and behavior for different audiences so you can deliver personalized app experiences without requiring users to update their apps. Now, Remote Config can be used with Firebase Predictions' dynamic user groups. This means you can change the look and feel of your app for users based on their predicted behavior (such as churn or in-app purchase). Wondering how this works? Learn how Halfbrick Studios grew their 7-day retention rate from 25% to 30% by combining Predictions with Remote Config.
And that's not all that's new with Remote Config! In the past, Remote Config allowed you to perform simple A/B testing, but now, we've gone ahead and added an entirely new experiment layer in Firebase that works wonderfully with Remote Config so you can set up, run, and measure sophisticated A/B tests.
We were also delighted to see that Firebase Auth and Firebase Crash Reporting are experiencing high growth as well, according to MightySignal's findings. After welcoming the Fabric team to Firebase, we worked together to add new features to Auth (such as phone number authentication), which we unveiled in June. More recently, we launched a beta version of Firebase Crashlytics, a powerful realtime crash reporting tool that will help you track, prioritize, and fix issues that erode app stability. Firebase Crashlytics is now our primary crash reporter. If you want to learn more about how app stability can lead to growth in user engagement and retention, check out how Doodle used Crashlytics to grow user engagement by 42%.
MightySignal's data on the fastest growing SDKs is available here. We're very thankful to be part of the developer community and committed to helping you build better apps and grow your business. Stay tuned for more product updates next year and, in the meantime, happy building!
