The Firebase Blog: Mart 2017

Using Firebase Cloud Messaging with Android O

29 Mart 2017

Diego Giorgini
Software Engineer

Firebase Cloud Messaging (FCM) is a cross-platform messaging solution that lets you reliably deliver messages to your apps and sites. It provides two types of messages:

Notification Messages display a simple notification popup, with optional data payload.

Data Messages deliver a JSON payload to your application and let your code handle it.

Data Messages are a great way to build custom notifications, when the layout provided by notification messages is not enough, or to trigger background operations like a database sync or the download of additional content (image attachments, emails, etc.) How should Data Messages trigger background operations? The best way to trigger a background operation from a data message is by using the Work Manager to schedule your operation at a point when it's best for the user (like avoiding extra work when the battery is very low, or when the CPU is already heavily used by other foreground applications). Background Process Optimizations in Android O To get started, check out the Android O Developer Preview site where you will find instructions on downloading and installing the required SDKs. For Firebase Development, you'll also need to install the Firebase SDKs for Android. Be sure to use version 10.2.1 or later for Android O development. Android O introduces new background processes optimizations, which make the use of JobScheduler (or wrapper libraries like the Work Manager) a requirement for long-running background operations. Due to these optimizations, the FCM (hence GCM as well) callbacksonMessageReceived()and onTokenRefresh() have a guaranteed life cycle limited to 10 seconds (same as a Broadcast Receiver). After the guaranteed period of 10 seconds, Android considers your process eligible for termination, even if your code is still executing inside the callback. To avoid your process being terminated before your callback is completed, be sure to perform only quick operations (like updating a local database, or displaying a custom notification) inside the callback, and use JobScheduler to schedule longer background processes (like downloading additional images or syncing the database with a remote source). @Override public void onMessageReceived(RemoteMessage remoteMessage) { if (/* Check if data needs to be processed by long running job */ true) { // For long-running tasks (10 seconds or more) use the Work Manager scheduleJob(); } else { // Handle message within 10 seconds handleNow(); } } /** * Schedule a job using the Work Manager. */ private void scheduleJob() { WorkManager.getInstance().enqueue( new OneTimeWorkRequest.Builder(MyWorker.class).build()); } /** * Perform and immediate, but quick, processing of the message. */ private void handleNow() { Log.d(TAG, "Short lived task is done."); } We hope this helps you understand how to use FCM to schedule long-running background operations. This solution greatly helps Android to preserve battery life and ensures that your application works fine on Android O. In case you have any questions don't hesitate to ask us on our support channels.

How to Schedule (Cron) Jobs with Cloud Functions for Firebase

24 Mart 2017

Abe Haskins
Developer Programs Engineer

Cloud Pub/Subreact clean up extra data How to Schedule Functions

reliable task scheduling on Google Compute Engineonly functions-samples Deploying the App Engine Appfirebase/functions-croncron.yamlcron.yaml Reference 1. Prerequisites

git

Python 2.7

Python pip

Google Cloud SDK

2. Clone this repositorygit clone https://github.com/firebase/functions-croncd functions-cron 3. Deploy to App Enginegcloud config set project appengine/cd appengine/

$ pip install -t lib -r requirements.txt

Create an App Engine App

gcloud app create

Deploy the application to App Engine.

gcloud app deploy app.yaml \ cron.yamlGoogle Cloud Logging/_ah/start 4. Deploy to Google Cloud Functions for Firebasecd .. Deploy the sample hourly_job function to Google Cloud Functions

firebase deploy --only functions --project

Warning: This will remove any existing functions you have deployed. If you have existing functions, copy the example from functions/index.js into your project's index.js

5. Verify your Cron JobsTask Queuehourly_job/publish/hourly-tickfirebase functions:log --project console.loghourly_job You're Done!hourly-tickdaily-tickweekly-tickcron.yaml

BigQuery Tip: The UNNEST Function

22 Mart 2017

Todd Kerpelman
Developer Advocate

Firebase Analytics data to BigQuery
At first, the data set in BigQuery might seem confusing to work with. If you've worked with any of our public BigQuery data sets in the past (like the Hacker News post data, or the recent San Francisco public data that our Developer Advocate Reto Meier had fun with), it probably looked a lot like a big ol' SQL table. Something like this:

The truth of the matter is that BigQuery can get much more sophisticated than that. The rows of a BigQuery table don't just have to be straightforward key-value pairs. They can look more like rows of JSON objects, containing some simple data (like strings, integers, and floats), but also more complex data like arrays, structs, or even arrays of structs. Something a little more like this:

Firebase Analytics takes advantage of this format to bundle all of your users' user properties together in the same row. Rather than have you perform some kind of join against a separate user_properties table, all of your user properties are included in the same BigQuery row as an array of structs.

A slightly simplified version of the user_properties struct in your BigQuery data

The same thing holds true for your events. Your event parameters are included inside your events as an array of structs. And it turns out these events themselves are stored inside of an array. One single row of data in BigQuery will often contain 2 or 3 Firebase Analytics events all bundled together.

This means a single row in your BigQuery table can contain an array of user properties, as well as an array of events, which in turn also have their own arrays of event parameters. I know combining all of that information into a data structure like this seems confusing at first, but in the long run, it actually makes your life easier because there aren't any JOINs with other tables for you to worry about.

Important note: For all of these examples, I'm going to be using standard SQL, which is what all the cool kids are doing this days¹. If you want to follow along, turn off Legacy SQL in your BigQuery options. Also, you'll need to follow this link to access the sample Firebase Analytics data we'll be using.

For example, I can see all of my event data at once just by calling

#standardSQL
SELECT event_dim 
FROM `firebase-analytics-sample-data.android_dataset.app_events_20160607` 
LIMIT 50

and I'll get back all of my event data, along with all of the event parameters, in one nice little table

And then if we want to get a list of all of my "Round completed" events, I can just write some SQL like this…

#standardSQL
SELECT event_dim 
FROM `firebase-analytics-sample-data.android_dataset.app_events_20160607` 
WHERE event_dim.name = "round_completed"

...which gives me a nice result of...

Error: Cannot access field name on a value with type ARRAY<STRUCT<date STRING, name STRING, params ARRAY<STRUCT<key STRING, value STRUCT<string_value STRING, int_value INT64, float_value FLOAT64, ...>>>, ...>> at [2:17]

Oh. Oh dear.

Okay, so this won't win any awards for "Best Error Message of 2017"² , but if you think about it, the reason it's barfing makes sense. You're trying to compare a string value to "an element of a struct that's buried inside of an array". Sure, that element ends up being a string, but they're fairly different objects.

So to fix this, you can use the UNNEST function. The UNNEST function will take an array and break it out into each of its individual elements. Let's start with a simple example.

Calling:

#standardSQL
WITH data AS (
  SELECT "primes under 15" AS description,
  [1,2,3,5,7,11,13] AS primes_array)
SELECT * 
FROM data

will give you back a single row consisting of a string, and that array of data.

Instead, try something like this:

#standardSQL
WITH data AS (
  SELECT "primes under 15" AS description,
  [1,2,3,5,7,11,13] AS primes_array)
SELECT description, prime 
FROM data CROSS JOIN UNNEST (primes_array) as prime

What you're basically saying is, "Hey, BigQuery, please break up that primes_array into its individual members. Then join each of these members with a clone of the original row." So you end up with a data structure that looks more like this:

The results are similar as before, but now each prime is in its own row:

You'll notice that the original primes_array is still included in the data structure. In some cases (as you'll see below), this can be useful. In this particular case, I found it was a little confusing, which is why I only asked for the individual fields of description and prime instead of SELECT *.³

It's also common convention to replace that CROSS JOIN syntax with a comma, so you get a query that looks like this.

#standardSQL
WITH data AS (
  SELECT "primes under 15" AS description,
  [1,2,3,5,7,11,13] AS primes_array)
SELECT description, prime 
FROM data, UNNEST (primes_array) as prime

It's the exact same query as the previous one; it's just a little more readable. Plus, I can now stand by my original statement that this data format means you don't have perform any JOINs. :)

And the nice thing here is that I now have one piece of "prime" data per column that I can interact with. So I can start to do comparisons like this:

#standardSQL
WITH data AS (
  SELECT "primes under 15" AS description,
  [1,2,3,5,7,11,13] AS primes_array)
SELECT description, prime 
FROM data, UNNEST (primes_array) as prime
WHERE prime > 8

To get just that list of prime numbers between 8 and 15.

So going back to our Firebase Analytics data, I can now use the UNNEST function to look for events that have a specific name.

#standardSQL
SELECT event.name, event.timestamp_micros
FROM `firebase-analytics-sample-data.android_dataset.app_events_20160607`, 
  UNNEST(event_dim) as event
WHERE event.name = "round_completed"

As you'll recall, events have their own params array, which contains all of the event parameters. If I were to UNNEST those as well, I'd be able to query for specific events that contain specific event parameter values:

#standardSQL
SELECT event, event.name, event.timestamp_micros
FROM `firebase-analytics-sample-data.android_dataset.app_events_20160607`, 
  UNNEST(event_dim) as event,
  UNNEST(event.params) as event_param
WHERE event.name = "round_completed"
AND event_param.key = "score"
AND event_param.value.int_value > 10000

Note that in this case, I am selecting "event" as one of the fields in my query, which gives me the original array of all my event parameters nicely grouped together in my table results.
Querying against user properties works in a similar manner. Let's say I'm curious as to what language my users prefer using for my app, something our app is tracking in a "language" user property. First, I'll use the UNNEST query to get just a list of each user and their preferred language.

#standardSQL
SELECT
 user_dim.app_info.app_instance_id as unique_id,
  MAX(user_prop.key) as keyname,
  MAX(user_prop.value.value.string_value) as keyvalue
FROM `firebase-analytics-sample-data.android_dataset.app_events_20160607`,
  UNNEST(user_dim.user_properties) AS user_prop
WHERE user_prop.key = "language"
GROUP BY unique_id

And then I can use that as my inner selection to grab the total number of users⁴ that fits into that group.

#standardSQL
SELECT keyvalue, count(*) as count
FROM (
  SELECT
   user_dim.app_info.app_instance_id as unique_id,
    MAX(user_prop.key) as keyname,
    MAX(user_prop.value.value.string_value) as keyvalue
  FROM `firebase-analytics-sample-data.android_dataset.app_events_20160607`,
    UNNEST(user_dim.user_properties) AS user_prop
  WHERE user_prop.key = "language"
  GROUP BY unique_id
) 
GROUP BY keyvalue
ORDER BY count DESC

I can also UNNEST both my event parameters and my user properties if I want to create one great big query (no pun intended) where I want to look at events of a specific name where an event parameter matches a particular criteria, while also filtering by users who meet a certain criteria:

#standardSQL
SELECT user_dim, event, event.name, event.timestamp_micros
FROM `firebase-analytics-sample-data.android_dataset.app_events_20160607`,
  UNNEST(event_dim) as event,
  UNNEST(event.params) as event_param,
  UNNEST(user_dim.user_properties) as user_prop
WHERE event.name = "round_completed"
  AND event_param.key = "squares_daubed"
  AND event_param.value.int_value > 20
  AND user_prop.key = "elite_powers"
  AND (CAST(user_prop.value.value.string_value as int64)) > 1

Once you start playing around with the UNNEST function, you'll find that it's really powerful and it can make working with Firebase Analytics data a lot more fun. If you want to find out more, you can check out the Working with Arrays section of BigQuery's standard SQL documentation.

And don't forget, you get 1 terabyte of usage data for free every month with BigQuery, so don't be afraid to play around with it. Go crazy, you array expander, you!

1 ^↩ The BigQuery team has asked me to inform you that this is really because standard SQL is the preferred SQL dialect for querying data stored in BigQuery. But I'm pretty sure they're just saying that so they get invited to all the good parties.

2 ^↩ Yet another year the Messies have slipped from our grasp!

3 ^↩ I could have also done this by saying "SELECT * EXCEPT (primes_array)", which can be pretty convenient sometimes.

4 ^↩ Okay, technically, each "App Instance" -- a user interacting with my app from multiple devices would get counted multiple times here.

Take Control of Your Firebase Init on Android

18 Mart 2017

Doug Stevenson
Developer Advocate

how Firebase initializes on Android What if you have a customized build system?Google Services Gradle PluginBazeldocumentation for the plugin What if you need to select your app's Firebase project at runtime?select its Firebase project at runtime 1. Disabling FirebaseInitProvidernode markertools:node"remove"<provider android:name="com.google.firebase.provider.FirebaseInitProvider" android:authorities="${applicationId}.firebaseinitprovider" tools:node="remove" /> <manifest xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" package="your.package" > 2. Calling FirebaseApp.initializeApp() to initializeFirebaseApp.initializeApp(Context, FirebaseOptions)FirebaseOptionsBuilderFirebaseOptions.Builder builder = new FirebaseOptions.Builder() .setApplicationId("1:0123456789012:android:0123456789abcdef") .setApiKey("your_api_key") .setDatabaseUrl("https://your-app.firebaseio.com") .setStorageBucket("your-app.appspot.com"); FirebaseApp.initializeApp(this, builder.build()); documentation for the plugin

Profiling your Realtime Database Performance

14 Mart 2017

Tyler Rockwood
Software Engineer

profiler built into the Firebase CLIFirebase CLIfirebase database:profile

Speed*

This table displays 4 items: the path, the number of times that location has been hit, the number of milliseconds it took for the server to process the request, and the number of times that the path has been denied by rules.

Bandwidth**

The table displays 3 items: the path, the total amount of bandwidth for the path, and the average bandwidth per operation.

Unindexed Queries

What if this isn't enough?

profiler reference page

Multiplying the Power of Firebase Storage

10 Mart 2017

Mike McDonald
Product Manager

Use multiple buckets in your projectsBlaze payment plan

Logically separate different types of data (e.g. user data from developer data).

Store data in a location closer to users, either to optimize performance or support regulatory compliance.

Reduce cost by storing infrequently accessed data (e.g. backups) in a different storage class.

Link existing buckets to your projectsconfigure your security rules to allow access

Integrate with Google Cloud Functions

Cloud Functions for Firebaseconverting imagesgenerating thumbnailsmoderating imagesextracting metadata Same feature, new nameCloud Storage for FirebaseFirebase SDKs for Cloud StorageCloud Storage server SDKsTwitterFacebookSlackGoogle Group

Introducing Cloud Functions for Firebase

9 Mart 2017

Brendan Lim
Product Manager

Cloud Functions for Firebase
Cloud Functions is a versatile tool for building your mobile app. Here are a just a few of the many tasks you can perform with the integrations available at launch:
Firebase Analytics integration lets you trigger a function when a specific conversion event is fired. You can create functions to automate growth and retention workflows for your mobile apps, all without ever needing to update your client code.

Send a notification with a coupon code to users who have just completed a purchase

Send a survey request to users after they upgrade their app

Firebase Authentication integration lets you trigger a function when a new user is created or deleted.

Send an email to welcome new users immediately after they sign up

Clean up user associated data from your Realtime Database when a user account is deleted

Firebase Realtime Database integration lets you trigger a function when data is created, updated, or deleted at a specific path in the database.

Send a notification using FCM when data is written to the database

Moderate and remove abusive language from data written to the database

Cloud Storage integration lets you trigger a function when an object is written, updated, or deleted within a particular storage bucket.

Resize and convert images when a new image has been uploaded to your Cloud Storage bucket

Moderate abusive images uploaded to Cloud Storage using the Cloud Vision API

HTTP endpoint integration gives your Cloud Function a URL that can be used as a webhook. These functions are triggered when a request is made to their own unique, secure URLs.

Post new GitHub commits to a Slack channel

Process in-app payments using Stripe

We'll continue to add more integrations in the future. "We were early testers of Cloud Functions for Firebase and were excited to see how easy it was to extend the Realtime Database to export data and integrate with other services."
- Erling Mårtensson, Master Architect, Sony

Firebase SDK and tooling

Cloud Functions for Firebase provides a first-class experience for Firebase developers, built on top of Google Cloud Functions. Cloud Functions are single-purpose JavaScript functions that are executed in a secure, managed Node.js environment. The Firebase SDK for Cloud Functions gives you an API that allows you to choose an event source (such as writes to Firebase Realtime Database at a specific data location) and implement a function that triggers on every matching event. Our SDK also works with TypeScript to support code completion and help you catch syntax errors early.
The SDK works in tandem with the Firebase CLI to provide a seamless experience when deploying your functions. This tight integration allows you to deploy all of your functions using only a single command.

Pricing

Cloud Functions is available on all Firebase pricing plans, including our free tier. The free tier allows you to quickly experiment and try out integration with other Firebase products. For our Blaze plan, you only pay for what you use. Blaze customers also receive a monthly allotment of free usage for Cloud Functions.

"Thanks to Cloud Functions for Firebase, I built a company with no permanent employees but myself (so far), with no serious scaling concerns, and no major costs maintaining or upgrading the backend as the app grows. It's something of a miracle."

- Paul Budnitz, Founder/CEO, Wuu

Create and deploy your first Cloud Function today

Getting started is easy! Walk through our step-by-step codelab, which takes you through setting up your first Cloud Function. You can refer to our full documentation for all the details.

We can't wait to see what you build!

Working Closer with Google Cloud Platform

9 Mart 2017

James Tamplin
Product Manager

Since expanding Firebase to become Google's mobile application development platform at Google I/O last year, our amazing community of developers has created over 1 million Firebase projects. We're thrilled so many of you use and trust us. While Firebase is a full suite of products for building and growing apps, we know that some apps need more than we offer out-of-the-box. That's why we're bringing Firebase much closer to Google Cloud Platform (GCP) to serve even the most demanding applications, whether you're a new startup or a large enterprise. Product Integrations Firebase already shares the same account and billing system as GCP, so you can attach Firebase services to your GCP project and vice-versa. This makes for powerful combinations, such as exporting raw event data from Firebase Analytics into BigQuery for ad-hoc analysis. Starting today, we're beginning to share products too. First, Firebase developers have been asking for ways to extend their app's functionality without spinning up a server, and Cloud Functions for Firebase lets you do just that. Cloud Functions is our new event-driven serverless compute offering that enters public beta today. The infrastructure is shared between Cloud and Firebase, allowing you to invoke a function or access resources from throughout the Cloud/Firebase ecosystem. For more information, read the announcement on the Firebase blog and Cloud blog. Next, we're bringing Firebase Storage closer to Cloud Storage. Firebase Storage launched 10 months ago, and lets you easily upload and download files from your device directly to Cloud Storage. Previously we gave you a single bucket for your files. Now we're fully aligning the two products and letting you use any Cloud Storage bucket, from any global region and from any storage class, all straight from the Firebase SDK. To reflect this alignment we're renaming the product Cloud Storage for Firebase. Read more in our blog post. Stay tuned for more product integrations in the future as Firebase continues to provide direct client-side access to GCP infrastructure through our iOS, Web, and Android SDKs. Streamlined Terms of Service We love lawyers almost as much as developers, so we're extending GCP's Terms of Service to cover several Firebase products. This makes Firebase and Cloud simpler to evaluate and use together. Products to be covered include: Authentication, Hosting, Storage, Functions, and Test Lab. Our streamlined Terms of Service will take effect soon. The Big Picture Firebase brings together the best of Google on mobile -- whether that's Google's flagship advertising solutions like AdMob and AdWords, or Google's analytics expertise in the form of Firebase Analytics. Google Cloud Platform lets you to benefit from the institutional knowledge Google has developed from almost two decades of running global-scale computing infrastructure. By bringing together the ease-of-use of Firebase with the full-range of GCP infrastructure offerings, we're better able to serve you up and down the stack. If you're a startup using Firebase to quickly get to market, you can now easily scale into a full public cloud. If you're an existing business running on GCP who wants to ship a mobile app, we've got you covered too. We can't wait to see what you build with Firebase and Google Cloud Platform!

Updates to Firebase Notifications: Reach your users with greater focus

1 Mart 2017

Laurence Moroney
Developer Advocate

This allows you to target or exclude users who are members of a particular set of audiences. . Using a combination of several targeting criteria, as well as user-defined audiences, you have great control over which users will receive your notification.

Also new is the facility to filter your desired target segment based on user property. Firebase User Properties are attributes that allow you to define segments of your user base, beyond the typical app level properties. So, for example, if you’re building a fitness app, and you want to track how many people have met their fitness goals, you could define a user property for that, and then send notifications to them accordingly.

Now, you can target a notification by selecting a user property to compare against a desired value. So, if the user property contains a string, you can check if it contains, does not contain, is in, or is not in a range of values, or you can target users with user properties that match a Regex using RE2 syntax.

And of course if the user property contains a numeric value, there are several operators that you can use to compare it to a value -- such as equals, greater than, less than and more.

With these tools you’re equipped to more accurately target your notifications, to improve response rates while avoiding notification spam. We’re continuing to improve all of Firebase, including Firebase Notifications, and would love to hear your feedback on this or any other features.

To learn more, check out the Firebase Notifications documentation site.

Realtime Analytics For Everyone

1 Mart 2017

Steve Ganem

Product Manager

Back in November we launched real-time conversions. This meant that your most important events were delivered immediately from end-user devices. Around the same time, we also launched real-time export to BigQuery, so that your client events could be analyzed within BigQuery as soon as they were sent to our servers.
But for those of you not using BigQuery, you'd often have to wait several hours before you could start to see any meaningful results from Firebase Analytics. And when it comes to tasks like debugging analytics within your app, or just getting a quick snapshot of how your community is reacting to your latest changes, those hours can feel realllly long.
To address those needs, we are pleased to announce that StreamView and DebugView from Firebase Analytics are being rolled out to the general public.
StreamView visualizes events as they flow into Firebase Analytics, and gives you a general sense of how your users are interacting with your app. With StreamView you can find out if a brand new feature is as well-received as you were hoping, watch the roll-out of the latest version of your app, or gauge the response to your latest re-engagement efforts - all in real-time as events are received by Firebase Analytics.
StreamView can give you a sense of where in the world people are using your app, right down to the city level…

...or what user properties are most common among your users….

You can also get an overview of what kind of events are being triggered in your app, and for any of those events, you can see a breakdown of all the custom event parameters you've been recording...

...and you can use all of these values as report filters to see where in the world certain events are occurring or certain products are popular, and you can see what user properties or events are common among users in certain countries or cities.
One unique feature available in StreamView is the User Snapshot. This will give you a live stream of the events from a random user. You can use the view to gain insight into the individual user journeys within your app.

Within the User Snapshot view you can see the set user properties, the user's device and location, and drill into the parameters of the events they're sending. You can filter the users included by their location and app version.

While you may still need to use BigQuery to get full and complete reports, or cross-reference certain sets data against others, there's a surprising amount you can learn about your app just by following the events as they happen in StreamView. The Santa Tracker team was an early customer of StreamView and they've assured me that it's (and this is a direct quote), "Pretty bad-ass."
You can can check out StreamView right away in our Demo Project!
While StreamView is a great real-time picture of your app once it's gone out to the general public, event delivery from device is still batched. This is great for users and battery optimization, but difficult when you're developing your app and need to check right away if your analytics setup is correct!
With DebugView, you can immediately see which events are being reported to Firebase Analytics. This will be activated for any device for which you've turned on Analytics Debugging, and is great for making sure you're logging the right events with the right parameters while you're building your app.

DebugView shows you events, parameters and user properties for any individual development device. It can also highlight any events that contain invalid parameters, so that you can get those fixed before you publish your app.

To get started yourself, take a look at our documentation on DebugView, or just look for these features in your own project console in the coming days and try them out!