When we think of scaling we usually imagine spiky charts of users hitting a database or processing computationally expensive queries. What we don't always think about it is deleting data. Handling large amounts of deletes is an important part of scaling a database. Imagine a system that's required to delete historical records at a specific deadline. If these records are hundreds of gigabytes in size, it will likely be difficult to delete them all without bogging the database down for the rest of its users. This exact scenario hasn't always been easy with the Firebase Realtime Database, but we're excited to say that it just got a lot easier.
Today, we're introducing a new way to efficiently perform large deletes!
If you want to delete a large node, the new recommended approach is to use the Firebase CLI (> v6.4.0). The CLI automatically detects a large node and performs a chunked delete efficiently.
$ firebase database:remove /path/to/delete
Keep in mind that in order to delete a large node, the Firebase CLI has to break it down into chunks. This means that clients can see a partial state where part of the data is missing. Writes in the path that is being deleted will still succeed, but the CLI tool will eventually delete all data at this path. This behavior is acceptable if no app depends on this node. However, if there are active listeners within the delete path, please make sure the listener can gracefully handle partial documents.
If you want consistency and fast deletion, consider using a special field, a.k.a a tombstone to mark this document as hidden, and then run a cloud function cron job to asynchronously purge the data. You can use Firebase Rules to disallow access to hidden documents.
We've also added a configuration option (defaultWriteSizeLimit) to the Realtime Database that allows you to specify a write size limit. This limit allows you to prevent operations (large deletes and writes) from being executed on your database if they exceed this limit.
defaultWriteSizeLimit
You can use this option to prevent app code from accidentally triggering a large operation, which would make your app unresponsive for a time. For more detail, please see our documentation about this option.
You can check and update the configuration via the CLI tool (version 6.4.0 and newer). There are four available thresholds. You can pick appropriate threshold based on your application requirement
small
write
medium
large
unlimited
Note: The target time is not a guaranteed cutoff off. The estimated time may be off from the actual write time.
$ firebase database:settings:set defaultWriteSizeLimit unlimited --instance <database-name> $ firebase database:settings:get defaultWriteSizeLimit --instance <database-name>
For REST requests, you can override defaultWriteSizeLimit with the writeSizeLimit query parameter. In addition, REST queries support a special writeSizeLimit value:
writeSizeLimit
tiny
firebase database:remove
For example:
$ curl -X PUT \ "https://<database-name>.firebaseio.com/path.json?writeSizeLimit=medium"
The default defaultWriteSizeLimit for new databases is large. In order to avoid affecting existing apps, the setting will remain at unlimited for existing projects for now.
We do want to extend this protection to everyone. So this summer (June~August, 2019), will set defaultWriteSizeLimit to large for existing databases that have not configured defaultWriteSizeLimit. To avoid disruption, we will exclude any databases that have triggered at least one large delete in the past three months.
defaultWriteSizeLimit.
These controls can help you keep your apps responsive and your users happy. We suggest setting defaultWriteSizeLimit for your existing apps today.
Let us know what you think of this new feature! Leave a message in our Google group.
In Firebase Crashlytics, you can view crashes and non-fatals by versions. Several of our customers take advantage of this filtering, especially to focus on their latest releases.
But sometimes too many versions can be a bad thing. Firebase Crashlytics - by default - shows you the last 100 versions we have seen. If you have a lot of debug versions created by developers, and by continuous Integration and deployment pipeline, you might soon start to not see the versions that really matter e.g., production builds.
So how do you make sure that this does not happen? Disabling Crash reporting initialization for debug builds is the simplest way of achieving this. Let's explore how to do this on iOS and Android.
For iOS apps, first check if you are manually initializing Crashlytics (this happens for Fabric apps that were linked to a Firebase app).
If you use Swift, search for the line Fabric.with([Crashlytics.self]) in AppDelegate.swift. If this line is present, then you are manually initializing Crashlytics, otherwise you are using automatic initialization.
Fabric.with([Crashlytics.self])
If you use ObjectiveC, search for the line [Fabric with:@[[Crashlytics class]]]; in AppDelegate.m. If this line is present, then you are manually initializing Crashlytics, otherwise you are using automatic initialization.
[Fabric with:@[[Crashlytics class]]];
For apps that are using manual initialization, you can just not initialize Crashlytics for DEBUG versions.
For Swift
#if !DEBUG Fabric.with([Crashlytics.self]) #endif
For ObjectiveC
#if !DEBUG [Fabric with:@[[Crashlytics class]]]; #endif
Firebase Crashlytics apps are automatically initialized by Firebase. You can turn off automatic collection with a new key to your Info.plist file:
Info.plist
firebase_crashlytics_collection_enabled
no
Then you can initialize it as shown in the examples above for Swift and ObjectiveC
For Android apps, first check if you are manually initializing Crashlytics (this happens for Fabric apps that were linked to a Firebase app). Search for Fabric.with in your project. If this line is present, then you are manually initializing Crashlytics. Otherwise, Crashlytics is being automatically initialized through Firebase.
Fabric.with
To disable Crashlytics in debug builds, you can make use of the BuildConfig.DEBUG flag. Edit the Fabric.with statement you found previously, adding a check for the DEBUG flag:.
BuildConfig.DEBUG
if (!BuildConfig.DEBUG) { Fabric.with(this, new Crashlytics()); }
Turn off Crashlytics initialization in your debug builds by creating a debug folder in your src directory and creating an AndroidManifest.xml with this snippet:
debug
src
<manifest xmlns:android="http://schemas.android.com/apk/res/android"> <application> <meta-data android:name="firebase_crashlytics_collection_enabled" android:value="no" /> </application> </manifest>
This snippet will be merged into the manifest of all of your debug variants, and will disable Crashlytics in those builds. If you'd prefer finer-grained control, you can use this approach for any variants you'd like to exclude.
Firebase Auth client SDKs make it possible to sign in as a Firebase user from federated identity providers, including Google, Facebook, and Twitter. The Firebase Auth team is always looking for opportunities to improve the auth experience for developers and users. We know that more sign-in options mean more opportunities to create the best app experience. That's why we are pleased to announce that you can now sign in to Firebase Auth using Microsoft and Yahoo!
Firebase Auth has added two new federated identity providers! Today, Microsoft and Yahoo join our growing list of providers. Applications that rely on these providers no longer have to handle custom credentials on the backend. With this simplified auth flow, developers can spend less time on implementing authentication and more time to spend on the core features of your application. Developers who implement Microsoft or Yahoo sign in for Firebase Auth will also get the benefit of a new, simpler way to get the provider credentials.
Signing in to Firebase with a given identity provider requires garnering a credential from that provider. This often involves including the provider's SDK and implementing the provider's sign-in methods before passing the credentials to Firebase Auth. For some providers, this can be particularly difficult on a native client, especially ones which do not support their own native SDKs. In order to remove the headache of implementing sign-in flows for these identity providers, we now offer generic OAuth2 provider sign-in.
Generic identity provider sign-in provides an easy means for developers to get that credential and use it to sign in by popping up a ChromeCustomTab on Android or a SafariViewController on iOS which will use the Web flow to generate the Identity Provider Credential, and then continue the sign in. Generic sign-in gives you many benefits:
Generic sign-in is available for Microsoft and Yahoo, and will be a feature of other identity providers in the future.
Ready to get started with Microsoft and Yahoo sign-in for Firebase Auth? Check out the guides linked below:
Microsoft Authentication for iOS
Microsoft Authentication for Android
Microsoft Authentication for Web
Yahoo Authentication for iOS
Yahoo Authentication for Android
Yahoo Authentication for Web
Quickstart for iOS
Quickstart for Android
Quickstart for Web
Hey, there Firebase Developers! We wanted to let you know of a new feature we added to Cloud Firestore, and that's the ability to increment (or decrement) numeric values directly in the database! Why is this a big deal? Because previously, if you wanted to increment a value in the database, you would need to use a transaction to keep it safe from concurrent writes.
For instance, imagine that your fitness app has a feature that allows groups of friends to pool their steps together for a "team step counter". Then, imagine Bob and Alice both have their current team's total steps recorded locally and try to add their values to it…
If both clients send down these new values at around the same time, it's possible for Alice's changes to be written first...
...and then for Bob's changes to come in and overwrite Alice's changes.
In the past, if you wanted to prevent this from happening, you would have to use a transaction. And while transactions are still a fine solution, they are a little more difficult to write, they don't support cases where the user is offline, and frankly, they seem a bit heavy-handed for something as simple as adding two numbers together.
So now, if you want Bob to record his 500 steps in the app, you can simply do that by asking the server to increment the step_counter value. In an iOS app, for instance, you would write code that looks a little something like this.
document("fitness_teams/Team_1"). updateData(["step_counter" : FieldValue.increment(500)])
With this call, the database would instantly make the change based on whatever value it has. So even if Alice sneaks in a change before Bob's request reaches the server, everything still works.
Now, there are two things to keep in mind when it comes to performing these operations:
First of all, if you do want to add some logic to your operation (like, for instance, making sure this new value doesn't go over or under a certain limit), you'll still need to use a transaction. Luckily, we will soon release a fantastic video all about transactions that I can heartily recommend as a completely unbiased observer.
Second, don't forget that documents are still limited to a sustained write limit of 1 QPS per document. I know it's tempting to look at these numeric functions and start making counters left and right in your database, but if you think these are values you're going to be updating more than once per second, you're still going to want to rely on something like our distributed counter solution.
As always, if you have questions, feel free to contact us on any of our support channels, or post questions on Stack Overflow with the google-cloud-firestore and firebase tags. Good luck, and have fun!
google-cloud-firestore and firebase tags
Firebase is returning to the Game Developers Conference this year in San Francisco to share how we're making Firebase even more powerful for game developers!
At the Google Mobile Developer Day on Monday, we announced:
In case you missed these announcements on stage, read on below for a quick recap. We'll also be at booth #P1501 starting tomorrow until the end of the week, and have a special session on Firebase for Games in the Cloud Developer Day track, so if you're at the GDC this year, please come say hello! We're excited to get your feedback, and learn more about how we can make these features and Firebase for Games even better in the future.
Here's a recap of the new Firebase updates for games developers:
Firebase Crashlytics is now available for Unity! Firebase Crashlytics helps you track, prioritize, and fix stability issues in real time. It synthesizes an avalanche of crashes into a manageable list of issues, provides contextual information, and highlights the severity and prevalence of crashes so you can pinpoint the root cause faster. You can also easily reproduce issues by reviewing breadcrumbs, a list of actions that the user performed before the error occurred.
Crashlytics was previously only available for Android and iOS, but you can now use that same powerful crash-reporting functionality in your Unity games!
With the same simplicity of all our Firebase for Unity SDKs, Crashlytics can be integrated directly via your Unity IDE with no extra steps required. Your team can also be alerted of issues using our Email, Slack, Jira and PagerDuty Integrations, or write your own triggers using Cloud Functions with Firebase Crashlytics triggers.
And last, but not least, you can export Crashlytics data into BigQuery for deeper-dives. For example, let's say you log a custom key when reporting a Crashlytics event like so:
Crashlytics.SetCustomKey("current_level", "3");
After you export your Crashlytics data to BigQuery, you will be able to see the distribution of crashes by level 3, or any other level or custom key you've used for your events! And you can use our Data Studio template to quickly analyze this data. This will make tracking and fixing issues in your Unity game development a whole lot easier.
Refer to our documentation to get started with Firebase Crashlytics for Unity.
For C++ developers out there, you can now use an open source SDK for your Firebase for Games integration! This means more control than ever when integrating with the Firebase toolset. You can now easily dig into the code to see or change what's happening within the SDK to better suit your game, or maintain your own branch of the client SDK and cherry pick specific changes you care about. And if there are other platforms you want to use that we haven't yet built support for, you have the option to extend the SDK.
Firebase Test Lab is a cloud-based app-testing infrastructure. With one operation, you can test your Android or iOS app across a wide variety of devices and device configurations, and see the results in the Firebase console. For native Android apps, what are called Robo tests can intelligently analyze the UI elements on-screen and crawl through them. For games, however, this isn't possible because all of your elements are typically in a GL/Vulkan/Metal View. Instead Robo will resort to Monkey Actions, where it will tap randomly.
It could take quite some time for a smoke test like this to complete though, which is why we're happy to announce new AI-assisted Monkey Actions to help test your games with Firebase Test Lab. This will significantly reduce the time it takes for a robot to make progress through your game's user interface and provide useful test results. Check out the before / after videos for Monkey Actions with and without AI-assistance to see the difference. Note the before video showing Monkey Actions without AI assistance takes about 35 seconds to start a game, versus 13 seconds with AI-assistance (the orange dots on the screen represent where the robot is tapping on the screen):
Monkey Actions without AI assistance starting a game.
AI-assisted Monkey Actions starting a game.
Firebase Authentication provides an end-to-end identity solution for your app, which allows you to authenticate and verify users with passwords, phone numbers and popular federated identity providers like Google, Facebook and Twitter.
For iOS games developers, we are pleased to announce that Firebase Authentication now also supports Game Center Authentication, so if an iOS user is already signed into Game Center, they can also sign-in to your app without having to go through an additional sign-in flow.
We'll be at booth #P1501 all throughout the conference starting from March 20th - March 22nd, and have a special presentation on the Cloud Developer Day track on Wednesday at 5:00PM! We hope to see you there.
And of course, you can always learn more about Firebase for Games at:
https://firebase.google.com/games/
