Today we are announcing the release of two new features to ML Kit: Language Identification and Smart Reply.
You might notice that both of these features are different from our existing APIs that were all focused on image/video processing. Our goal with ML Kit is to offer powerful but simple-to-use APIs to leverage the power of ML, independent of the domain. As such, we are excited to expand ML Kit with solutions for Natural Language Processing (NLP)!
NLP is a category of ML that deals with analyzing and generating text, speech, and other kinds of natural language data. We're excited to start out with two APIs: one that helps you identify the language of text, and one that generates reply suggestions in chat applications. Both of these features work fully on-device and are available on the latest version of the ML Kit SDK, on iOS (9.0 and higher) and Android (4.1 and higher).
Generate reply suggestions based on previous messages
A new feature popping up in messaging apps is to provide the user with a selection of suggested responses, either as actions on a notification or inside the app itself. This can really help a user to quickly respond when they are busy or a handy way to initiate a longer message.
With the new Smart Reply API you can now quickly achieve the same in your own apps. The API provides suggestions based on the last 10 messages in a conversation, although it still works if only one previous message is available. It is a stateless API that fully runs on-device, so we don't keep message history in memory nor send it to a server.
textPlus app providing response suggestions using Smart Reply
We have worked closely with partners like textPlus to ensure Smart Reply is ready for prime time and they have now implemented in-app response suggestions with the latest version of their app (screenshot above).
Adding Smart Reply to your own app is done with a simple function call (using Kotlin in this example):
val smartReply = FirebaseNaturalLanguage.getInstance().smartReply smartReply.suggestReplies(conversation) .addOnSuccessListener { result -> if (result.status == SmartReplySuggestionResult.STATUS_NOT_SUPPORTED_LANGUAGE) { // The conversation's language isn't supported, so the // the result doesn't contain any suggestions. } else if (result.status == SmartReplySuggestionResult.STATUS_SUCCESS) { // Task completed successfully // ... } } .addOnFailureListener { // Task failed with an exception // ... }
After you initialize a Smart Reply instance, call suggestReplies with a list of recent messages. The callback provides the result which contains a list of suggestions.
suggestReplies
result
For details on how to use the Smart Reply API, check out the documentation.
Tell me more ...
Although as a developer, you can just pick up this new API and easily get it integrated in your app, it may be interesting to reveal a bit on how it works under the hood. At the core of Smart Reply is a machine-learned model that is executed using TensorFlow Lite and has a state-of-the-art modern architecture based on SentencePiece text encoding[1] and Transformer[2].
However, as we realized when we started development of the API, the core suggestion model is not all that’s needed to provide a solution that developers can use in their apps. For example, we added a model to detect sensitive topics, so that we avoid making suggestions in response to profanity or in cases of personal tragedy/hardship. Also, we included language identification, to ensure we do not provide suggestions for languages the core model is not trained on. The Smart Reply feature is launching with English support first.
Identify the language of a piece of text
The language of a given text string is a subtle but helpful piece of information. A lot of apps have functionality with a dependency on the language: you can think of features like spell checking, text translation or Smart Reply. Rather than asking a user to specify the language they use, you can use our new Language Identification API.
ML Kit recognizes text in 110 different languages and typically only requires a few words to make an accurate determination. It is fast as well, typically providing a response within 1 to 2 ms across iOS and Android phones.
Similar to the Smart Reply API, you can identify the language with a function call (using Kotlin in this example):
val languageIdentification = FirebaseNaturalLanguage.getInstance().languageIdentification languageIdentification .identifyLanguage("¿Cómo estás?") .addOnSuccessListener { identifiedLanguage -> Log.i(TAG, "Identified language: $identifiedLanguage") } .addOnFailureListener { e -> Log.e(TAG, "Language identification error", e) }
The identifyLanguage functions takes a piece of a text and its callback provides a BCP-47 language code. If no language can be confidently recognized, ML Kit returns a code of und for undetermined. The Language Identification API can also provide a list of possible languages and their confidence values.
identifyLanguage
und
For details on how to use the Language Identification API, check out the documentation.
Get started today
We're really excited to expand ML Kit to include Natural Language APIs. Give the two new NLP APIs a spin today and let us know what you think! You can always reach us in our Firebase Talk Google Group.
As ML Kit grows we look forward to adding more APIs and categories that enables you to provide smarter experiences for your users. With that, please keep an eye out for some exciting ML Kit announcements at Google I/O.
When we think of scaling we usually imagine spiky charts of users hitting a database or processing computationally expensive queries. What we don't always think about it is deleting data. Handling large amounts of deletes is an important part of scaling a database. Imagine a system that's required to delete historical records at a specific deadline. If these records are hundreds of gigabytes in size, it will likely be difficult to delete them all without bogging the database down for the rest of its users. This exact scenario hasn't always been easy with the Firebase Realtime Database, but we're excited to say that it just got a lot easier.
Today, we're introducing a new way to efficiently perform large deletes!
If you want to delete a large node, the new recommended approach is to use the Firebase CLI (> v6.4.0). The CLI automatically detects a large node and performs a chunked delete efficiently.
$ firebase database:remove /path/to/delete
Keep in mind that in order to delete a large node, the Firebase CLI has to break it down into chunks. This means that clients can see a partial state where part of the data is missing. Writes in the path that is being deleted will still succeed, but the CLI tool will eventually delete all data at this path. This behavior is acceptable if no app depends on this node. However, if there are active listeners within the delete path, please make sure the listener can gracefully handle partial documents.
If you want consistency and fast deletion, consider using a special field, a.k.a a tombstone to mark this document as hidden, and then run a cloud function cron job to asynchronously purge the data. You can use Firebase Rules to disallow access to hidden documents.
We've also added a configuration option (defaultWriteSizeLimit) to the Realtime Database that allows you to specify a write size limit. This limit allows you to prevent operations (large deletes and writes) from being executed on your database if they exceed this limit.
defaultWriteSizeLimit
You can use this option to prevent app code from accidentally triggering a large operation, which would make your app unresponsive for a time. For more detail, please see our documentation about this option.
You can check and update the configuration via the CLI tool (version 6.4.0 and newer). There are four available thresholds. You can pick appropriate threshold based on your application requirement
small
write
medium
large
unlimited
Note: The target time is not a guaranteed cutoff off. The estimated time may be off from the actual write time.
$ firebase database:settings:set defaultWriteSizeLimit unlimited --instance <database-name> $ firebase database:settings:get defaultWriteSizeLimit --instance <database-name>
For REST requests, you can override defaultWriteSizeLimit with the writeSizeLimit query parameter. In addition, REST queries support a special writeSizeLimit value:
writeSizeLimit
tiny
firebase database:remove
For example:
$ curl -X PUT \ "https://<database-name>.firebaseio.com/path.json?writeSizeLimit=medium"
The default defaultWriteSizeLimit for new databases is large. In order to avoid affecting existing apps, the setting will remain at unlimited for existing projects for now.
We do want to extend this protection to everyone. So this summer (June~August, 2019), will set defaultWriteSizeLimit to large for existing databases that have not configured defaultWriteSizeLimit. To avoid disruption, we will exclude any databases that have triggered at least one large delete in the past three months.
defaultWriteSizeLimit.
These controls can help you keep your apps responsive and your users happy. We suggest setting defaultWriteSizeLimit for your existing apps today.
Let us know what you think of this new feature! Leave a message in our Google group.
In Firebase Crashlytics, you can view crashes and non-fatals by versions. Several of our customers take advantage of this filtering, especially to focus on their latest releases.
But sometimes too many versions can be a bad thing. Firebase Crashlytics - by default - shows you the last 100 versions we have seen. If you have a lot of debug versions created by developers, and by continuous Integration and deployment pipeline, you might soon start to not see the versions that really matter e.g., production builds.
So how do you make sure that this does not happen? Disabling Crash reporting initialization for debug builds is the simplest way of achieving this. Let's explore how to do this on iOS and Android.
For iOS apps, first check if you are manually initializing Crashlytics (this happens for Fabric apps that were linked to a Firebase app).
If you use Swift, search for the line Fabric.with([Crashlytics.self]) in AppDelegate.swift. If this line is present, then you are manually initializing Crashlytics, otherwise you are using automatic initialization.
Fabric.with([Crashlytics.self])
If you use ObjectiveC, search for the line [Fabric with:@[[Crashlytics class]]]; in AppDelegate.m. If this line is present, then you are manually initializing Crashlytics, otherwise you are using automatic initialization.
[Fabric with:@[[Crashlytics class]]];
For apps that are using manual initialization, you can just not initialize Crashlytics for DEBUG versions.
For Swift
#if !DEBUG Fabric.with([Crashlytics.self]) #endif
For ObjectiveC
#if !DEBUG [Fabric with:@[[Crashlytics class]]]; #endif
Firebase Crashlytics apps are automatically initialized by Firebase. You can turn off automatic collection with a new key to your Info.plist file:
Info.plist
firebase_crashlytics_collection_enabled
no
Then you can initialize it as shown in the examples above for Swift and ObjectiveC
For Android apps, first check if you are manually initializing Crashlytics (this happens for Fabric apps that were linked to a Firebase app). Search for Fabric.with in your project. If this line is present, then you are manually initializing Crashlytics. Otherwise, Crashlytics is being automatically initialized through Firebase.
Fabric.with
To disable Crashlytics in debug builds, you can make use of the BuildConfig.DEBUG flag. Edit the Fabric.with statement you found previously, adding a check for the DEBUG flag:.
BuildConfig.DEBUG
if (!BuildConfig.DEBUG) { Fabric.with(this, new Crashlytics()); }
Turn off Crashlytics initialization in your debug builds by creating a debug folder in your src directory and creating an AndroidManifest.xml with this snippet:
debug
src
<manifest xmlns:android="http://schemas.android.com/apk/res/android"> <application> <meta-data android:name="firebase_crashlytics_collection_enabled" android:value="no" /> </application> </manifest>
This snippet will be merged into the manifest of all of your debug variants, and will disable Crashlytics in those builds. If you'd prefer finer-grained control, you can use this approach for any variants you'd like to exclude.
Firebase Auth client SDKs make it possible to sign in as a Firebase user from federated identity providers, including Google, Facebook, and Twitter. The Firebase Auth team is always looking for opportunities to improve the auth experience for developers and users. We know that more sign-in options mean more opportunities to create the best app experience. That's why we are pleased to announce that you can now sign in to Firebase Auth using Microsoft and Yahoo!
Firebase Auth has added two new federated identity providers! Today, Microsoft and Yahoo join our growing list of providers. Applications that rely on these providers no longer have to handle custom credentials on the backend. With this simplified auth flow, developers can spend less time on implementing authentication and more time to spend on the core features of your application. Developers who implement Microsoft or Yahoo sign in for Firebase Auth will also get the benefit of a new, simpler way to get the provider credentials.
Signing in to Firebase with a given identity provider requires garnering a credential from that provider. This often involves including the provider's SDK and implementing the provider's sign-in methods before passing the credentials to Firebase Auth. For some providers, this can be particularly difficult on a native client, especially ones which do not support their own native SDKs. In order to remove the headache of implementing sign-in flows for these identity providers, we now offer generic OAuth2 provider sign-in.
Generic identity provider sign-in provides an easy means for developers to get that credential and use it to sign in by popping up a ChromeCustomTab on Android or a SafariViewController on iOS which will use the Web flow to generate the Identity Provider Credential, and then continue the sign in. Generic sign-in gives you many benefits:
Generic sign-in is available for Microsoft and Yahoo, and will be a feature of other identity providers in the future.
Ready to get started with Microsoft and Yahoo sign-in for Firebase Auth? Check out the guides linked below:
Microsoft Authentication for iOS
Microsoft Authentication for Android
Microsoft Authentication for Web
Yahoo Authentication for iOS
Yahoo Authentication for Android
Yahoo Authentication for Web
Quickstart for iOS
Quickstart for Android
Quickstart for Web
Hey, there Firebase Developers! We wanted to let you know of a new feature we added to Cloud Firestore, and that's the ability to increment (or decrement) numeric values directly in the database! Why is this a big deal? Because previously, if you wanted to increment a value in the database, you would need to use a transaction to keep it safe from concurrent writes.
For instance, imagine that your fitness app has a feature that allows groups of friends to pool their steps together for a "team step counter". Then, imagine Bob and Alice both have their current team's total steps recorded locally and try to add their values to it…
If both clients send down these new values at around the same time, it's possible for Alice's changes to be written first...
...and then for Bob's changes to come in and overwrite Alice's changes.
In the past, if you wanted to prevent this from happening, you would have to use a transaction. And while transactions are still a fine solution, they are a little more difficult to write, they don't support cases where the user is offline, and frankly, they seem a bit heavy-handed for something as simple as adding two numbers together.
So now, if you want Bob to record his 500 steps in the app, you can simply do that by asking the server to increment the step_counter value. In an iOS app, for instance, you would write code that looks a little something like this.
document("fitness_teams/Team_1"). updateData(["step_counter" : FieldValue.increment(500)])
With this call, the database would instantly make the change based on whatever value it has. So even if Alice sneaks in a change before Bob's request reaches the server, everything still works.
Now, there are two things to keep in mind when it comes to performing these operations:
First of all, if you do want to add some logic to your operation (like, for instance, making sure this new value doesn't go over or under a certain limit), you'll still need to use a transaction. Luckily, we will soon release a fantastic video all about transactions that I can heartily recommend as a completely unbiased observer.
Second, don't forget that documents are still limited to a sustained write limit of 1 QPS per document. I know it's tempting to look at these numeric functions and start making counters left and right in your database, but if you think these are values you're going to be updating more than once per second, you're still going to want to rely on something like our distributed counter solution.
As always, if you have questions, feel free to contact us on any of our support channels, or post questions on Stack Overflow with the google-cloud-firestore and firebase tags. Good luck, and have fun!
google-cloud-firestore and firebase tags
