In today's fast-moving world, people have come to expect mobile apps to be intelligent - adapting to users' activity or delighting them with surprising smarts. As a result, we think machine learning will become an essential tool in mobile development. That's why on Tuesday at Google I/O, we introduced ML Kit in beta: a new SDK that brings Google's machine learning expertise to mobile developers in a powerful, yet easy-to-use package on Firebase. We couldn't be more excited!
Getting started with machine learning can be difficult for many developers. Typically, new ML developers spend countless hours learning the intricacies of implementing low-level models, using frameworks, and more. Even for the seasoned expert, adapting and optimizing models to run on mobile devices can be a huge undertaking. Beyond the machine learning complexities, sourcing training data can be an expensive and time consuming process, especially when considering a global audience.
With ML Kit, you can use machine learning to build compelling features, on Android and iOS, regardless of your machine learning expertise. More details below!
If you are a beginner or want to implement a solution quickly, ML Kit gives you five ready-to-use ("base") APIs that address common mobile use cases:
With these base APIs, you simply pass in data to ML Kit and get back an intuitive response. For example: Lose It!, one of our early users, used ML Kit to build several features in the latest version of their calorie tracker app. Using our text recognition based API and a custom built model, their app can quickly capture nutrition information from product labels to input a food's content from an image.
ML Kit gives you both on-device and Cloud APIs, all in a common and simple interface, allowing you to choose the ones that fit your requirements best. The on-device APIs process data quickly and will work even when there's no network connection, while the cloud-based APIs leverage the power of Google Cloud Platform's machine learning technology to give a higher level of accuracy.
See these ready-to-use APIs in the Firebase console:
Heads up: We're planning to release two more APIs in the coming months. First is a smart reply API allowing you to support contextual messaging replies in your app, and the second is a high density face contour addition to the face detection API. Sign up here to give them a try!
If you're seasoned in machine learning and you don't find a base API that covers your use case, ML Kit lets you deploy your own TensorFlow Lite models. You simply upload them via the Firebase console, and we'll take care of hosting and serving them to your app's users. This way you can keep your models out of your APK/bundles which reduces your app install size. Also, because ML Kit serves your model dynamically, you can always update your model without having to re-publish your apps.
But there is more. As apps have grown to do more, their size has increased, harming app store install rates, and with the potential to cost users more in data overages. Machine learning can further exacerbate this trend since models can reach 10's of megabytes in size. So we decided to invest in model compression. Specifically, we are experimenting with a feature that allows you to upload a full TensorFlow model, along with training data, and receive in return a compressed TensorFlow Lite model. The technology behind this is evolving rapidly and so we are looking for a few developers to try it and give us feedback. If you are interested, please sign up here.
Since ML Kit is available through Firebase, it's easy for you to take advantage of the broader Firebase platform. For example, Remote Config and A/B testing lets you experiment with multiple custom models. You can dynamically switch values in your app, making it a great fit to swap the custom models you want your users to use on the fly. You can even create population segments and experiment with several models in parallel.
Other examples include:
We can't wait to see what you'll build with ML Kit. We hope you'll love the product like many of our early customers:
Get started with the ML Kit beta by visiting your Firebase console today. If you have any thoughts or feedback, feel free to let us know - we're always listening!
It’s hard to believe that it’s only been two years since we expanded Firebase at I/O 2016 from a set of backend services to a full app development platform. In the time since then, it’s been humbling to watch the developer community embrace Firebase. We now have 1.2 million apps actively using Firebase every month!
No matter how much we grow, our mission remains the same: to help mobile app teams be successful across every stage of your development cycle, from building your app, to improving app quality, to growing your business.
Having such an amazing developer community is both a huge honor and a huge responsibility. Thank you for trusting us with your apps. It’s inspiring to hear the stories about what you’ve built with Firebase and your success is the reason we’re excited to come to work everyday!
Today, we’re announcing a number of improvements to Firebase. Let’s take a look.
Machine learning just got easier for mobile developers. We’re excited to announce ML Kit, an SDK available on Firebase that lets you bring powerful machine learning features to your app whether it's on Android or iOS, and whether you're an experienced ML developer or you're just getting started.
ML Kit comes with a set of ready-to-use APIs for common use cases: recognizing text, detecting faces, scanning barcodes, labeling images and recognizing landmarks. These APIs can run on-device or in the cloud, depending on the functionality. The on-device APIs process data quickly and will work even when there's no network connection, while the cloud-based APIs leverage the power of Google Cloud Platform's machine learning technology to give a higher level of accuracy. You can also bring in your own TensorFlow Lite models for advanced use-cases, and ML Kit will take care of the hosting and serving, letting you focus on building your app.
These five APIs are just the first step. We'll be rolling out more in the future and if you want to be involved as an early tester, please visit our signup form to join the waiting list.
Whether you're building on Android or iOS, you can improve the experience for your users by leveraging machine learning. And with ML Kit, we hope to make it easy for developers of all experience levels to get started today. Visit our docs to learn more.
At I/O last year, we launched Performance Monitoring into beta to help you gain insight into your app's performance so you can keep it fast and responsive. Since then, we've seen tremendous adoption. Some of the largest apps in the world — like Flipkart, Ola, and Swiggy — have started using Performance Monitoring and we now report 100 billion performance metrics every day, helping developers improve their app's quality and make their users happy!
Now that the SDK is battle-tested, we've decided to graduate Performance Monitoring out of beta. With this change comes a couple of improvements that you'll see rolling out into the console today.
First, you'll now see an issues feed at the top of the Performance Monitoring dashboard. This feed gives you a quick and easy look at any performance issues occurring across your app, as well as Firebase's opinion on the severity of the issue.
Second, you can now easily identify parts of your app that stutter or freeze. Performance monitoring identifies rendering issues, telling you how many frames are dropped per screen in your app, so you can quickly troubleshoot the issue. If you have apps in the Play store, this is a great way to get detailed information on rendering issues reported in Android vitals, without writing additional code. You can get started with Performance Monitoring today by visiting our documentation.
With Google Analytics for Firebase, you've always been able to see analytics for each of your project's apps. Last year, we added the ability to see your data in real time, with the addition of the StreamView and DebugView reports. Now, you'll notice that we've added real time cards throughout your Analytics reports to give you a better idea of what your users are doing right now.
Analytics is also getting two more upgrades with the addition of project level reporting and flexible filters. Project level reporting lets you see what's happening across all the apps in a project, so you have a more holistic view of your app business, while flexible filters allows you to slice your data more precisely to produce key insights. These updates will be rolling out in the coming weeks.
We're launching another update to the Firebase console today: improved identity and access management. This will allow you to more easily invite others to collaborate on your projects and control what they have access to, all from within the Firebase console.
At Firebase, it's always been vitally important to us to build products that work for development on both Android and iOS. That's why it's particularly exciting to announce that we're expanding Test Lab to include iOS, in addition to Android.
Test Lab provides you with physical and virtual devices that allow you to run tests to simulate actual usage environments. With the addition of Test Lab for iOS, we help you get your app into a high quality state - across both Android and iOS - before you even release it.
Test Lab for iOS will be rolling out over the coming months. If you want to be an early tester of the product, you can sign-up in this form to get on the waiting list today.
It’s been an amazing journey at Firebase so far and we believe that we’re only getting started. By continuing to deepen our integrations with Google Cloud Platform, we aim to make it easy for you to leverage the enormous scale of Google’s infrastructure. We’re also immensely excited about the possibilities that machine learning holds for empowering developers like you. Predictions and ML Kit are the first two steps, but there’s much more we hope to do.
Thank you, as always, for being part of the journey with us. To hear about many of these announcements and more in detail, you can check out our YouTube playlist for recordings of all our talks at Google I/O. If you’re not already part of the Firebase Alpha program, please join and help shape the future of the platform. We can’t wait to see what you build next.
