Getting feedback on your app's releases before they reach the store is critical. App Distribution makes it easy to share pre-release builds of your Android and iOS apps with your testers. You can distribute your apps to testers using the Firebase console, the Firebase CLI, or the App Distribution plugins for Gradle and fastlane.
We've also heard that you'd like to integrate your own tools directly with App Distribution, or implement custom pre-release testing workflows. That's where the Firebase App Distribution REST API comes in. With this API, you can build custom logic into your team's tools and services to add and remove testers, upload new app binaries, distribute your releases, update release notes, deleting releases, and more.
For example, let's say that your company gives all employees early access to the latest builds of your app. You also have a personnel management tool that automates the employee onboarding and offboarding process. Wouldn't it be nice if employee access was automatically granted and revoked as employees join and leave your company? With the REST API, you can build this functionality directly into your company's workflow.
Let's walk through the process to automatically grant and revoke access.
To access the API, enable the Firebase App Distribution API for your project and choose a way to authenticate your integration. In this scenario, download a service account key and use oath2l to get an access token:
oauth2l fetch \ --credentials /path/to/service-account.json \ --scope https://www.googleapis.com/auth/cloud-platform
Next, in hooks that are triggered when an employee joins or leaves the company, use the testers.batchAdd and testers.batchRemove endpoints to add or remove the employee as a tester in App Distribution using an HTTP request to the API. For example, here's how you can add a new tester using curl:
SERVICE=https://firebaseappdistribution.googleapis.com curl --request POST \ $SERVICE/v1/projects/[PROJECT_NUMBER]/testers:batchAdd \ --header 'Authorization: Bearer [ACCESS_TOKEN]' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \ --data '{"emails":["newemployee@company.com"]}'
Let's say you periodically audit access to your pre-release apps to make sure that only authorized internal testers have access. To automate this audit, you can use the testers.list endpoint to get the list of testers in your Firebase project. You can then write your own logic to compare that to your list of employee email addresses, or check the domains:
curl \ $SERVICE/v1/projects/[PROJECT_NUMBER]/testers \ --header 'Authorization: Bearer [ACCESS_TOKEN]' \ --header 'Accept: application/json'
Tester access and auditing is just one example of custom functionality you can build using the API. Other examples of functionality that you can build include keeping release notes in sync with your bug tracking system, fetching details about your most recent release, adding testers to releases from third-party tools, deleting releases in bulk, or fetching test certificate info for your Android app bundle releases.
App Distribution helps you gain confidence in your releases, and the REST API empowers you to integrate pre-release testing into your existing workflows. So get started with the API today! To learn more, and to try out the API using the embedded API Explorer, see the Firebase App Distribution API documentation.
Try out the API using the API Explorer in the documentation.
When Test Lab was originally launched with Firebase in 2016, it supported only Android devices. At Google I/O 2018 in May, Test Lab launched closed beta support for iOS. This included a limited set of iOS devices and a basic UI.
Building comprehensive tests for Android involves writing code, using Espresso and UIAutomator, that acts as a sort of "remote control" for your app. Similarly, on iOS, testing is performed using XCTests. In both cases, Test Lab can run your tests against actual devices in a cloud-hosted device farm.
At the Firebase Summit in Prague at the end of October, the Test Lab team announced general availability of support for iOS, including ten models of iPhones and iPads running seven different versions of iOS, including iOS 12. We have also improved the iOS documentation and console experience for you.
Test Lab also launched a number of improvements to Robo, a tool which runs fully automated tests against your app running on Android devices. Here's what's new with Robo.
Games are difficult to crawl because they often have a highly customized UI, rather than using system widgets. This makes it difficult for Robo to crawl the game's experience. Now, if Robo detects that the app under test is actually a game, it will perform random taps and swipes in an effort to interact with the game's UI. This can yield useful crash and performance data and is an early but significant step towards more meaningful automated game crawling.
Test Lab now detects and warns if your APK makes use of internal Android APIs. On Android P and newer, using such APIs can crash your app. Whenever such an API is accessed during a Robo crawl a stack trace is recorded in the device logs. This pinpoints the location in your app's code where the violation occurs.
Test Lab now warns developers when it notices that Robo got stuck in a crawl. For example, if the user is presented with a complicated sign-up form or a login screen, it may be difficult for Robo to satisfy the requirements of the form. In situations like this, Robo will suggest an action to the developer to help it continue a full crawl, such as providing test credentials or writing a Robo Script.
If you aren't in the habit of regularly testing your app, consider giving Test Lab a try at no cost using the free daily quota of tests. No coding is necessary to run a Robo test on Android - just upload your APK to get started. And be sure to let us know what you think in the #test-lab channel of the Firebase Slack.
Last year, we launched the beta of Cloud Functions for Firebase to help you build backend functionality for your app without worrying about managing servers. It lets you write in JavaScript or Typescript, deploy to Google's Cloud infrastructure, and execute in response to app events. Today we're pleased to release version 1.0 of the Cloud Functions for Firebase SDK - along with new, frequently requested improvements for the development, testing and monitoring of Functions.
The Firebase SDK for Google Cloud Functions enables you to extend other Firebase products, giving users the features they need in your apps. One of our earliest customers, Posse, makers of the Hamilton app, needed to create a lottery system to give fans a chance to win tickets to the massively popular Broadway play - without the overhead of managing their own servers. When a user enters the lottery, Functions powered logic runs in the cloud with results stored directly to Cloud Firestore. Lottery results are pushed automatically to lucky fans' Hamilton mobile app.
Posse also needed to scale the Hamilton app. When the lottery opens, the app's usage can suddenly surge multifold before normalizing a few minutes later. Functions automatically scales without needing any action from Posse, and their client only pays for the resources they use.
In addition to ticket lotteries, Posse also used Functions to integrate with other Firebase products to:
There's a great deal of things you can do with Functions. For more use cases and sample code, see our popular GitHub repo.
Today's v1.0 release comes with a number of new features based on your feedback - here's the lowdown!
One of your biggest requests was for an easy way to directly call Functions from your client apps. To facilitate that, we're adding a new type of function, along with SDKs for iOS, Android, and web clients. The new type is a Callable HTTPS function that manages an HTTPS request from the client SDK. Callable HTTPS functions drastically reduce the amount of boilerplate code you have to write to make HTTPS requests to Functions.
Here's how it works:
For more, check out our docs!
While Functions Shell gave you the ability to interact with your functions locally, it was not optimal for writing unit tests such as those that run in a CI system. That's why we are launching a new firebase-functions-test npm module that simplifies writing unit tests. firebase-functions-test takes care of the necessary setup and teardown, allowing easy mocking of test data. Now, you can either write completely offline tests that have no side effects, or tests that interact with a development Firebase project - where you can observe the success of actions such as database writes.
Last fall, we introduced the ability to emulate your functions locally via the Firebase CLI through both the "firebase serve --only functions" and "firebase experimental:functions:shell" command. These were experimental features, and now we are officially supporting them. "firebase experimental:functions:shell" has been renamed to "firebase functions:shell", and "firebase serve" will emulate all HTTPS functions by default without the need for the "--only" flag.
After deploying functions, you may be wondering: "What's happening with my functions? How is my app performing? Is anything broken?" With today's release, you can now keep tabs on any errors in your functions via the new health monitor:
Additionally, you can track your functions' performance, latency and memory usage:
The APIs for writing Cloud Functions with the Firebase SDK have changed in this v1.0 SDK release. This means that upgrading to the new SDK will require some updates to any code you've already written during the beta. But don't worry - this doesn't affect any functions you have already deployed. To see what has changed and how to adapt to the new format, just follow our migration guide!
If you're just getting started with the Firebase SDKs for Cloud Functions, try following our step-by-step Codelab and visiting the documentation. There is also a video tutorial to help get you set up using TypeScript as the recommended programming language.
We hope you find these new features helpful. While the Firebase SDKs for Cloud Functions are fully released, the Cloud Functions platform is still in beta. Stay tuned for more updates!
When I think back 10 years or so when I was writing code for a monolithic codebase written in C++, one of the worst memories I have of the time was waiting for a build to complete just to run a couple lines of new code. The pain was so common among engineers that it bore the creation of one of the all-time most popular XKCD comics:
My office at the time didn't have swords to play with, so waiting for a build to complete was really boring.
The pain is similar if I had to deploy my Cloud Functions code every time I want test a function. Fortunately, the Firebase CLI now has a local emulator that works for both HTTPS and other types of functions. This lets me run the functions on my development machine by emulating the events that would trigger it. You can read more about that in the documentation.
One of the emulator's handy features is the fact that it will reload any JavaScript files that change over time. So, if you modify the code of a function, you can test it right away in the emulator without having to restart it:
It's fast and easy, and keeps you "in the zone".
However, things become less easy when you switch from JavaScript to TypeScript. Running TypeScript on Cloud Functions requires a compilation phase to convert the TypeScript into JavaScript so that it can be executed by the emulator. It adds a bit of a hassle:
npm run build
There's an easier way to do this that doesn't involve running a command every time. The TypeScript compiler (tsc) has a special "watch mode" that lets it pick up changes to your TypeScript and automatically compile them. You can run it from the command line by changing to your functions folder and running this command:
tsc
./node_modules/.bin/tsc --watch
With this running alongside the local emulator, which automatically picks up changes to the compiled JavaScript, your development workflow goes back to this:
And, if you're using VSCode, you can even have it directly run tsc in watch mode. You can start it up with Tasks -> Run Build Task..., then choosing "tsc: watch".
It will fire up a command shell right in the IDE, run tsc --watch, and show you the output. You'll see messages like this, along with any compilation errors, any time a TypeScript file changes:
tsc --watch
2:38:38 PM - File change detected. Starting incremental compilation... 2:38:38 PM - Compilation complete. Watching for file changes.
Now, bear in mind that tsc in watch mode won't run TSLint against your code, so be sure to set that up in VSCode so you can always see its recommendations, as I showed in a prior blog post. If you're not linting your code, seriously consider setting that up, because you'll catch potential mistakes that could be costly in production.
It's December, folks, and you know what that means: holiday cheer!
The Firebase Test Lab team is fully invested in making this season the greatest of seasons. Remember Halloween? What a hoot! Also, this happened:
(Note: these are actual Test Lab engineers, in costume, actually beating each other up with foam sticks at a Halloween party. Both get lumps of coal this year.)
We're getting ready for the holidays! So, sit back, pour yourself some eggnog, and read about what's new for your Android testing enjoyment.
Many of you are using Robo to automatically test your apps in Test Lab. Since you don't have to write any code to make it work, it's the gift that keeps on giving. Even better, you can have it fill in specific form fields and push buttons with some easy configuration.
We've found that some apps require more of a nudge to navigate into the parts that need the most testing. (Hey, even Santa needs help from a few elves!) Now, with Robo scripts, you can record a series of actions to take in your app, and play that back before Robo takes over. It works a lot like Espresso Test Recorder, except the output is a JSON file that you upload along with your APK when running a Robo test. With these extra instructions, you can guide your app past introductions or login screens.
Of course, your best bet to maximize the test coverage is writing Espresso tests that drive your app. I heard that it's easier than driving a team of reindeer!
Do you use the screenshots in Test Lab results to check if your app displays correctly? It's a great way to see if you app renders "naughty or nice" on many different types of screens. But if you test with lots of devices in a single test matrix, it can be kind of a pain to sort through all the results to compare the same screen among multiple devices. Now, Test Lab will cluster them together in your test results, so you can see all the different screen sizes, densities, and orientations from your test in a single place.
The Test Lab team is always busy at the North Pole (located at a data center in Atlanta) bringing you new devices to test with. The latest additions are the Sony Xperia XZ Premium, the Moto G4 Play, and the Huawei P8lite, delivered straight to your digital stocking. However, sometimes old toys break and need to be recycled. At the Test Lab workshop, we call that "device deprecation", which means we take old devices out of commission as they become unreliable. To see a (twice-checked) list of devices that are currently available, in addition to those being deprecated, click through to this page. Once a device is marked as "deprecated", it'll remain available for a month, then removed.
Deprecated devices look like this in the Firebase console:
And like this in the gcloud command line (note the "deprecated" tag in red):
You better not pout, you better not cry ‐ these devices served longer than their expected lifetime!
Or, just join us on the Firebase Slack in the #test-lab channel. We're all friendly there, so be good, for goodness sake!
Here's how it works, in a nutshell. We'll use a Realtime Database trigger as an example.
Imagine you have an existing project with a single function in it called makeUppercase. It doesn't have to be deployed yet, just defined in your index.js:
exports.makeUppercase = functions.database.ref('/messages/{pushId}/original').onCreate(event => { const original = event.data.val() console.log('Uppercasing', event.params.pushId, original) const uppercase = original.toUpperCase() return event.data.ref.parent.child('uppercase').set(uppercase) })
This onCreate database trigger runs when a new message is pushed under /messages with a child called original, and writes back to that message a new child called uppercased with the original value capitalized.
Now, if you can kick off the emulator shell from your command line using the Firebase CLI:
$ cd your_project_dir $ firebase experimental:functions:shell
Then, you'll see something like this:
i functions: Preparing to emulate functions. ✔ functions: makeUppercase firebase>
That firebase prompt is waiting there for you to issue some commands to invoke your makeUppercase function. The documentation for testing database triggers says that you can use the following syntax to invoke the function with incoming data to describe the event:
makeUppercase('foo')
This emulates the trigger of an event that would be generated when a new message object is pushed under /messages that has a child named original with the string value "foo". When you run this command in the shell, it will generate some output at the console like this:
info: User function triggered, starting execution info: Uppercasing pushId1 foo info: Execution took 892 ms, user function completed successfully
Notice that the console log in the function is printed, and it shows that the database path wildcard pushId was automatically assigned the value pushId1 for you. Very convenient! But you can still specify the wildcard values yourself, if you prefer:
makeUppercase('foo', {params: {pushId: 'custom_push_id'}})
After emulating this function, if you look inside the database, you should also see the results of the function on display, with /messages/{pushId}/uppercased set to the uppercased string string value "FOO".
You can simulate any database event this way (onCreate, onDelete, onUpdate, onWrite). Be sure to read the docs to learn how to invoke them each correctly.
In addition to database triggers, you can also emulate HTTPS functions, PubSub functions, Analytics functions, Storage functions, and Auth functions, each with their own special syntax.
The Cloud Functions shell is currently an experimental offering, and as such, you may experience some rough edges. If you encounter a problem, please let us know by filing a bug report. You can also talk to other Cloud Functions users on the Firebase Slack in the #functions channel.
Typing the function invocation each time can be kind of a pain, so be sure to take advantage of the fact that you can navigate and repurpose your invocation history much like you would your shell's command line using the arrow keys.
Also note that the shell is actually a full node REPL that you can use to execute arbitrary JavaScript code and use special REPL commands and keys. This can be useful for scripting some of your test code.
Since you can execute arbitrary code, you can also dynamically load and execute code from other files using the require() function that you're probably already familiar with.
And lastly, if you're like me, and you prefer to use a programmer's editor such as VS Code to write your all JavaScript, you can easily emulate functions by sending code you want to run to the Firebase CLI. This command will run test code from a file redirected through standard input:
$ firebase experimental:functions:shell < tests.js
Happy testing!
Sometimes the worst bugs to track down are the ones that seem to be impossible to reproduce. Or worse, inconsistent performance problems that can cause "Application Not Responding" errors in Android apps. No matter how much test code you write, these types of errors seem to have a way of sneaking into your app and causing problems for your users. However, with some clever use of Android's StrictMode API, alongside Firebase Test Lab for Android, you can find out about these problems before they reach production!
Over the years since Android was first available, a number of best practices have been observed for writing solid apps. For example, you shouldn't be performing blocking I/O on the main thread, and you shouldn't store references to Activity objects that are held after the Activity is destroyed. While it's likely that no one will force you to observe these practices (and you may never see a problem during development), enabling StrictMode in your app will let you know where you've made a mistake. These are called "policy violations", and you configure these policies with code in your app.
Personally, I don't want any of my code to cause any StrictMode violations. In fact, I'd like to consider them bugs that are just as serious as a regular crash, because they could crash my app in some cases! And, in fact, I can configure all StrictMode violations to crash the app. The Java code for that looks like this:
private static final StrictMode.ThreadPolicy FATAL_THREAD_POLICY = new StrictMode.ThreadPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build(); private static final StrictMode.VmPolicy FATAL_VM_POLICY = new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .penaltyDeath() .build(); public static void enableFatalStrictMode() { if (shouldEnableFatalStrictMode()) { StrictMode.setThreadPolicy(FATAL_THREAD_POLICY); StrictMode.setVmPolicy(FATAL_VM_POLICY); } }
Here I have both a ThreadPolicy (for the main thread) and a VmPolicy (for the entire app) that look for all known violations, and will both log that violation and crash the app. Take a look through the javadoc for those builders to learn about all the situations they can catch.
Also notice that I'm enabling the policies conditionally. You probably don't want an accidental violation to crash on your users in production, so the method shouldEnableFatalStrictMode() typically looks like this, which activates it for your debug builds only:
shouldEnableFatalStrictMode()
private static boolean shouldEnableFatalStrictMode() { return BuildConfig.DEBUG; }
What I'd like to do is take this further and have StrictMode crash my app also when it's running in Firebase Test Lab. This is handy because the crash will fail the test, and I'll get a report about that in the test results. For that to happen, I can change the method to query a system setting on the device that's unique to devices in Test Lab. Note that this requires an Android Context to obtain a ContentResolver:
private static boolean shouldEnableFatalStrictMode() { ContentResolver resolver = context.getContentResolver(); String isInTestLab = Settings.System.getString(resolver, "firebase.test.lab"); return BuildConfig.DEBUG || "true".equals(isInTestLab); }
Now, my instrumented tests (and the automated Robo test) will crash with any StrictMode violations, and I'll see those very clearly in my Test Lab report. Here's an app that crashed in Test Lab with a StrictMode violation during a Robo test:
This doesn't tell you exactly what happened, though. Digging into the logs in the test report, I see the details of the violation right before the crash:
D/StrictMode(6996): StrictMode policy violation; ~duration=80 ms: android.os.StrictMode$StrictModeDiskReadViolation: policy=327743 violation=2 D/StrictMode(6996): at android.os.StrictMode$AndroidBlockGuardPolicy.onReadFromDisk(StrictMode.java:1415) D/StrictMode(6996): at java.io.UnixFileSystem.checkAccess(UnixFileSystem.java:251) D/StrictMode(6996): at java.io.File.exists(File.java:807) D/StrictMode(6996): at android.app.ContextImpl.getDataDir(ContextImpl.java:2167) D/StrictMode(6996): at android.app.ContextImpl.getPreferencesDir(ContextImpl.java:498) D/StrictMode(6996): at android.app.ContextImpl.getSharedPreferencesPath(ContextImpl.java:692) D/StrictMode(6996): at android.app.ContextImpl.getSharedPreferences(ContextImpl.java:360) D/StrictMode(6996): at android.content.ContextWrapper.getSharedPreferences(ContextWrapper.java:167) D/StrictMode(6996): at com.google.firebasesandbox.MainActivity.onCreate(MainActivity.java:25) D/StrictMode(6996): at android.app.Activity.performCreate(Activity.java:6982) D/StrictMode(6996): at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1213) D/StrictMode(6996): at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2770) D/StrictMode(6996): at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2892) D/StrictMode(6996): at android.app.ActivityThread.-wrap11(Unknown Source:0) D/StrictMode(6996): at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1593) D/StrictMode(6996): at android.os.Handler.dispatchMessage(Handler.java:105) D/StrictMode(6996): at android.os.Looper.loop(Looper.java:164) D/StrictMode(6996): at android.app.ActivityThread.main(ActivityThread.java:6541) D/StrictMode(6996): at java.lang.reflect.Method.invoke(Native Method) D/StrictMode(6996): at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240) D/StrictMode(6996): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)
So we've discovered here that an activity's onCreate() called getSharedPreferences(), and this method ends up reading the device filesystem. Blocking the main thread like this could be the source of some jank in the app, so it's worth figuring out a good way to move that I/O to another thread.
onCreate()
getSharedPreferences()
Now we have a way to check for StrictMode violations during a test, but there's still a few more details to work out.
If you do this immediately when the app launches (in an Application subclass or a ContentProvider), there is a subtle bug in some versions of Android that may need to be worked around in order to avoid losing your StrictMode policies when the first Activity appears.
There might be some times when you know there is a violation, but you can't fix the code right away for whatever reason (for example, it's in a library you don't control). In that case, you might want to temporarily suspend the crashing behavior and simply log it instead. To temporarily suspend the crash, you can define a couple other non-fatal policies and swap them in where you know there's an issue to ignore:
private static final StrictMode.ThreadPolicy NONFATAL_THREAD_POLICY = new StrictMode.ThreadPolicy.Builder() .detectAll() .penaltyLog() .build(); private static final StrictMode.VmPolicy NONFATAL_VM_POLICY = new StrictMode.VmPolicy.Builder() .detectAll() .penaltyLog() .build(); public static void disableStrictModeFatality() { StrictMode.setThreadPolicy(NONFATAL_THREAD_POLICY); StrictMode.setVmPolicy(NONFATAL_VM_POLICY); }
Similarly, you may not want to track all the possible violations all the time. In that case, you might want to instruct the policy builder to only detect a subset of the potential issues.
While StrictMode violations are not always the worst thing for your users, I've always found it educational to enable StrictMode in order to find out where my app might be causing problems. In combination with Firebase Test lab, it's a great tool for maximizing the quality of your app.
Testing your application is a great way to help maximize its quality, and many of you know that Firebase Test Lab for Android has some useful tools for testing Android apps. If you're the type of engineer who likes to maximize test coverage by writing instrumented tests (and regular unit tests), you can send those to Test Lab for execution. Even if you don't like writing tests, you have some options. You can record instrumented tests by interacting with your app using Espresso Test Recorder in Android Studio. And there's almost no effort required at all to run a Robo test that automatically crawls your app.
These tests are helpful for data-driven apps because there are robust test frameworks that understand how to navigate the Android platform widgets used to receive input and display data on screen. However, most games don't work with platform widgets. Games typically take over the screen using their own UI elements, and provide their own touch controls. As a result, it's extremely difficult to write instrumented tests for testing games, and Robo test won't know how to navigate the game at all.
To help deal with these challenges with testing games, the Test Lab team has come up with a way for game developers to test their games effectively across many of the devices that Test Lab offers. It's a new type of test called a Game Loop Test, and it's available today in beta.
If you've seen arcade video games operate, you know that they're always showing something on screen, typically some automated demo of the game, called "attract mode". With Firebase Test Lab, game developers can now use this concept of attract mode to construct test scenarios, and Test Lab will arrange for those scenarios to be invoked in sequence. This gives developers the opportunity to test a wide variety of game levels and situations in a single test run on all the devices provided by Test Lab. If there are any problems with a scenario, you'll find out which ones are problematic, so you can focus your efforts on improving that specific case.
Even better, Test Lab now provides performance data with every report. Of particular interest to game developers is a graph of the rendered frame rate over time throughout the test, tied to a video of the device display. This helps you quickly identify the parts of your game that aren't rendering at an acceptable FPS.
In addition to FPS, there are other performance metrics available for all apps. You'll be able to see your app's CPU utilization, memory usage, and network ingress and egress. Notice how you can jump directly to the point in the video where you're observing performance problems.
If you're a game developer, now is a great time to check out Firebase Test Lab for Android to see what it can do to help the quality of your game.
