[Editor's note: Smash.gg is an esports platform used by players and organizers worldwide, running nearly 2,000 events per month with 60,000+ competitors, and recently hosted brackets for EVO 2017, the world's largest fighting game tournament. This is its first post in a multi-part series about migrating to Google Cloud Platform (GCP) -- what got them interested in GCP, why they migrated to it, and a few of the benefits they've seen as a result. Stay tuned for future posts that will cover more technical details about migrating specific services.]
Players in online tournaments running on smash.gg need to be able to interact in real time. Both entrants must confirm that they are present, set up the game, and reach a consensus on the results of the match. They also need a simple chat service to resolve any issues with joining or reporting the match, to talk to one another and to tournament moderators.
We built our initial implementation of online match reporting with an off-the-shelf chat service and UI interactions that weren't truly real-time. When the chat service failed in a live tournament, it became clear that we needed a better solution. We looked into building our own using a websocket-based approach, and a few services like PubNub and Firebase. Ultimately, we decided to launch with Firebase because it's widely used, is backed by Google, and is incredibly well-priced.
Two players checking into, setting up, and reporting an online match using the Firebase Realtime Database for real-time interactions.
We got our start with Firebase in May, 2016. Our first release used the Firebase Realtime Database as a kind of real-time cache to keep match data in sync between both entrants. When matches were updated or reported on our backend, we also wrote the updated match data to Firebase. We use React and Flux so we made a wrapper component to listen to Firebase and dispatch updated match data to our Flux stores. Implementing a chat service with Firebase was similarly easy. Using Firechat as inspiration, it took us about a day to build the initial implementation and another day to make it production-ready.
Compared with rolling our own solution, Firebase was an obvious choice given the ease of development and time/financial cost savings. Ultimately, it reduced the load on our servers, simplified our reporting flow, and made the match experience truly real-time. Later that year, we started using Firebase Cloud Messaging (FCM) to send browser push notifications using Cloud Functions triggers as Firebase data changed (e.g., to notify admins of moderator requests). Like the Realtime Database, Cloud Functions was incredibly easy to use and felt magical the first time we used it. Cloud Functions also gave us a window into how well Firebase interacts with Google Cloud Platform (GCP) services like Cloud Pub/Sub and Google BigQuery.
In March of 2017 we attended Google Cloud Next '17 for the Cloud Functions launch. There, we saw that other GCP products had a similar focus on improving the developer experience and lowering development costs. Current products like Pub/Sub, Stackdriver Trace and Logging, and Google Cloud Datastore solved some of our immediate needs. Out of the box, these services gave us things that we were planning to build to supplement products from our existing service provider. And broadly speaking, GCP products seemed to focus on improving core developer workflows to reduce development and maintenance time. After seeing some demos of the products interacting (e.g., Google Container Engine and App Engine with Stackdriver Trace/Logging, Stackdriver with Pub/Sub and BigQuery), we decided to evaluate a full migration.
We started migrating our application in mid May, using the following services: Container Engine, Pub/Sub, Google Cloud SQL, Datastore, BigQuery, and Stackdriver. During the migration, we took the opportunity to re-architect some of our core services and move to Kubernetes. Most of our application was already containerized but had previously been running on a PaaS-like service so Kubernetes was a fairly dramatic shift. While Kubernetes had many benefits (e.g., industry standard, more efficient use of cloud instances, application portability, and immutable infrastructure defined in code), we also lost some top-level application metrics that our previous PaaS service had provided: for instance overall Requests Per Second (RPS), RPS by status, and latency. We were able to easily recreate these graphs from our container logs using log-based metrics and logs export from Stackdriver to BigQuery. You could also do this using other services, but our GCP-only approach was a quick and mostly free way for us to get to parity while experimenting with GCP services.
Request timing and analysis using Stackdriver Trace was another selling point in GCP that we didn't have with our previous service. However, at the time of our migration, the Trace SDK for PHP (our backend services are in PHP, but I promise it's nice PHP!) didn't support asynchronous traces. The Google Cloud SDK for PHP has since added async trace support, but we were able to build async tracing by quickly gluing some GCP services together:
The Google Cloud SDK is certainly a more appropriate solution for tracing in production, but the fact that this combination of services worked well speaks to how easy it is to develop in GCP.
After running our production environment on GCP for a month, we've saved both time and money. Overall costs are ~10% lower without any Committed Use Discounts, with capacity to spare. Stackdriver logging/monitoring, Container Engine, and Kubernetes have made it easier for our engineers to perform DevOps tasks, leveling up our entire team. And being able to search all our logs in one centralized place allows us to easily cross-reference logs from multiple systems, making it possible to track down root causes of issues much faster. This combined with fully-managed, usage-priced services like Datastore and Firebase means development on GCP is easier and more accessible to all of our engineers. We're really glad we migrated to GCP, and look forward to telling you more about how we did it in future posts. Meanwhile, if you're a developer who loves competitive play and would like to help us build cool things on top of GCP, we'd love to hear from you. We recently closed our Series A from Spark Capital, Accel, and Horizon Ventures, and we're hiring!
The Firebase Admin SDK for Go is now generally available. This is the fourth programming language to join our growing family of Admin SDKs, which already includes support for Java, Python and Node.js. Firebase Admin SDKs enable application developers to programmatically access Firebase services from trusted environments. They complement the Firebase client SDKs, which enable end users to access Firebase from their web browsers and mobile devices. The initial release of the Firebase Admin SDK for Go comes with some Firebase Authentication features: custom token minting and ID token verification.
Similar to the other Firebase Admin SDKs, the Go Admin SDK can be initialized with a variety of authentication credentials and client options. The following code snippet shows how to initialize the SDK using a service account credential obtained from the Firebase console or the Google Cloud console:
import ( "golang.org/x/net/context" firebase "firebase.google.com/go" "google.golang.org/api/option" ) opt := option.WithCredentialsFile("path/to/key.json") app, err := firebase.NewApp(context.Background(), nil, opt)
If you are running your code on Google infrastructure, such as Google App Engine or Google Compute Engine, the SDK can auto-discover application default credentials from the environment. In this case you do not have to explicitly specify any credentials when initializing the Go Admin SDK:
import ( "golang.org/x/net/context" firebase "firebase.google.com/go" ) app, err := firebase.NewApp(context.Background(), nil)
The initial release of the Firebase Admin SDK for Go comes with support for minting custom tokens and verifying Firebase ID tokens. The custom token minting allows you to authenticate users using your own user store or authentication mechanism:
client, err := app.Auth() if err != nil { return err } claims := map[string]interface{}{ "premium": true, "package": "gold", } token, err := client.CustomToken("some-uid", claims)
The resulting custom token can be sent to a client device, where it can be used to initiate an authentication flow using a Firebase client SDK. On the other hand, the ID token verification facilitates securely identifying the currently signed in user on your server:
client, err := app.Auth() if err != nil { return err } decoded, err := client.VerifyIDToken(idToken) uid := decoded.UID
To learn more about using the Firebase Admin SDK for Go, see our Admin SDK setup guide.
We plan to further expand the capabilities of the Go Admin SDK by implementing other useful APIs such as user management and Firebase Cloud Messaging. This SDK is also open source. Therefore we welcome you to browse our Github repo and get involved in the development process by reporting issues and sending pull requests. To all Golang gophers out there, happy coding with Firebase!
Every mobile developer needs to take app screenshots in order to have their app listed on the app stores. Like a book cover, screenshots are crucial in depicting the best parts of your app and convincing potential users to download it.
Unfortunately, generating app screenshots is a huge pain because they take a ton of time, especially if your app supports different locales and languages. For example, if you need to take 5 screenshots for your app store listing - but your app supports 20 languages for 6 devices - you'll manually have to take 600 screenshots (5 x 20 x 6)! It makes us shudder to think how many precious hours that would eat up.
fastlane's snapshot tool automates the process of taking screenshots (in the background) so you can focus on building features users love. Today, we're excited to share that snapshot now supports multiple, concurrent simulators for iOS apps in Xcode 9. Taking screenshots just got even faster because you can now generate screenshots for all of your devices at the same time!
Before Xcode 9, only one simulator could be running at a time, which means that you had to run snapshot once for each device you wish to support. While snapshot automated the process of taking screenshots, we wanted to make things even easier.
The launch of Xcode 9 gave us another opportunity to improve snapshot. In Xcode 9, multiple UI tests can run simultaneously, so we added multiple simulator support to snapshot as well. Now, you can take screenshots for all specified devices with a single command, at the same time. This drastically shortens the time it takes to generate your screenshots.
Here's an example:
fastlane's mission is to save you time by automating the cumbersome tasks of app deployment, even as mobile evolves. That's why we're fully committed to updating the fastlane toolset to take advantage of new releases and features - such as Xcode 9.
And since fastlane is open source, we're so thankful that our community also helps us make fastlane better by building and using plugins. In fact, we now have more user-generated plugins available for you to try than native fastlane actions. We recently reorganized these plugins to make it easier to find the right plugins for your unique needs.
We always strive to anticipate your needs and build our tools to be ready for the future. To start using the new version of snapshot, simply update fastlane and run snapshot as you normally would. If you're taking screenshots manually, check out our guide to start using snapshot (and enjoy the extra free time!). As always, we can't wait to hear what you think!
Web site engineers do this to protect their site from spam and abuse from bots, while allowing legitimate human use. Why is protection needed? Maybe you have some backend code that's expensive in time and storage and you only want actual users of your web to access it.
If you have a web site, you can also use reCAPTCHA to protect its services. And, if you're building your site with Firebase Hosting, it's pretty easy to get it integrated with the help of Cloud Functions for Firebase to provide a secure, scalable backend to verify the completion of the reCAPTCHA.
In this blog post, I'll walk you through a few steps that will get you to a very basic integration that you can extend later for your own site. For this walkthrough, I'm assuming you already have some experience with web development, the Firebase console, and the Firebase CLI.
Navigate to the Firebase console and create a new project. There's no need to add billing to this project - you can experiment fully without providing a credit card. Once you create the project, there's nothing else you need to do in the console.
Using the Firebase CLI, make sure you're logged in with the same Google account that you used to create the project:
$ firebase login
Now, create a root project directory and initialize it:
$ mkdir my_project $ cd my_project $ firebase init
When running firebase init, be sure to select both hosting and functions. When you're asked to choose a project, select the one you just created earlier. Take the defaults for every other prompt. You'll end up with a directory structure that contains a public folder for web content, and a functions folder for your backend code.
firebase init
public
functions
For the Cloud Functions backend, we'll need a couple modules from npm to help verify the reCAPTCHA. The reCAPTCHA API requires you to make an HTTP request for verification from your backend, and you can do that with the request and request-promise modules. Pull them into your project like this:
$ cd functions $ npm install request request-promise
Your package.json file should now show those two new modules in addition to firebase-functions and firebase-admin.
package.json
Make sure you can deploy web content by running this deploy command:
$ firebase deploy --only hosting
When this finishes, you'll be given the public URL to your new web site, which will look something like this:
✔ Deploy complete! Project Console: https://console.firebase.google.com/project/your-project/overview Hosting URL: https://your-project.firebaseapp.com
where your-project is the unique id that was given to your project at the time it was created in the console. If you paste the Hosting URL into your browser, you should see a page that says "Firebase Hosting Setup Complete".
your-project
reCAPTCHA requires a couple API keys for operation, one for the web client and one for the server API. You can get those from the reCAPTCHA admin panel, so navigate there. Create a new site and give it a name. Select "reCAPTCHA V2". For domains, put the full hostname of your Firebase Hosting site name (e.g. "your-project.firebaseapp.com").
After you register, you'll be given a Site key and a Secret key. The Site key will be used in your frontend HTML, and the Secret key will be used in your backend hosted by Cloud Functions.
Now we'll add a new HTML page to display the reCAPTCHA. In the public directory in your project, add a new HTML file called recaptcha.html to display the reCAPTCHA. Simply copy and paste the following content directly into that new file:
recaptcha.html
<html> <head> <title>Firebase + reCAPTCHA</title> <script src="https://www.google.com/recaptcha/api.js" async defer></script> <script type="text/javascript"> function dataCallback(response) { console.log("dataCallback", response) window.location.href = "/checkRecaptcha?response=" + encodeURIComponent(response) } function dataExpiredCallback() { console.log("dataExpiredCallback") } </script> </head> <body> <div class="g-recaptcha" data-sitekey="PASTE_YOUR_SITE_KEY_HERE" data-callback="dataCallback" data-expired-callback="dataExpiredCallback"/> </body> </html>
Notice in the body there is a div with the class "g-recaptcha". The first thing you should do here is copy your reCAPTCHA site key into the div's data-sitekey attribute value. This div will get automatically transformed into a reCAPTCHA UI after the first script at the top is loaded. You can read more about that here in the docs.
You can see it right away if you firebase deploy again, then navigate to /recaptcha.html under your Hosting URL. Don't bother dealing with the reCAPTCHA yet, because we still need some backend code to complete the verification!
firebase deploy
The JavaScript code in this page defines two functions dataCallback and dataExpiredCallback. These are referenced in the div, and provide callbacks for the reCAPTCHA to tell you when the reCAPTCHA has been satisfied, or if the user took too long to proceed.
dataCallback
dataExpiredCallback
The important thing to note in dataCallback is that it redirects the browser to another URL in the site with the path /checkRecaptcha, and pass it a parameter named response. This response string is generated by reCAPTCHA and looks like a random collection of characters.
/checkRecaptcha
response
The path /checkRecaptcha in your web site obviously doesn't exist yet, so we need to create a Cloud Function to validate the response string it's going to receive.
In the functions directory in your project, edit the existing index.js file. This has some sample code, but you can delete it. In its place, paste the following JavaScript code:
const functions = require('firebase-functions') const rp = require('request-promise') exports.checkRecaptcha = functions.https.onRequest((req, res) => { const response = req.query.response console.log("recaptcha response", response) rp({ uri: 'https://recaptcha.google.com/recaptcha/api/siteverify', method: 'POST', formData: { secret: 'PASTE_YOUR_SECRET_CODE_HERE', response: response }, json: true }).then(result => { console.log("recaptcha result", result) if (result.success) { res.send("You're good to go, human.") } else { res.send("Recaptcha verification failed. Are you a robot?") } }).catch(reason => { console.log("Recaptcha request failure", reason) res.send("Recaptcha request failed.") }) })
The first thing you should do here is paste your reCAPTCHA secret key from the registration site in place of "PASTE_YOUR_SECRET_CODE_HERE".
(Astute readers may note that the reCAPTCHA API endpoint host is "recaptcha.google.com", while the docs say "www.google.com". This is OK! You have to use recaptcha.google.com as shown in order to make the call on the Spark plan, because that host has been whitelisted for outgoing traffic from Cloud Functions.)
This code defines an HTTPS function that, when triggered, will make another HTTPS request (using the request-promise module) to the reCAPTCHA API in order to verify the response that was received in the query string. Notice that there are three cases with three different responses to the client. Either:
It's important to send a response to the client in all cases, otherwise the function will time out with an error message in the Firebase console log.
To deploy this new function (and the web content at the same time) run the following command:
$ firebase deploy
You'll notice in the output that the function is assigned its own URL, which looks something like this:
https://us-central1-your-project.cloudfunctions.net/checkRecaptcha
This is clearly a different host than the one with your web content. However, what we really want instead is for the function to be referenced through your web host at a URL that looks like this:
https://your-project.firebaseapp.com/checkRecaptcha
This makes the function look like it's part of your web site. With Firebase Hosting a Cloud Functions, this can be done!
Edit the file firebase.json in the project root directory and paste the follow JSON configuration as its contents:
firebase.json
{ "hosting": { "public": "public", "rewrites": [ { "source": "/checkRecaptcha", "function": "checkRecaptcha" } ] } }
What you've done here is add a new section for rewrites, and you can read more about those in the docs. Specifically what this does is allow access to the URL path /checkRecaptcha invoke the function called checkRecaptcha that you pasted into your functions/index.js file.
checkRecaptcha
functions/index.js
Remember that the JavaScript code in recaptcha.html redirects to this path when the reCAPTCHA is satisfied by the user, so this effectively sends to user to the function after they complete the reCAPTCHA.
Now do one final deploy to send everything to Firebase:
Navigate to /recaptcha.html under your hosting URL, then solve the reCAPTCHA. It may ask you to identify some cars or roads in a set of pictures. Once you've satisfied the reCAPTCHA with your humanity, the JavaScript in your HTML should redirect you to your function, which verifies with the server that you're indeed human, and you should see the message "You're good to go, human."
/recaptcha.html
This example of how to use reCAPTCHA with Cloud Functions for Firebase is much more simple than what you'd probably do in your own web site. You have several options for how to send the reCAPTCHA response to your function, and you'd obviously want to provide something more useful than a message to the user. But this should get you started protecting your web content from abuse from bots.
Ever spend an hour wondering why Remote Config wasn't working, only to realize that you forgot to call activateFetched? Or didn't read a Dynamic Link because you forgot to implement the application:continue:restorationHandler method? Well, now there's a tool to help stop those mistakes before they happen!
activateFetched
application:continue:restorationHandler
SwiftLint is a great open source tool that makes it easier for you to follow Swift style and conventions. It also helps with identifying possible errors early by highlighting problematic usage. You can run SwiftLint on your Xcode project to see all the style guide exceptions on the lines where they occur, and fix them quickly. I found it was a great help when I migrated my code from Objective-c to Swift.
In the spirit of making SwiftLint even more useful for Firebase developers, we've added some experimental new Firebase rules into SwiftLint. These rules will display warnings on common mistakes that might lead to errors when using the Firebase SDK.
Currently we are hosting the rules on our fork in a firebase_rules branch. Our pre-release binary holds the Firebase rules. You simply download the .pkg file and double click to install. You can also build the binary from the source.
Since the rules are opt-in, you'll need to add a .swiftlint.yml file in the same folder as your Swift source files, containing the following text:
``` opt_in_rules: - firebase_config_activate - firebase_config_defaults - firebase_config_fetch - firebase_core - firebase_dynamiclinks_customschemeURL - firebase_dynamiclinks_schemeURL - firebase_dynamiclinks_universallink - firebase_invites ```
Then, just run SwiftLint on your project like normal.
If you're interested in how we put the rules together, you can read our post with all the development details.
We'd love for you to give it a try and send us feedback on Twitter with #FirebaseLinter. You can also ask questions on StackOverflow using the firebase and swiftlint tags together.
firebase
swiftlint
