Arrays haven't always been the best data structure for multi-user environments like Cloud Firestore. As Kato describes in the "Arrays are evil" section of his blog post, bad things can happen if you have multiple clients all trying to update or delete array elements at specific indexes. In the past, Cloud Firestore addressed these issues by limiting what you can do with arrays. That means that until now, you could really only "update" arrays by replacing the entire array (no appending or deleting!), and you couldn't perform meaningful queries on arrays, either.
This was problematic for those of you who wanted to use arrays in simple cases like keeping a list of tags or keywords. Previously, we've recommended you try a workaround (some might say, a "hack") of converting your arrays into maps like this:
Well, with our latest improvements to arrays, none of this is necessary! For starters, we've added the ability to query for elements within arrays using the new "array-contains" feature. This means you can keep your elements as an array, and easily query for them without having to resort to the map hack.
array-contains
Even better, you can query for items in arrays that aren't strings, which was a problem with the previous "convert your array into a map" workaround.
You also have the ability to add or remove elements from an array. But in order to avoid some of the issues that can arise in a multi-user environment, you'll be adding them with more of a set-like functionality. So rather than asking to delete an item at index 3, you would ask to remove, for example, all elements of the string "sly" with the arrayRemove operator.
sly
arrayRemove
With the arrayUnion operator, you can append an element to an array, but only if it doesn't exist in the array already.
arrayUnion
Adding "clever" to our array doesn't do anything, because it already exists.
But adding "fuzzy" adds a new element to the array!
These changes also come with some improvements to security rules as well. Now you can create security rules that allow queries based on whether or not a certain element exists inside of an array. So doing things like querying for a list of documents, but only allowing that query if the user is listed inside of the documents' "viewers" array is significantly easier than before.
viewers
This kind of query request is now possible in Cloud Firestore
All of these features should be available with the latest client SDKs, so make sure you update to the latest versions of your libraries, and start having fun with arrays! As always, if you have questions, you can join the Cloud Firestore Google discussion group, or use the google-cloud-firestore tag on Stack Overflow.
Well, I think it's safe to say there's been more news coming out of Cloud Next 2018 than you can shake a stick at. And while your manager is probably asking you to stop shaking sticks at news stories and get back to work, you might still be wondering, "How does all of this affect me, your typical app developer using Cloud Firestore?"
Good question! It turns out we've made some really nice improvements in Cloud Firestore that are either out already, or coming your way soon. Let's go over them together, shall we?
One new feature we've announced is the ability to disable the automatic indexing of a field in your documents. Why would you ever want to do this? Primarily, the issue is that Cloud Firestore will index the value of any field in a document, but if those fields happen to contain maps or arrays, it will then go ahead and recursively index every single value within those objects, too.
In many cases -- like storing a mailing address as a map -- this is exactly what you want, so you can search for documents by city or zip code. But if you're storing a bunch of, say, raw drawing data into a massive array, you definitely don't need those fields indexed. That's going to cost you unnecessary time and storage costs, and runs the risk of bumping up against your "20k indexed fields per document" limit.
So in these situations, we've given you the ability to disable the automatic indexing of those fields. This will stop Cloud Firestore from indexing the value of that field, along with recursive indexing any arrays or maps it might find inside that field. So you can continue to index your "address" field above, while leaving your "big_drawing" field unindexed.
address
big_drawing
Single field index controls also help with that whole "You can't do more than 500 writes per second in a collection where documents have constantly increasing or decreasing values" limit, which is something you could run into if you were, for example, trying to store timestamps in a very large collection. By disabling indexing on that timestamp field, you won't have to worry about this limit, although it does mean you will no longer be able to query that collection by that timestamp.
Many customers have been asking us for the ability to keep their Cloud Firestore data in specific locations around the world. This can be a significant performance boost if you think the majority of your customers are going to be located in a certain part of the world, and it may help you comply with local regulations around how your data is stored.
So we're pleased to announce that we're starting the process of adding new locations to host your Cloud Firestore data. We're going to start with Frankfurt, Germany and South Carolina (in the U.S.), but you can expect to see more locations being added over the next several months.
You should see the "Cloud Firestore location" option when you create a new project in the Firebase console -- this will determine where your Cloud Firestore data is located. Please note that once you've selected a location, it can't be changed later, so choose carefully!
We've also added the ability for you to import and export your Cloud Firestore data. This is useful if you ever want to make backups of your data, it gives you the freedom to migrate your data to another database if you ever wanted to, and it makes it easy for you to copy data between projects. That last feature can come in really handy if you want to migrate data from your production project into your "test", "dev" or "staging" project.
Exports from Cloud Firestore will be stored into your Google Cloud Storage bucket, and from there you can use any number of tools to move your data into other systems. For more details, make sure to check out our documentation.
We're also happy to report that we'll be speeding up the time in which security rules are deployed and made active in your project. Thanks to some serious improvements our engineers have made under the hood, the time it takes for security rules to become active in your project will no longer be measured in minutes. This should make testing security rules in your app a much better experience than before. Look for these security changes to roll out sometime this month.
As we've made improvements on the backend, we've been able to increase some of the limits that were placed on Cloud Firestore while the product is in beta. You can now perform up to 10,000 writes per second per database (up from 2500) and Cloud Firestore supports up to 1 million concurrent connections (up from 100,000) .
Cloud Firestore now runs in two modes -- Native mode and Datastore mode. Native mode is probably what you've been using all this time as a Firebase developer, and it's now available to Google Cloud Customers who want to add Cloud Firestore to their GCP projects.
Datastore mode is a 100% backwards-compatible version for developers who have been using Cloud Datastore up until now. It doesn't contain all of the features of Cloud Firestore (like real-time updates or offline support), but it does add some nice improvements like strong consistency and removes some limits around writes and transactions.
All current Cloud Datastore customers will be seamlessly upgraded to Cloud Firestore in Datastore mode when Cloud Firestore reaches General Availability. For more information, be sure to see this post on the Google Cloud Platform blog.
So there ya go, folks. Lots of fun new features to play with in Cloud Firestore. Give 'em a try and let us know what you think! As always, if you have questions, you can join the Cloud Firestore Google group, or use the google-cloud-firestore tag on Stack Overflow.
Firebase has an impressive feature list for getting your apps up and running quickly. I love that I can go from idea to working app in a matter of hours! But for developers who haven't had much experience with asynchronous programming, it's easy to get bogged down in the details. Heck, I've been working with Firebase for two years and I still find myself pausing to think about how I want to handle asynchronous calls. What will my view look like while waiting to populate data? How should I set up my data class so that it can alert the view controller about updates? After lots of practice and exploring options, I feel like I finally have an answer that works for me.
Now I'm not going to get into the what and why of asynchronous programming with Firebase because Doug Stevenson already addressed this in this great blog post. Instead, I want to get into how to get a function that uses an async callback working the way you want it to in Swift.
Take this example function I have below from this photo app I've created. It queries Cloud Firestore for 10 documents in a collection called "cats". Once the Cat data is downloaded, I want to use it to populate a UITableView with information about the cat photos.
// ViewController.swift var dbRef = Firestore.firestore().collection("cats") var cats = [Cat]() func downloadCats() { // order the posts by timestamp let query = dbRef.order(by: "timestamp", descending: true).limit(to: 10) query.getDocuments { snapshot, error in print(error ?? "No error.") // iterate through the documents and create Cat objects guard let snapshot = snapshot else { return } for doc in snapshot.documents { let cat = Cat(snapshot: doc) self.cats.append(cat) } } print(cats) // 1 what will this print? tableView.reloadData() }
What will be printed at "1"? Since the call to getDocuments is asynchronous, the closure will be run when the download of data is completed sometime in the future. Commands that I write following the closure will be run immediately -- they do not wait for the work in the closure to be complete. So chances are, in the scenario above, that print statement is going to print an empty array. The UITableView will reload data immediately, so any Cat objects that were supposed to populate the view will not appear. That's not the behavior we want. So what can we do?
getDocuments
print
.
UITableView
Cat
One simple option is to move the functionality I want into the closure, like so:
// ViewController.swift func downloadCats() { let query = dbRef.order(by: "timestamp", descending: true).limit(to: 10) query.getDocuments { snapshot, error in print(error ?? "No error.") for doc in snapshot!.documents { let cat = Cat(snapshot: doc) self.cats.append(cat) } print(cats) // 1 what will this print now? tableView.reloadData() } }
What is printed at "1" now? We should now see the full array of Cat objects, and when tableView.reloadData() is called, the UITableView can populate with the data of those Cats, assuming you've set up your UITableViewDataSource for it. This is a viable method for handling data in the closure, but it means you'll have to include this functionality in your UIViewController. As you add more functionality, such as different kinds of queries, Auth, Cloud Storage, and callable Cloud Functions, you'll quickly have a Massive View Controller on your hands.
tableView.reloadData()
UITableViewDataSource
UIViewController
I like to keep this functionality separate from my view controller whenever possible. I can do this by wrapping getDocuments() in a function with a completion handler. If I include a completion handler in this function, I can call completion() when the array of Cats is populated.
getDocuments()
completion()
// CatManager.swift var dbRef = Firestore.firestore().collection("cats") var cats = [Cat]() // function includes an completion handler, marked with @escaping func downloadCats(completion: @escaping () -> Void) { let query = dbRef.order(by: "timestamp", descending: true).limit(to: 10) query.getDocuments { snapshot, error in print(error ?? "No error.") self.cats = [] guard let snapshot = snapshot else { completion() return } for doc in snapshot.documents { let cat = Cat(snapshot: doc) self.cats.append(cat) } completion() } } }
Then inside the view controller, I can call my function like this:
// ViewController.Swift func initializeCats() { activityIndicator.startAnimating() CatManager.sharedInstance.downloadCats() { // Data for UITableView is populated from the CatManager singleton self.tableView.reloadData() self.activityIndicator.stopAnimating() } }
Now I'm controlling the view from the view controller, reloading the tableView and indicating when the activityIndicator should run. The data is being handled in the CatManager class. This does, however, require a singleton, so while it may prevent a massive view controller from developing, singletons come with their own host of issues.
tableView
activityIndicator
CatManager
Instead of passing void in the completion, I can pass some useful values: an array of Cats and an optional error.
error
// CatDownloader.swift // Use instead of CatManager func downloadCats(completion: @escaping ([Cat], Error) -> Void) { var catArray = [Cat]() let query = dbRef.order(by: "timestamp", descending: true).limit(to: 10) query.getDocuments { snapshot, error in if let error = error { print(error) completion(catArray, error) return } for doc in snapshot!.documents { let cat = Cat(snapshot: doc) catArray.append(cat) } completion(catArray, nil) } } }
With this function, since I pass the error in the closure, the view controller can be made aware of an error and display it to the user. Here I made a function called alert() that displays a UIAlertController. I can also create an instance of CatDownloader in the class because I don't need to depend on a single CatManger to keep track of the Cat array.
alert()
CatDownloader
CatManger
// ViewController.swift // create an instance of CatDownloader let catDownloader = CatDownloader() //… func initializeCats() { activityIndicator.startAnimating() catDownloader.downloadCats() { catArray, error in self.activityIndicator.stopAnimating() if let error = error { self.alert(title: "Error", message: error.localizedDescription) return } self.cats = catArray self.tableView.reloadData() } } func alert(title: String, message: String) { let alertController = UIAlertController(title: title, message: message, preferredStyle: .alert) let action = UIAlertAction(title: "OK", style: .default, handler: nil) alertController.addAction(action) navigationController?.present(alertController, animated: true, completion: nil) }
You may want to display a custom error message to the user instead of the error description, but you can see how the closure made it easy to get the info I needed to the right place in my app. You can use this same configuration for other Firebase features that incorporate closures, including the Realtime Database, Authentication, and ML Kit.
And there you have it! Now go forth and write your own functions with closures to help you update your views. What other examples would you like to see? Do you have any cool solutions you've found that you'd like to share? Reach out to me on Twitter at @ThatJenPerson to tell me all about it!
In this edition: Cloud Firestore, Flutter, search with Algolia, Phone Auth, and the brand new ML Kit!
Hey there, Firebase developers! The weather is really starting to warm up here in California as we approach summer, and if you're like me, it's the perfect time of year to go outside, find a nice shady tree, and cozy up with a programming project.
Ok, maybe some of you prefer going to the beach when it's nice out, but I can't seem to figure out how to keep the sand out of my computer! Anyway, when I'm feeling inspired to start a new project, I love having a tutorial to help out. Luckily, the Firebase community is creating new material all the time to keep me busy! If you're looking to find out more about a Firebase product, there's a good chance someone has tried it out and shared their experiences in a blog or video.
Since content can be scattered all over, I figured I'd do the work of scouring the Internet to bring you some of the best Firebase tutorials released over the past couple months. While I'd love to highlight every article and video about Firebase, there simply isn't time or space for them all, so I'm going to showcase a few that I'm most excited about.
Author: The Net Ninja
In this series, Shaun walks you step by step through incorporating Cloud Firestore into a simple web app. He starts with an intro to Cloud Firestore, and continues on through reading data, creating queries, writing data, and real-time updates. The illustrations really helped me visualize the content. The videos also include links to the code on GitHub.
Author: Tensor Programming
As the title suggests, this tutorial shows developers how to get users logged into your Flutter app with Firebase Authentication. This video picks up in the middle of a series, so you may want to go back to the beginning to learn about incorporating Cloud Firestore with Flutter. I've never worked with Dart before, but the instructor does such a great job of explaining each step, it was still very easy to follow along!
Author: Angular Firebase
The Angular Firebase channel creates a myriad of rich tutorials that use different features of Firebase. I'm highlighting this one in particular because so many developers want to know how to implement full text search in Firestore. I also like the use of Cloud Functions, especially the demonstration on how to hide secret API keys in environment variables. Even if you don't use Angular, you'll still learn a lot from this video. If you do use Angular, even better!
Don't want to choose between written and video tutorials?! Check out this Firebase Phone Authentication tutorial from Belal Khan, that also includes a video of the content. Khan's writing style includes larger blocks of code with explanations in between rather than adding and explaining line-by-line. This is great for those of you who like to learn by examining the code as a whole. I know when I'm learning a new feature, I like to look at a complete sample to see how the pieces fit together, so this was really cool to me.
Author: Anže Kravanja
I love this tutorial because, quite frankly, I know little about Google Analytics and nothing about Big Query. This is a great resource for developers who want to get more out of their data stored in Google Analytics for Firebase. The tutorial explains how sessions are tracked in Google Analytics for Firebase and demonstrates step by step how to group Firebase events into user sessions.
Adam Talcott looks at a couple of fun and useful cases for ML Kit for Firebase. As a fellow Swift developer, I love to see how other iOS folks are using Firebase.
I hope that these resources inspire you to build something great! I know after checking them out, I'm ready to slather on the sunblock and enjoy some UV rays while I code my next Firebase project!
Thanks to all of the developers who shared their knowledge with us through their tutorials! Firebase wouldn't be what it is without you. I'm going to feature tutorials here every couple of months, so when you find a tutorial you love, be sure to tell me about it! Find me on Twitter at @ThatJenPerson.
WWDC has drawn to a close and we wanted to share the photos of our party on the night of June 4th. It was great to meet and celebrate app development with many of you, including talented developers from Slack, Adidas, Coinbase, DJI and many more. We were blown away by your enthusiasm to come party with us!
Since we can't travel back in time, here are some highlights so you can relive the night of fun.
We were excited to roll out the red carpet for our guests at the entrance. Our friendly hosts greeted everyone at the door, and we had food+drinks ready for you as soon as you finished checking in.
To give the GlassHouse a homey feel, we designed the ambience of our event to resemble the look and feel of Firebase. Throughout the space, we added lighting in Firebase's colors, custom cushions, video backdrop and more.
A variety of passed hors d'oeuvre (with vegan options) were served throughout the night. We also designed three developer cocktails for the night: the Moscow ML, Old Functioned and Magarealtime. Thank you to all those who submitted creative names for our cocktails and congrats to the winners!
The glowing selfie-station with nifty LED sunglasses was a favorite at this year's party. We're glad that many of you loved the setup, took pictures, and shared them on social media!
We also had fun playing our AppShip arcade game with many of you. Some of our hosts lost their voice cheering everyone on for that one, final goal of the night. And if you scored an own-goal, you weren't the only one: over half of the balls that went into the goal were last touched by someone from their own team.
A party wouldn't be complete without some great music and dancing. This year, we had a bigger dance floor with a giant disco ball as well as a talented DJ as usual. Those who didn't feel like dancing and just wanted to relax, could do so at the outdoor terrace where you could get some fresh air and continue mingling.
No party can last forever, and goodbyes are always hard. This year, Firebase branded travel mugs and bottle openers were waiting to go home with guests right at the door.
Thank you to all those who joined us to celebrate app development - we were glad to share all the laughter with you. Check out our full photo album. To stay up-to-date on our next events, follow us on Twitter.
Here's to next year!
From the Twitterverse:
#firebaseparty was lit last night 🔥 The best part, of course, was catching up with friends I haven't seen in a long time! #WWDC18 pic.twitter.com/383TOsI8WG— NatashaTheRobot (@NatashaTheRobot) June 5, 2018
#firebaseparty was lit last night 🔥 The best part, of course, was catching up with friends I haven't seen in a long time! #WWDC18 pic.twitter.com/383TOsI8WG
Thanks #firebaseparty! Had so much fun meeting friends old and new. Thank you to everyone who said hi! 💕💕 #WWDC18 pic.twitter.com/GAwQzThRHL— mayuko (@hellomayuko) June 5, 2018
Thanks #firebaseparty! Had so much fun meeting friends old and new. Thank you to everyone who said hi! 💕💕 #WWDC18 pic.twitter.com/GAwQzThRHL
Meeting awesome people at #firebaseparty 😎🤟🍢🍤🍺🍻@Firebase #wwdc #wwdc18 #wwdc2018 https://t.co/9tyQ7hF8wz pic.twitter.com/wtcOBWXHL1— Roberto Garrido (@che1404) June 5, 2018
Meeting awesome people at #firebaseparty 😎🤟🍢🍤🍺🍻@Firebase #wwdc #wwdc18 #wwdc2018 https://t.co/9tyQ7hF8wz pic.twitter.com/wtcOBWXHL1
Thanks #firebaseparty for such a great time! #WWDC18 pic.twitter.com/axZfMUpQjl— Kelly Hutchison (@kelhutch17) June 5, 2018
Thanks #firebaseparty for such a great time! #WWDC18 pic.twitter.com/axZfMUpQjl
#WWDC18 #firebaseparty #layersconf pic.twitter.com/7GquJLiE79— Jess Moss (@jessmossisboss) June 5, 2018
#WWDC18 #firebaseparty #layersconf pic.twitter.com/7GquJLiE79
