Ever since we launched our Firebase bindings for AngularJS last March, we’ve been amazed by the Firebase and Angular apps we’ve seen from our community. This year, we’re incredibly excited to be a Platinum sponsor at the first ever Angular conference, starting this Thursday in Salt Lake City, Utah! We’re also happy to announce our new Angular + Firebase Quickstart Guide, which makes it even easier to get started with Angular and Firebase.
If you won’t be at the conference, head over the ng-conf home page for a link to a livestream of all the talks. You can also follow @Firebase and #ngconf on Twitter for the latest updates. The ng-conf organizers have pulled together a great lineup of events. Here’s where you can find us at the conference:
Hear the latest on Firebase + Angular, including a special announcement: On Friday at 10:35am MST (GMT -7), I will be speaking about the powerful combination of Firebase and Angular, and how the two tools are changing the future of modern web apps. I’ll also be making an announcement about Firebase.
Ask questions at Firebase office hours: For those of you attending ng-conf, get your technical Firebase + Angular questions answered in person and share your apps with the Firebase team! We’re holding office hours on Friday 1/17 from 7:30 - 9am. Sign up in advance or drop in at the conference.
Firebase Sponsoring ng-party: We are the sponsor of the official ng-conf party on Thursday night (1/16). The party will feature performances from a comedian, an illusionist, an improv group, and there will be some delicious desserts.
Firebase + Angular Quickstart Guide: Our new quickstart guide provides all the resources you need to start building realtime apps with Firebase and Angular.
Firebase + Angular Starter Pack: To see all of our example apps built with Firebase and Angular, simply clone our Angular Starter Pack on GitHub. The examples are a great place to start playing with code and learning about the integration.
The Complete Angular Guide: If you’re new to Angular, we recommend ng-book, a comprehensive guide to AngularJS with a chapter on adding a Firebase backend to your app.
Firebase + Angular Google Group: Join our Firebase + Angular Google Group to ask technical questions or share your apps with the community.
James, Andrew, Sara and I are looking forward to seeing many of you at ng-conf!
Update (November 4, 2014): While this post still contains some useful and relevant information,we have released advanced query functionality which solves a lot of the problems this post discusses.You can read more about it in our queries blog post.
In Part 1 of this post, we covered a series of common SQL queries and how they can be recreated in Firebase, building off our authoritative Denormalizing is Normal post from last year. We're going to build on several principles introduced in those articles, so be sure to check those out before digging into this post.
In this tutorial, we'll explore a fast and powerful approach to performing text searches, or content searches, in Firebase.
Why Not Just WHERE foo LIKE '%bar%'?
The 20 year old SQL paradigm for content queries (WHERE foo LIKE '%bar%') is a staple for static databases, but not a simple prospect for real-time data. Our team is hard at work on a series of tools to bring content searches into the lightning-fast realm of Firebase's NoSQL data store. Look for more news on these indexing and query-related tools in the coming months.
Until then, I'd like to introduce you to a few quick scripts that can add powerful content searches to your app. At the end of the article, I'll share a library that incorporates these strategies into a service you can clone, configure, and run on your own box.
Introducing ElasticSearch
ElasticSearch, based on Lucene,is an extremely powerful document storage and indexing tool. However, at its core is a very simple search feature, which is nearly plug-and-play compatible with Firebase.
While it's certainly not the only way to write content searches in Firebase, ElasticSearch's simple integration makes it fast to implement and takes very little code to utilize, while its powerful indexing capabilities provide for customization, allowing it to scale with your app.
You can set up a local instance for testing in three steps:
It's really that simple! And, surprisingly, deploying a free, hosted instance requires nothing more than a button click thanks to Heroku's Bonsai add-on.
Indexing Firebase Data
The first step is to get data into ElasticSearch so it can be indexed. A simple Node.js script can plug Firebase into ElasticSearch with a few lines of work. I utilized the node-elasticsearch-client library, which is optional, but simplifies the process by wrapping the lower level ElasticSearch client:
var Firebase = require('firebase'); var ElasticClient = require('elasticsearchclient') // initialize our ElasticSearch API var client = new ElasticClient({ host: 'localhost', port: 9200 }); // listen for changes to Firebase data var fb = new Firebase('<INSTANCE>.firebaseio.com/widgets'); fb.on('child_added', createOrUpdateIndex); fb.on('child_changed', createOrUpdateIndex); fb.on('child_removed', removeIndex); function createOrUpdateIndex(snap) { client.index(this.index, this.type, snap.val(), snap.key()) .on('data', function(data) { console.log('indexed ', snap.key()); }) .on('error', function(err) { /* handle errors */ }); } function removeIndex(snap) { client.deleteDocument(this.index, this.type, snap.key(), function(error, data) { if( error ) console.error('failed to delete', snap.key(), error); else console.log('deleted', snap.key()); }); }
Drop that in a hosting environment like Heroku or Nodejitsu, or onto your own host with forever to monitor up-time, and search indexing is done! Now it's time to read some of that data back.
A Brute Force Search
Once we have our data indexed in ElasticSearch, we could directly query our index using a wrapper like elastic.js. This is a perfectly reasonable option, but does add coupling and dependencies to the client:
<script src="elastic.min.js"></script> <script src="elastic-jquery-client.min.js"></script> <script> ejs.client = ejs.jQueryClient('http://localhost:9200'); client.search({ index: 'firebase', type: 'widget', body: ejs.Request().query(ejs.MatchQuery('title', 'foo')) }, function (error, response) { // handle response }); </script>
Since our clients are already using Firebase, wouldn't it be great to keep our client code agnostic and push the request to Firebase instead?
A Firebase Search Queue
This little node script listens at /search/request for incoming searches, handles the interactions with ElasticSearch, and pushes results back into /search/response:
/search/request
/search/response
var Firebase = require('firebase'); var ElasticClient = require('elasticsearchclient') // initialize our ElasticSearch API var client = new ElasticClient({ host: 'localhost', port: 9200 }); // listen for requests at https://<INSTANCE>.firebaseio.com/search/request var queue = new Firebase('https://<INSTANCE>.firebaseio.com/search'); queue.child('request').on('child_added', processRequest); function processRequest(snap) { snap.ref().remove(); // clear the request after we receive it var data = snap.val(); // Query ElasticSearch client.search(dat.index, dat.type, { "query": { 'query_string': { query: dat.query } }) .on('data', function(data) { // Post the results to https://<INSTANCE>.firebaseio.com/search/response queue.child('response/'+snap.key()).set(results); }) .on('error', function(error){ /* process errors */ }); .exec(); }
A Client Example
Now that we have a way to queue requests into Firebase, the client can simply push requests and listen for results:
<script> var queue = new Firebase('https://<INSTANCE>.firebaseio.com/search'); function search(index, type, searchTerm, callback) { // post search requests to https://<INSTANCE>.firebaseio.com/search/request var reqRef = queue.child('request').push({ index: index, type: type, query: searchTerm }); // read the replies from https://<INSTANCE>.firebaseio.com/search/response queue.child('response/'+reqRef.key()).on('value', function fn(snap) { if( snap.val() !== null ) { // wait for data snap.ref().off('value', fn); // stop listening snap.ref().remove(); // clear the queue callback(snap.val()); } }); } // invoke a search for *foo* search('firebase', 'widget', '*foo*', function(data) { console.log('got back '+data.total+' hits'); if( data.hits ) { data.hits.forEach(function(hit) { console.log(hit); }); } }); </script>
A Pre-Built Library for Your Use
We've implemented a content search for Firebase using ElasticSearch, set up a queue to ferry data transparently to and from our search engine, and finished all of this in a couple short scripts. We've also tapped into the powerful, flexible, and scalable world of ElasticSearch, which will grow with our app.
Easy enough? I've gone ahead and baked these scripts into a GitHub repo just to make things even simpler: Fork the Flashlight service on GitHub.
It's MIT Licensed and ready for cloning. Just edit the config file, start it up, and get back to work on your app!
We want your feedback!
Have fun and let us know how we did! We love getting feedback from our dev community. Chat with us in the comment thread or send an email to wulf@firebase.com. You can also follow @Firebase on twitter.
2013 has been an incredible year. We opened our beta to the public, launched iOS and Android SDKs, raised our first round of venture investment, doubled our team and took the beta label off Firebase -- and these are just a few!
As 2014 approaches, we wanted to take a moment to thank our amazing developers (that’s you!) for making Firebase what it is today. Your awesome apps and continued support make us excited to keep pushing Firebase forward every day. We hope you have a wonderful 2014 and we look forward to seeing what you build in the new year!
Here are the top highlights from the last year:
We’ve got some incredible things coming in early 2014, to get all the latest news you can follow us @Firebase on Twitter and join the Firebase conversation in our Google Group.
As always, we’re here to help however we can, so please don’t be shy!
Firebase is great for apps where data changes frequently, especially collaborative applications where users across the world are interacting with each other in realtime. We’ve seen some awesome collaborative drawing apps built with Firebase, and in this post we’d like to highlight Talkboard, an iPad app built by the team at Citrix. Talkboard is a beautifully designed app for whiteboarding and brainstorming ideas remotely. Below, Frederic Mayot from Citrix answers a few questions about building Talkboard and integrating Firebase.
How does Talkboard use Firebase?
We use Firebase to handle all our data, from user profiles to whiteboard data. The API was so simple that we implemented all the communications and data synchronization in a bit more than a week. Thanks to disk persistence, we allow our users to access and edit their projects and whiteboards anywhere, anytime, without worrying about being connected to the internet. Though very simple in its UI, Talkboard is a complex iOS application. However, we managed to model everything we needed with Firebase, including making the app secured and very responsive. The application is sending complex data structures representing the strokes at very high frame rates. We were very impressed to see such a low latency. This really enables the magic of showing the strokes being updated in real-time across devices.
What are some ways you've used Talkboard to collaborate at Citrix?
As soon as we got an initial version working, we used Talkboard to design Talkboard ;-) It is such an amazing experience to be able to draw in real-time with someone and be able to talk about it! The app makes it very easy to produce beautiful artifacts, even if you’re not a designer.
What is the coolest use case of Talkboard you've seen so far?
We’ve seen many different use cases and are still learning from our users. Talkboard is picking up in the education space but also with designers and all kinds of business users. To me, the most amazing thing is to see how people have fun when using the app for the first time.
Download Talkboard for your iPad now to start collaborating with co-workers and friends in realtime!
Peter, AJ and Arun are the co-founders of Nitrous.IO, a platform that enables developers to do their best work by easily creating and managing development environments in the cloud. The collaborative text editing features of Nitrous.IO are powered by Firepad, Firebase's open-source collaborative code and text editor.
What inspired you to build Nitrous.IO?
Configuring a development environment is a painful experience. Most people own multiple devices and installing and maintaining multiple development environments is a waste of time. Businesses often allocate a few days for every new hire for workstation setup and configuration and keeping an entire engineering team’s development environments in sync is a daunting task.
The Nitrous.IO Co-founders Peter, AJ and Arun had to work on multiple projects simultaneously in their past lives where each project had its own programming languages, databases, tools and libraries it depended on. Problems often arose when different versions or configurations of software were required to develop an application. Managing multiple development environments on a single machine is often impractical due to the fact that many software packages only allow a single version and configuration to be installed at a time, and syncing development environments across multiple computers is also a tedious and protracted process. We wasted countless hours of time and energy troubleshooting development environment issues, easily worth thousands of dollars of our time. We realized that if we could move environments to a central location (the cloud), we could create them, replicate them and share them much more easily than before.
There’s also the benefit of workspace accessibility. Peter always wanted to develop on his iPad (the crazy guy). We think the iPad is still in its infancy when it comes to fulfilling its potential as a powerful development machine, but we felt that with a cloud-based development platform, the iPad could become a powerful weapon in the developer’s arsenal. This prophecy has already come true - we have developers using Nitrous.IO on their iPads and other tablets, including engineers from well-known companies like Heroku.
In addition to feeling this need ourselves, we found that engineers at a number of companies, like Facebook and Quora are already developing on remote development servers, which are set up by teams solely dedicated to managing them. For most companies, however, it doesn’t make sense to dedicate a team to managing development environments. We think the future is in services like Nitrous.IO that make development environments much easier to manage across teams.
How does Nitrous.IO use Firepad?
Firepad enables Nitrous.IO users to collaboratively edit code in the browser in real time. This means that Pair Programming is as easy giving someone an URL that they can visit in any modern browser. It also means that cross functional teams can collaborate easily and iterate on their product faster. A developer can build something, and then discuss it with their Lead Designer in the Nitrous.IO IDE collaboratively. With the Nitrous.IO preview feature, they can have the application available on a public URL for the Design Lead to share with private beta users as soon as the developer has completed his work without waiting for a build and deploy cycle. A screenshot of our collaboration feature can be seen below.
We use Firepad to handle the Operational Transformation (basically handling the merges that occur on the document as multiple people edit the same text). Firepad also helps us to manage and display the cursor position of all users, undoing text, highlighting text and showing a username above the cursor, and when other users connect or disconnect to the document. Firepad saved us weeks of work, and helped us get the prototype of Nitrous.IO collaboration out in just a matter of days! A lot of people might spend months of their time developing something like this, but we were able to do it very quickly using Firepad.
What are some of the coolest uses of Nitrous.IO you've seen so far?
We are always excited and surprised by the many ways our customers use Nitrous.IO. Of course, we love it when the latest new framework or cool side project by our favorite celebrity developers are built on Nitrous but what really get us jazzed is when we get to introduce the world of web development to newcomers. We’re very pleased to be a key part of the teaching people who are new to web development. We have been a key part the Journeyman Project, Rails Bridge who teach women how to code in Rails, and Rails Girls Programs.
Are there any exciting features on your roadmap you can share?
We’re working on a few really powerful tools around sharing environments that we think are going to help developers become considerably more efficient with their workflow. We’ll be continuing to work on our collaboration features and deployment integration as well, so stay tuned to @nitrousio on Twitter to hear when we launch new features!
Do you have any advice for other developers building collaborative applications?
Before implementing certain features by yourself, you should definitely study the underlying data structure of Firepad. It will help you considerably when you want to do certain customizations like implementing cursor colors, highlighting text, user presence etc...
You should also consider using Firebase if you are extending your implementation of Firepad. We used Firebase to power the chat and file notifications within the feed window pane on our editor and it saved us a ton of time!
