James, Andrew, Sara and I are back from ng-conf -- and we all had a blast! The conference was very well put together (kudos to Domo) and it was inspiring to speak with luminaries from across the Angular community in person. Just in time for ng-conf, we released a new version of AngularFire (0.6), deployed improvements to our account dashboard, and announced a new Firebase + Angular quickstart guide. I was also excited to demo on stage how easily you can wire up a backend to your Angular app using Firebase.
Watch this video of my talk on building realtime apps with Firebase and Angular:
The conference started off with an amazing keynote by the creator of Angular himself, Miško Hevery, along with Brad Green from Google. They gave a bit of history behind Angular, how it came to be, where they are today and what their plans are for the future, including how they want to help the community extend the Angular feature set. We were extremely flattered when Miško said in the keynote that Firebase “completes the original vision” of Angular:
After a long day of excellent talks by several preeminent speakers, it was time for the Firebase sponsored party. The party featured a comedian, magician and an impromptu comedy troupe. A spread of desserts was accompanied with our very own Firebase chocolate. We hope everyone had a great time - conferences are so much better when they’re mixed in with a little live entertainment!
We gave away tons of Firebase swag: T-shirts, light-up bracelets, stickers, and of course, our very own hot sauce! We were very happy with all the interest in Firebase, and were delighted to meet everyone who came to our booth with questions, compliments or feedback.
We wrapped up the conference with a day on the slopes with many other ng-conf attendees. It’s one of the best conferences we’ve participated in so far, and we’re definitely looking forward to the next one!
Ever since we launched our Firebase bindings for AngularJS last March, we’ve been amazed by the Firebase and Angular apps we’ve seen from our community. This year, we’re incredibly excited to be a Platinum sponsor at the first ever Angular conference, starting this Thursday in Salt Lake City, Utah! We’re also happy to announce our new Angular + Firebase Quickstart Guide, which makes it even easier to get started with Angular and Firebase.
If you won’t be at the conference, head over the ng-conf home page for a link to a livestream of all the talks. You can also follow @Firebase and #ngconf on Twitter for the latest updates. The ng-conf organizers have pulled together a great lineup of events. Here’s where you can find us at the conference:
Hear the latest on Firebase + Angular, including a special announcement: On Friday at 10:35am MST (GMT -7), I will be speaking about the powerful combination of Firebase and Angular, and how the two tools are changing the future of modern web apps. I’ll also be making an announcement about Firebase.
Ask questions at Firebase office hours: For those of you attending ng-conf, get your technical Firebase + Angular questions answered in person and share your apps with the Firebase team! We’re holding office hours on Friday 1/17 from 7:30 - 9am. Sign up in advance or drop in at the conference.
Firebase Sponsoring ng-party: We are the sponsor of the official ng-conf party on Thursday night (1/16). The party will feature performances from a comedian, an illusionist, an improv group, and there will be some delicious desserts.
Firebase + Angular Quickstart Guide: Our new quickstart guide provides all the resources you need to start building realtime apps with Firebase and Angular.
Firebase + Angular Starter Pack: To see all of our example apps built with Firebase and Angular, simply clone our Angular Starter Pack on GitHub. The examples are a great place to start playing with code and learning about the integration.
The Complete Angular Guide: If you’re new to Angular, we recommend ng-book, a comprehensive guide to AngularJS with a chapter on adding a Firebase backend to your app.
Firebase + Angular Google Group: Join our Firebase + Angular Google Group to ask technical questions or share your apps with the community.
James, Andrew, Sara and I are looking forward to seeing many of you at ng-conf!
Update (November 4, 2014): While this post still contains some useful and relevant information,we have released advanced query functionality which solves a lot of the problems this post discusses.You can read more about it in our queries blog post.
In Part 1 of this post, we covered a series of common SQL queries and how they can be recreated in Firebase, building off our authoritative Denormalizing is Normal post from last year. We're going to build on several principles introduced in those articles, so be sure to check those out before digging into this post.
In this tutorial, we'll explore a fast and powerful approach to performing text searches, or content searches, in Firebase.
Why Not Just WHERE foo LIKE '%bar%'?
The 20 year old SQL paradigm for content queries (WHERE foo LIKE '%bar%') is a staple for static databases, but not a simple prospect for real-time data. Our team is hard at work on a series of tools to bring content searches into the lightning-fast realm of Firebase's NoSQL data store. Look for more news on these indexing and query-related tools in the coming months.
Until then, I'd like to introduce you to a few quick scripts that can add powerful content searches to your app. At the end of the article, I'll share a library that incorporates these strategies into a service you can clone, configure, and run on your own box.
Introducing ElasticSearch
ElasticSearch, based on Lucene,is an extremely powerful document storage and indexing tool. However, at its core is a very simple search feature, which is nearly plug-and-play compatible with Firebase.
While it's certainly not the only way to write content searches in Firebase, ElasticSearch's simple integration makes it fast to implement and takes very little code to utilize, while its powerful indexing capabilities provide for customization, allowing it to scale with your app.
You can set up a local instance for testing in three steps:
It's really that simple! And, surprisingly, deploying a free, hosted instance requires nothing more than a button click thanks to Heroku's Bonsai add-on.
Indexing Firebase Data
The first step is to get data into ElasticSearch so it can be indexed. A simple Node.js script can plug Firebase into ElasticSearch with a few lines of work. I utilized the node-elasticsearch-client library, which is optional, but simplifies the process by wrapping the lower level ElasticSearch client:
var Firebase = require('firebase'); var ElasticClient = require('elasticsearchclient') // initialize our ElasticSearch API var client = new ElasticClient({ host: 'localhost', port: 9200 }); // listen for changes to Firebase data var fb = new Firebase('<INSTANCE>.firebaseio.com/widgets'); fb.on('child_added', createOrUpdateIndex); fb.on('child_changed', createOrUpdateIndex); fb.on('child_removed', removeIndex); function createOrUpdateIndex(snap) { client.index(this.index, this.type, snap.val(), snap.key()) .on('data', function(data) { console.log('indexed ', snap.key()); }) .on('error', function(err) { /* handle errors */ }); } function removeIndex(snap) { client.deleteDocument(this.index, this.type, snap.key(), function(error, data) { if( error ) console.error('failed to delete', snap.key(), error); else console.log('deleted', snap.key()); }); }
Drop that in a hosting environment like Heroku or Nodejitsu, or onto your own host with forever to monitor up-time, and search indexing is done! Now it's time to read some of that data back.
A Brute Force Search
Once we have our data indexed in ElasticSearch, we could directly query our index using a wrapper like elastic.js. This is a perfectly reasonable option, but does add coupling and dependencies to the client:
<script src="elastic.min.js"></script> <script src="elastic-jquery-client.min.js"></script> <script> ejs.client = ejs.jQueryClient('http://localhost:9200'); client.search({ index: 'firebase', type: 'widget', body: ejs.Request().query(ejs.MatchQuery('title', 'foo')) }, function (error, response) { // handle response }); </script>
Since our clients are already using Firebase, wouldn't it be great to keep our client code agnostic and push the request to Firebase instead?
A Firebase Search Queue
This little node script listens at /search/request for incoming searches, handles the interactions with ElasticSearch, and pushes results back into /search/response:
/search/request
/search/response
var Firebase = require('firebase'); var ElasticClient = require('elasticsearchclient') // initialize our ElasticSearch API var client = new ElasticClient({ host: 'localhost', port: 9200 }); // listen for requests at https://<INSTANCE>.firebaseio.com/search/request var queue = new Firebase('https://<INSTANCE>.firebaseio.com/search'); queue.child('request').on('child_added', processRequest); function processRequest(snap) { snap.ref().remove(); // clear the request after we receive it var data = snap.val(); // Query ElasticSearch client.search(dat.index, dat.type, { "query": { 'query_string': { query: dat.query } }) .on('data', function(data) { // Post the results to https://<INSTANCE>.firebaseio.com/search/response queue.child('response/'+snap.key()).set(results); }) .on('error', function(error){ /* process errors */ }); .exec(); }
A Client Example
Now that we have a way to queue requests into Firebase, the client can simply push requests and listen for results:
<script> var queue = new Firebase('https://<INSTANCE>.firebaseio.com/search'); function search(index, type, searchTerm, callback) { // post search requests to https://<INSTANCE>.firebaseio.com/search/request var reqRef = queue.child('request').push({ index: index, type: type, query: searchTerm }); // read the replies from https://<INSTANCE>.firebaseio.com/search/response queue.child('response/'+reqRef.key()).on('value', function fn(snap) { if( snap.val() !== null ) { // wait for data snap.ref().off('value', fn); // stop listening snap.ref().remove(); // clear the queue callback(snap.val()); } }); } // invoke a search for *foo* search('firebase', 'widget', '*foo*', function(data) { console.log('got back '+data.total+' hits'); if( data.hits ) { data.hits.forEach(function(hit) { console.log(hit); }); } }); </script>
A Pre-Built Library for Your Use
We've implemented a content search for Firebase using ElasticSearch, set up a queue to ferry data transparently to and from our search engine, and finished all of this in a couple short scripts. We've also tapped into the powerful, flexible, and scalable world of ElasticSearch, which will grow with our app.
Easy enough? I've gone ahead and baked these scripts into a GitHub repo just to make things even simpler: Fork the Flashlight service on GitHub.
It's MIT Licensed and ready for cloning. Just edit the config file, start it up, and get back to work on your app!
We want your feedback!
Have fun and let us know how we did! We love getting feedback from our dev community. Chat with us in the comment thread or send an email to wulf@firebase.com. You can also follow @Firebase on twitter.
