Posted on January 7, 2011 in: Web Development|Jump To Comments

Hey guys. Today I wanted to announce a new feature to the site called Spotlight – a regular roundup of recommended new Node, JavaScript & jQuery tools and resources for developers wishing to expand their utility belts further.

Because I regularly get to meet and interact with developers who are either working on are using quite powerful new tools and projects, this feature will hopefully give me a chance to share some of those recommendations with other developers in the community so they can assist as many as possible.

This Week’s Recommendations

This week, many of my recommendations include projects using Node.js. If you’ve begun using Node in your projects, you’ll already be aware of the potential fundamental impact it may have on the web application landscape. For anyone that isn’t yet familiar with what Node offers, here’s a very quick run-down of what it is and what it can do for you:

Catching Up With Node.js

Before we discuss Node, let’s talk about V8 – Google’s open-source JavaScript engine. V8 is an implementation of JavaScript which allows you to run standalone JavaScript applications. Node.js is simply a library written for V8 which provides evented I/O – think of this as something allowing you to fire an event when some IO task has been completed, rather than you waiting for it to finish before doing something else.

Node.js provides an easy way to build scalable network programs and is quite similar in design to and influenced by systems such Ruby’s Event Machine or Python’s Twisted.
It offers web development in a dynamic language (ie. JavaScript) on a VM (V8) that is faster than Ruby, Python or even Perl and so speed is most definitely at the fore-front of it’s offerings
Speaking of speed, Node is also quite efficient – it has the ability to handle thousands of concurrent connections with minimal overhead on a single process
A lot of developers will be able to pick it using it relatively quickly since JavaScript is arguably the most popular programming language at the moment.
The IO event loop inside node.js means that at any point in time, there is only one thing being done at most. This provides a number of performance gains because as there isn’t an overheard of thread switching, node is able to be fast without incurring some of the concurrency issues you may have experienced with something like Java.

I don’t want to spend too much of this post discussing how Node functions, but you may find Simon Willison’s original post on this topic quite good for further reading. A Gentle introduction to node may also be useful for beginners who are looking for a tutorial on getting setup.

Now that we’ve covered the basics, lets look at the recommended tools and projects for this week!.

Stack Testing With Zombie.js for Node: http://ajxn.it/f2LHxt

Zombie.js is a fast, simplified testing tool for Node.js which basically creates a ‘headless’ browser instance capable of requesting webpages and running customized tests against them. It’s written in CoffeeScript and offers full-stack testing out of the box.

I found simulating real user events with Zombie (think clicking buttons, filling forms etc) a breeze and it’s high level API means that even the most amateur of JavaScript developers can get started with it fairly quickly.

Zombie uses Sizzle.js (the same selector engine as jQuery does) as well as other tools you may already know so you’ll likely find aspects of using it quite a familiar experience.

If you’re interested in checking Zombie out, I strongly recommend reading this guide on it as it takes you through some useful examples of how it can be utilized in a short space of time.

You may also be interested in checking out Tobi: An alternative expressive server-side functional testing suite using jQuery that also allows you to test web apps in-browser https://github.com/LearnBoost/tobi

Working with MySQL On The Server-side With Node-MySQL from https://github.com/felixge/node-mysql

As you may know, Node.js has been going from strength to strength lately but one thing it’s been desperately missing is a stable, feature-complete MySQL driver.

Node-MySQL is a project that aims to solve this problem with providing a well-engineered non-blocking driver written in JavaScript compatible with mySQL >= 4.1. Once you’ve installed Node-MySQL, you’ll find using it quite straight-forward: it’s got a good parser that’s relatively stable, uses little memory and the project’s test-driven development approach is bound to keep issues with it to a minimum.

Unlike some smaller OSS projects, Node-MySQL actually has the benefit of sponsorship (via Joyent) at the moment meaning that updates to the project should be quite regular (excellent for any commercial sites wishing to utilize the project).

To download or fork the project see the GitHub link above.

Felix has also been involved in the related PostrgreSQL client for Node available at https://github.com/brianc/node-postgres

Routing Single Page Interface Apps With Faux.js And Backbone http://goo.gl/fb/aOcIU

If you write a lot of SPI (single app interface) apps and feel that MVC/MVP might be a little overboard for your needs, check out Faux. It’s a useful tool that uses Backbone’s library models, views and controllers and gives you a simple DSL for declaring views and wiring them up to a controller for client-side routes.

One of the things I like about Faux is that it gives you a solution that’s very route-centric and does tend to resemble traditional server-side apps a little more than some alternatives. For example, a server-side app might have a route such as ‘/books/javascript-the-good-parts’ whilst a faux-root would be ‘/#/books/javascript-the-good-parts’.

Faux prides itself on catering for apps that are easily bookmarkable and back-buttonable. Simpler apps may want to opt for Sammy.js whilst the converse for JMVC, however Faux really sits in the middle of these solutions, catering for an app that may need support for multiple ‘views/pages’ but has quite straight-forward interactions on each page. Check out the link above for Faux’s official GitHub page with examples and more.

Scraping Websites With Node.js, jQuery And Node-Scraper https://github.com/mape/node-scraper

Scraping websites for content is a common task server-side languages such as PHP, Ruby or ASP.net are regularly used to accomplish. If you’ve been following the surge of interest in Node.js over the past year, you may be interested in checking out Node-Scraper by Mathias Pettersson – a new project that makes web scraping fun, fast and simple using jQuery and Node.

Node-Scraper supports a number of different use modes and you’ll find that it’s great for both trivial tasks such as fetching single files to parse but also caters for more advanced needs such as fetching with specific user-agent headers or downloading pages in parallel. Let’s face it – we’ve all had to code up scrapers in a language at one point or another, but node-scraper genuinely makes this task quite trivial.

Another neat feature supported in throttle-limitation which helps avoid accidentally hammering a server you’re trying to scrape. Check out the repo for the project for complete usage examples.

High-Performance Server-side templating with Jade for Node: https://github.com/visionmedia/jade

Often described as robust and elegant, Jade is a high performance template engine for Node that’s quite heavily influenced by Haml. In addition to offering great readability, template compilation via the command line and memory caching it also supports Express JS (the compact server-side web development framework) right out of the box and even comes with its own TextMate bundle.

Jade’s public API supports rendering both files and strings, it’s syntax handlers cope well with a decent set of CSS selectors, attributes and doctypes and for those wishing to test it’s more advanced features, it also supports multiple classifications of executable code (eg. verbose iteration, escaped buffered code etc).

You’ll find a basic usage guide on the project repo’s readme but download the latest stable version to get access to a range of examples to help get you started.

Dropbox-Node: An OAuth-enabled node.js client for working with the Dropbox API https://github.com/evnm/dropbox-node

We all love Dropbox and luckily Evan Meagher recently wrote Dropbox-Node – a useful OAuth-enabled wrapper around the Dropbox API which covers all of the different Dropbox API methods in a clean, easy to use implementation for Node.js.

If you’re looking to learn how to easily authenticate with Dropbox and get access to features like direct uploading, file manipulation and so on, Dropbox-Node is of value to check out. It may have a few rooms for improvement but I certainly see the project lowering the barrier to a few new Dropbox apps being developed.

To wrap-up, Evan also wrote a simple Dropbox file browser to demonstrate usage of his wrapper and you can see the full example by browsing to his github repo above.

Asynchronous Client and Server-side object-relational Database Storage with Persistance.js https://github.com/zefhemel/persistencejs

Persistance.js is an asynchronous object-relational mapper that allows you to use SQLite or Gears (depending on support) in on the client-side and MySQL (through node-mysql) on the server-side for your data store needs.

It also has no dependencies on third-party JavaScript frameworks, meaning you can use it with Vanilla JS or jQuery/Dojo if you prefer. Whilst the project doesn’t support IE out of the box (something like http://code.google.com/p/dom-storage-query-language/ may help here), it does offer a wealth of features for those wishing to use an OOP approach to data storage.

Because persistence.js is asynchronous in nature, operations that can take time are provided with a callback that is invoked when the operation has completed. The project’s code is quite well commented and you get a number of decent usage examples on their repo. Check it out!.

The following are also worth taking a look at (but don’t all focus on Node specifically)

Node-validator: node-validator is a library of string validation, filtering and sanitization methods. https://github.com/chriso/node-validator

Sinon.js – Standalone and test framework agnostic JavaScript test spies, stubs and mocks compatible with everything from IE 6.0 upwards. https://github.com/cjohansen/Sinon.JS

ConsoleDummy.js – Super lightweight solution that lets you leave console statements in your JS without consequence. It does nothing for console methods that already exists. https://github.com/andyet/ConsoleDummy.js

jQuery UI Text Animation Effects: Highly configurable ports of the Dojo text effect animations over to jQuery by Sean Koole http://dev.seankoole.com/jquery/ui-effect/text.html

jQuery Inview: A compact event that is fired as soon as an element appears in the user’s viewport https://github.com/protonet/jquery.inview

and that’s it for this first Spotlight!. If there are JavaScript projects you’re involved with or using on Git that you feel would be useful to share with others, feel free to leave a comment with a link to it so we can all check it out.

Thanks and until next time, good luck with all your projects!

- Addy