In open-source parlance, the year 2010 can be summed up in one word: Hadoop. But what are the open-source projects that will be most important in 2011? Will NoSQLs be hanging around, or will a newfangled dynamic language framework like Rails be the hip new thing? We’ve taken out our crystal balls and, with the help of some open-source experts, picked five projects that we feel will be the most important for developers in 2011.
#!
With so many shops now playing with Hadoop, the architectural questions tend to revolve around, “Why do we need two giant data stores; one for live data and one for Hadoop’s after-the-fact number crunching?” And indeed, this is a question asked by StumbleUpon, the popular website-rating browser plug-in. Rather than live with two data stores, StumbleUpon has been contributing caching capabilities to HBase, the database built for Hadoop.
With HBase’s new caching capabilities, entire sites can now be built with Hadoop as the live back end. This saves time and money as all the information remains in one place, without the need to load information off of production servers and onto the Hadoop cluster after the fact. If HBase continues to evolve in this way in 2011, it might become a viable alternative to MySQL for simpler database-backed applications. And while the conventional wisdom has said that Hadoop is ill-suited to hosting live data, HBase users are starting to find that this isn’t necessarily the case.
#!
Eclipse has long been an essential tool for developers. With the arrival of the RCP project for Eclipse, the platform moved from just being an IDE into the new category of application platform.
But the transformations don’t end there. Mike Milinkovich, director of the Eclipse Foundation, said that Project Virgo offers an even further expansion of the Eclipse platform into the realm of the runtime.
“We’ve been talking for a number of years now around the idea of Eclipse getting more into runtimes,” he said. “With Virgo, Gemini and Jetty coming to Eclipse in the last 18 months or so, it has been a watershed moment for Eclipse and runtimes. We’ve now hit critical mass, and you can get a pretty cool runtime server from Eclipse. I can’t think of anything I can talk about specifically, but I am already seeing projects and products looking at Virgo as something to move to.”
And as previous years have shown, never bet against Eclipse expanding into new areas of development usage; developers will always find ways to use more Eclipse.
#!
The flirtation with NoSQLs is over; the databases will actually have to perform in the real world to remain relevant in 2011. One NoSQL that isn’t receiving heaps of scorn in the trough of disillusionment is CouchDB.
Mu Dynamics’ CTO Kowsik Guruswamy was so moved by the use of CouchDB in one of his company’s products that he was driven to blog, tongue-in-cheek, about how the project makes building a startup too easy. “CouchDB written in Erlang is honestly spooky,” he wrote.
“With an uptime of eight months and memory utilization of 0.1%, we think it’s actually survived a few power outages too! The joy of running an online community site is the debugging, thinking and solving locking and concurrency issues, outages, and how best to scale the site as the traffic continues to increase.
“CouchDB has taken all that fun away. Things just work, we never have to see why something’s broken and not worry too much about scaling. This means we don’t have to hire more people to manage the site, which means we can’t show the investors and our customers that we are growing rapidly.”
CouchDB is also quickly become a go-to database for mobile developers. That’s because CouchDB is all about distributed unreliably connected data stores, where information can still be stored while devices are disconnected from the network. Data corruption in CouchDB is nigh impossible, as the database has no turn-off button, just the kill switch. CouchDB remains a compelling choice for both server and handheld development.
#!
The time has come to admit that Selenium is no longer the only answer for testing Web applications. In the past, it was simply the only option, and became quite popular. But after a few years of use, it has become painfully clear that maintaining Selenium tests is tedious and can become quite a chore. Atlassian CEO Mike Cannon-Brookes said that his developers are finding Selenium to be time-consuming and frustrating. Not to mention the fact that Selenium is notoriously unreliable with browsers other than Firefox.
Enter Windmill. This small Web testing framework has been slowly gathering fans as an alternative to Selenium. Windmill controller methods offer extensive options for pushing the browser to its limits, and you can even write your tests in JavaScript to make them run faster. Best of all, Windmill works with all browsers. While it has not yet become the default replacement for Selenium, further frustration with that platform could make Windmill a popular alternative.
#!
Another year, another scripting layer for Java. But Cannon-Brookes said Gosu is a compelling new idea in Java scripting languages.
Instead of all that crazy dynamic stuff Groovy introduces, Gosu is a statically typed language that is 100% bytecode compatible with Java. It’s also object-oriented, imperative, and features both type inference and closures.
The language itself is actually eight years old, but was made open source in 2009. It began life as a way for insurance software development company Guidewire Software to write business rules. In that time, the language has gathered quite a few capabilities, and perhaps most importantly, has lost a lot of the verbosity common in Java.
Overall, Gosu is a compelling new Java scripting language, and a reasonable alternative to Ruby and Groovy.
