No one likes waiting, Kyle Kingsbury least of all. As a long-time sysadmin and developer, he was growing weary of the standard monitoring tools when when he began working on Riemann five years ago. Named after German mathematician Georg Friedrich Bernhard Riemann, Kingsbury’s tool pulls streams of monitoring data into a manageable form that can instantly show what’s going on in your network of applications.
The origins of Riemann came from Kingsbury’s need to see the results of an IT change instantly. With existing tools and monitoring software, he said, it can take two to three minutes to see what a change has done to an application cluster. But with Riemann, all that information can be seen instantly.
(Related: Why Riemann may be a big deal)
Kingsbury said he built Riemann because using brittle things like Nagios and shell scripts is time consuming and, in the end, still has a delay between observing and gathering metrics. “I wanted everything: from user metrics to CPU and load distributions,” he said.
“When you’re looking for correlations, you, don’t need six different tools. A lot of people take Riemann and use it as a system-monitoring tool, but I built it for application-level things, so you can ask quesitions like, ‘I’ve got 10 thumbnailers. Which one is hung?’
“I wrote it just for me. I thought anyone else who used it would have to be crazy. I made zero compromises. The design specifically rejects a lot of things that would be important to people because I had a specific need. No instrument can make every sound. The beauty of the tool is how well the design space has been constrained. The thing people like about Riemann is it’s extremely focused in its design. You can’t do historical state; it’s difficult to reason about simultaneity. A lot of people come at it from a traditional monitoring perspective and it doesn’t work. But it excels in the fuzzy, low-latency, high- throughput space. You get most of the answers now in exchange for nothing later.”
Those compromises are indeed quite large for a monitoring tool. While Riemann presents itself as a library and a domain-specific language with handlers for stream processing, it’s atavistically focused on just this use case: seeing what’s happening right now while ignoring the past. As Kingsbury said, there is no support for metric histories in Riemann; that task is left to whatever log aggregation solution you choose, he said.
Thus, Riemann is almost like an odometer or speedometer: It shows you only what’s happening right now. Paul Kim, software engineer at Apigee, said that Riemann is “written in Clojure and runs on the JVM.” He is currently evaluating Riemann for use at Apigee.
Kim explained that there were a few factors thatwhich drew him and his coworkers to Riemann. “One of the big things was that it was designed specifically for monitoring. It implemented the primitives for alerting in a very expressive way. Because it was built into the language, you create the language functions in Clojure,” he said. This also means that Java libraries and existing applications can be pulled into the Riemann stream of consciousness thanks to Clojure’s JVM basis.
“The idea is that you have the data coming out of your systems, all the metrics about system health, how many requests you’re returning, how many errors, etc., you turn them into metrics and a stream of events, and Riemann, as a monitoring system, will take these streams of events and evaluate them and start filtering things out,” said Kim. “You can say you only care about HTTP events, or TCP connections, or CPU load: For each of those streams, they filter out even further. You have this way to treat each stream individually and give it its own separate logic. Because Clojure has immutable data structures, you don’t have to worry about stepping on each other’s toes. It’s easy to duplicate the stream and check this stream for latencies and that stream for hard errors.”
Despite the power and utility of Riemann, however, Kingsbury won’t be leaving his day job at Factual, a location data company. He said he has zero interest in making Riemann into a commercial product, or in offering professional support services for the open-source project. He said he simply doesn’t find that appealing, despite the fact that many Silicon Valley investors have attempted to woo him into starting a Riemann business.
Kingsbury said, however, that building a business around the software would require him to make compromises on how the project works, which he said would remove the unique power of the platform. When an open-source project goes commercial, he said, customers dictate the product road map, not the developers, and this is precisely why he has no interest in commercializing Riemann. He refuses to compromise.
