If you can estimate – to the right order of magnitude – the significant quantities in your software system, you will have a much better chance of making good decisions about architecture, algorithms, data structures and deployment.

By the numbers
The single point I want to focus on is the role of numbers in decision making about software systems. I’m talking about the numbers that define characteristics of the finished system — number of connected users/machines, scope, scale, speed, data throughput, acceptable down time and so on.

My belief is that you don’t need to know these numbers accurately, but it is absolutely vital to know them to the right scale, the right order of magnitude — tens, hundreds, thousands, millions? To make good decisions at any level, one of the many things needed is to be able to identify which numbers are relevant, then estimate them. From number of records for a sorting algorithm to number and complexity of moving objects in a video game scene rendering system, the numbers guide your decisions. Most of the time, being within a factor of 10 will be good enough. But most of the time isn’t all the time — safety-critical software usually needs better estimates!

New rules for Internet of Things
IoT is changing expectations, and introducing new relevant quantities. One example is sensor data.

How much data should you expect when you have five thousand sensors being scanned once per second? Most people would say 5,000 data points per second. This is not wrong, but in practice it is an upper limit. Why? Because some sensors, or the first level of processing of sensor signals, report changes, not actual values. Imagine a lighting control system in which software scans the on/off switches, then decides what to do. On average, the switch setting stays the same for long periods of time. But everyone expects fast response of a light to its switch. So the switch setting must be scanned 10 or more times per second. But there is no need to report long sequences of ‘no change’, just react to a few switch ‘events’ per day.

I recently saw an example for over 150 thousand sensors. The ‘typical’ scan rate was believed to be once per second, so the rough estimate for data rate was ‘one or two hundred thousand data points per second.’ When measured, this system generated between 850 and 1,350 events per second. So the reality — one or two thousand events per second — was two orders of magnitude, a factor of 100 times, smaller than the estimate.

The point is that a communications system optimized for 100K data points per second is probably suboptimal for 1K data points per second in almost every respect, including hardware required, store-and-forward queue sizes, data structures used for event reporting, error detection/correction strategy, failover sequences and the interface to the management and reporting systems. Even choices of development environment, software technology and test strategy may be wrong.

So the designers of this system need serious conversations about what they should do. Do they need to handle a theoretical peak of 100 times ‘normal’ capacity? Or perhaps the estimate was just wrong.

It’s not just speed!
There are many quantities that may not be specified in requirements — for example, dataset size, range of screen resolutions, number of simultaneous users, time before an inactive user is logged out, max and min update frequency for the code. Software engineers need to decide what numbers matter to the development method and the software design. Sometimes it’s right to pass these questions back to the requirements author.

Dig inside the code, and other numbers become significant — how many times will that code be triggered, is there a maximum iteration count for that loop, how many ways should we index that data, and so on. This is almost always territory where there’s no way of passing the problem.

Ask, then answer or estimate
So in all cases, it’s right to question…how many, how big, how frequent, how long? Then think what would change if your estimate changed by an order of magnitude. This will help you build insights into the problem and the solution, and these insights will lead you to build better software.