However you voted on Nov. 6 (and whether or not you were satisfied or disappointed with the election results), there are two aspects of the race that the Monday-morning quarterbacks have failed to understand, and this is why so many of them have been caught by surprise. Computers have been changing politics for some time, but this year the differences were profound.

Any election is going to be a test of character as well as an endurance match for the candidates. Sometimes the issues get swamped by a candidate’s personal popularity, sometimes a campaign gets derailed by a series of unrecoverable gaffes—but ultimately, it all boils down to the ground game on election day. Which side can turn out the most voters?

This year, the Democrats surprised even themselves. They ran a better ground game than anyone expected. The Democrats’ secret was a database system called Narwhal. It was a tightly controlled, very secret operation, and it was run separately from the campaign.

Narwhal was the largest and most sophisticated data-processing system in political history. It was actually several systems, working together, but each with its own focus. Its job was to measure and test, report and advise—all in real time. Narwhal monitored millions of registered voters, so the Democrats were able to test their ads, their e-mails and their mailers across large focus groups. They were able to look at polling data in real time across every key demographic and in every state and county and congressional district.

The system was designed to let campaign operatives know where they were being effective, where they had a safe margin, where they needed to focus their efforts and invest their resources, what were the most effective e-mails to send out to specific demographic blocs, and even the best places to buy commercials on cable and satellite channels. The result? The Democrats were able to spend (approximately) a hundred dollars less for each ad placed, and this economy was multiplied by tens of thousands of ad-buys.

The Democratic fund-raising machinery was equally sophisticated. With 6 million potential donors in the database, the campaign could send out targeted e-mails, sometimes asking for donations as small as $3. And while $3 might sound insignificant, as many as a million people might respond on a single day. A million one-click donations (sometimes to the president’s campaign, sometimes to a specific senate campaign or set of house races) created enormous maneuverability of the party’s resources in the swing states.

Narwhal worked.
#!
In case you didn’t know, a narwhal is a single-horned whale. Its most ferocious predator is the killer whale—the orca. Hence the Republican Party put together its own system and called it Orca.

Orca was designed to coordinate a small army of committed volunteers in swing states, sending them e-mails and text messages, letting them know who in their area had not yet voted, who needed a phone call, who needed a ride, and so on.

According to news reports, there were 30,000 committed volunteers in key districts all over the country. Unfortunately, Orca crashed on Nov. 6, paralyzing this part of the Republican effort. (Anonymous has claimed credit for crashing Orca, allegedly because it had a nefarious backdoor, but as of this writing, I haven’t seen any evidence of that beyond their own claim.) What is far more likely is that Orca’s failure was a management failure: The system hadn’t been properly load-tested and couldn’t handle its own message traffic.

The Democrats started planning for this election in May of 2011. Their boffins worked 14-hour days, testing, refining, developing and rewriting code, all the while working to ensure that their systems would be able to handle the increasing data loads as Election Day approached.

Apparently the Republican effort was simply not as well run. Did they load-test? Did they crash-test? There hasn’t been a lot of public discussion of what happened behind the scenes. and I doubt there will be. So we don’t know why Orca failed. What we do know is that it didn’t perform as hoped and expected. E-mails and text messages didn’t arrive. Calls to the data centers went unanswered. Critical information wasn’t available, and according to reports, too many in the army of Republican volunteers were left floundering with little or nothing to do on Election Day.

That may be the most important lesson to be learned for the future. You only get once chance at running an effective ground game, and it’s the most critical part of the campaign. Your entire effort has to be focused on turnout, not just for one day, but for every day of the campaign. The Democrats got it right this year. The Republicans did not. I think this is a lesson that both parties will be paying close attention to for the future.

But there’s a second lesson to be taken from this election as well, and this one is even more critical. It’s the part of the campaign that neither party has any control over and probably never will: the Internet.
#!
This was very much an Internet-influenced election. In theory, a well-educated and well-informed population will generally make wise choices at the polls. Many people, on both sides of the aisles, feel that the news media have not been living up to their responsibilities. And they’re right. The evidence shows that the news media has been chasing ratings instead of credibility. Our news outlets have consistently failed to adequately examine some the most serious issues challenging the nation. The media has failed to create a well-informed electorate. (Example: Neither candidate, neither party, neither campaign addressed global climate change, and Hurricane Sandy arrived just in time to demonstrate that failure. But aside from global climate change, there were many other issues as well that the media overlooked.)

But where the news media failed, the social networks excelled—in a bizarre, haphazard, out-of-control, marvelously chaotic perfect storm of interactions—a cascade of commentary, opinions, lies, propaganda, facts, rebuttals, insults, beliefs, convictions, commitments, outrages, insights, revelations, admissions, anecdotes, charts, data-dumps, sarcastic mini-posters, and inescapable memes. Facebook, Twitter, Reddit and all the other forums for online discussion became the white-hot ground zero of our national political conversation. This was the place to find the down-and-dirty conversations that the candidates and the campaigns weren’t having. The issues were held up to the light, examined, exhumed, exercised, exorcised, and beaten nearly to death on the anvil of debate. No gaffe went uncommented. No lie went unchallenged. No assertion went unexamined. No fumble went unnoticed. Reactions and responses went viral within minutes of the original event. Nothing and no one escaped the meat grinder of the Internet.

And regardless of how silly some of it was (birth certificates and Etch-A-Sketches), regardless how off-purpose much of it was, at the heart of it all was a passionate involvement by millions of people in the nation’s future. This was an electorate in action, informing and educating itself. Yes, the process was often clumsy and juvenile and occasionally appalling, but what the social networks provided was an immediacy of purpose. As a people, we were able to engage in our quadrennial national ritual to an unprecedented degree.

This heartfelt examination of the candidates and the issues is an indicator of the power of the social media; it’s the tip of a massive iceberg. It demonstrates that important national discussions can be held in open forums where college students and small businessmen and housewives and seniors and working-class Americans can meet and share and learn from each other, and do all of this separate and apart from the empty noises of the Sunday gasbags, the talk-radio pundits, and all the cable news explainers, away from the manipulators and propagandists who have polluted the national conversation with all their divorced-from-reality hidden agendas and facile talking points.

Now, why are these two points important to the software development community?

The first one is obvious. Political campaigns, like any other form of marketing, have a need for accurate information on the voters and how to reach them. The more accurate and sophisticated the information, the better the campaign can invest its resources. The Democratic campaign proved that effective campaigns in the future will require a highly-motivated team of data-meisters and a significant expenditure on hardware and information-gathering resources. On the other side, the Republican campaign demonstrated that dependence on a single centralized system is an invitation to disaster. The future belongs to those who can diddle the data in real time.

The second point? The social media have become the Internet. As valuable and necessary as all the rest of the Web might be, the average user spends most of his time interacting with friends and family on Facebook. Where we used to have e-mail and websites and blogs, now we have Facebook pages. This is a profound change in the way we experience our relationships and the communities we create around ourselves.

Quite simply, we are evolving toward a different relationship with ourselves, with others, and the planet we live on; we are immersing ourselves in a global conversation that is awakening our awareness of the greater scales of human experience. The first step toward maturity is empathy, a recognition that other beings can hurt. The more we know and understand the people around us, the more we care—and the harder it is to hate them.

Our phones and our tablets, our laptops and our desktops have become the new social ecology, evidence that human beings are hungry for ways to reach out and connect with each other. That’s the future of computing. And that’s what software developers should be looking at.

What do you think?

David Gerrold is the author of over 50 books, several hundred articles and columns, and over a dozen television episodes, including the famous “Star Trek” episode, “The Trouble with Tribbles.” He is also an authority on computer software and programming, and takes a broad view of the evolution of advanced technologies. Readers may remember Gerrold from the Computer Language Magazine forum on CompuServe, where he was a frequent and prolific contributor in the 1990s.