There is an old saying that “the map is not the territory.” Technical certification isn’t even a map, it’s a receipt from the map store. Certificates don’t mean much in smaller companies, where hiring is a hopefully rare effort in which every member of the team vets serious candidates. Nor are there meaningful certificates for high-level technical leaders: They keep themselves busy shipping products and blazing new technical trails.

But for medium-sized and large businesses that need to keep large teams fleshed out, and for those developers who are content in such stable, well-paying environments, certificates are a necessary, if clumsy, signifier of competence. I remember a candidate once presented me page after page of certifications, each in its acetate protector, neatly organized in a faux-leather binder. I wouldn’t have been surprised by tabs separating his database competencies from his language skills. Perhaps such binders are common sights in certain industries, but he was applying for a job at a startup and the binder had, I’m afraid, the opposite effect than he had hoped.

It’s a perverse outcome, I know, but I have to admit that if someone shows me a certification in a core technology, I suspect they are a technology follower, not a leader, and not therefore a likely fit for the teams I’ve most often worked with. On the other hand, I wish that I could validate basic competence on tangential technologies, like shell scripting, makefiles, the particular testing, version-control, and deployment tools in use, etc.

Not only is there the classic problem of distinguishing between people who know the territory from people who’ve bought a map, but the territory shifts. What good is a C# certificate that predates “async”? In a year or two, how worthwhile will a Java certification without lambdas be? Such major structural advances in technologies shift the entire landscape. Yet, perversely, it often seems like certifications emphasize the purely cosmetic aspects of a particular IDE or tool version.

Perhaps because certificates are a hallmark of technologies that have already achieved mass-market success, no startup seems to have come along to disrupt the marketplace and address the underlying needs. Back in the original dot-com era, I worked on a business plan for the seemingly obvious infrastructure of tying technical certification to encryption and verification protocols (literally, “certificates for certificates”). The obvious advantage is that validating a claimed certification is as trivial as a badge on the candidate’s online resume, but the disruptive advantage of such an infrastructure is that certificates can become extremely fine-grained.

Instead of a single certificate declaring one’s competence at designing “solutions” for a given language and SDK, one can imagine certifying competence at the namespace, version and technique levels. The complexity of the underlying graph could, of course, be rolled up into views appropriate for HR and automated screening (“We need a person expert in namespace X version Y, advanced in Model-View-Controller design and application, competent-or-better in our preferred IDE and testing infrastructure…”).

Hand-in-hand with certifications comes the problem of test creation and validation. Creating test questions that are statistically valid at determining whether a learning objective has or has not been achieved is, as it turns out, a decidedly non-trivial task. And, it seems, algorithms that adapt test questions to previous answers are also not commonly used. The best adaptive-questioning app I’ve ever seen is the amazing 20Q toy (online at www.20q.net). One would think that someone would have tried to adapt that model for professional-level testing, but as far as I can see, it has not happened.

Whether certifications stay in acetate binder pages or they evolve to have the harder-to-game elements that I’ve outlined, they are here to stay. They are too valuable to medium-sized and large businesses that, otherwise, simply cannot winnow resumes. Developers who have invested their time and money in achieving such certifications, though, should bear in mind when applying at smaller companies that not everyone agrees on the face value of a certificate.

Larry O’Brien is a developer evangelist/advocate for Xamarin. Read his blog at www.knowing.net.