Age of the Racecar Driver
I have an absolutely fascinating interview story to tell you.
The other day I phone-screened a guy who claimed he has an undergrad degree in Computer Science from a high-profile school. I say "claimed", because I don't know much about this school's CS program. This guy may have been a big fat liar. But his claim had the ring of truth to it, inasmuch as he obviously knew something about programming, even if it's not what you might call "Computer Science".
I did this screen, incidentally, at the request of one of our internal recruiters, who felt this candidate was a "very strong Java guy", and wanted me to recommend strong Java interviewers. I gave him a long list of names. My name happened to be on the list, and as pure chance would have it, I was "randomly" selected to be the first screener. Funny how that worked out.
Anyway, before the screen I told the recruiter that the candidate had a 50/50 chance of failing my phone screen, based on his resume, which was filled with references to J2EE. I've come to realize that overly-heavy emphasis on J2EE in a resume means the person is probably a Turing-Unaware Racecar Driver (aka "TURD"). It turns out I was way off; the guy had a 100% chance of failing my screen, but I guess that's sort of the way probability and hindsight work together, isn't it?
Next time, though, I'll give a resume like that 9-to-1 odds against.
I can't show you the resume, so, alas, you won't be able to read it and learn from it. It's more important to maintain the candidate's privacy than to save us hundreds of thousands of dollars in wasted phone screens. But I can give you the gist of it.
First of all, the resume had Two Big Highlights. (Reminds me of how I used to say there are Two Big Reasons to see the movie Tomb Raider.) The following prominent resume features really bulged out at you:
He has a CS degree from [name of prestigious but obviously non-technical university deleted]
He recently published a high-profile book on an interesting Java technology, with a big publisher.
Those two things were probably enough to attract the attention of whoever initially stumbled on his resume.
I'll tell you the resume's main red flags, all of which indicated that he was a probable weenie in spite of his two big credentials:
He spelled "honors" incorrectly as "honours". (Ha, ha! Just kidding. That's the lighthearted kind of humour we like to inject into our blogs here at Blog Central. We actually prefer the Deep South spelling, "onners", as in: "Mah suhn Jimbo is a onners studint." )
He was the "originator " of various "concepts" at some of his jobs.
He "participated in the design" of various projects, "using Rational Rose within the Unified Process".
He was "lead developer on a feasibility study in the use of the Java Micro Edition (J2ME) API".
His list of "languages" is: "Java, AspectJ, J2EE, EJB, J2ME, C++, C#, XML, XSLT, UML".
He is a "distributed systems architecture consultant" and a "Senior Technologist" with "over 10 years of software development experience".
He hasn't done "squat" in his entire "career".
OK, well, the last one wasn't directly from his resume; it's just something I inferred by "reading" between the "lines".
I nevertheless agreed to phone-screen him, because you never know: he does have the "CS" degree and the published book.
I started picking up warning signs early on; he was being awfully vague. For instance, he couldn't come up with anything in particular that he disliked about Java. Here's how it went:
Me: What languages do you use primarily?
Him: I tend to be a Java programmer by day. I've done a little C and C++ a few years ago; I've probably been doing above 80% on Java for the past, oh, probably 5 to 6 years.
Me: Tell me some of the things you wish were different about Java.
Him:: Uh, things I dislike... hmmm... well, I can see a lot of weight in the C# argument that there's such a thing as properties of a class. Explicitly declaring gets and sets in Java can weigh you down. IDEs like Eclipse can help, but I wish the language recognized the fact directly. Ummm, other than that... I can't really think of anything.
Me: C'mon, bonehead, you just published a book on FooXXX, which is a completely new technology, designed to address inherent deficiencies of Java, and you can't think of anything you wish were different about Java? Tell me again why I'm talking to you?
OK, I didn't say that last part out loud. Instead, I noted it as a Red Flag: "Thinks in Java, but doesn't realize he thinks in Java, even when he's writing books about situations where you can't think clearly in Java."
At this point in the screen, I decided it might be relevant to ask him what courses he took in his degree program. It turns out that what he really had obtained was a bit more complex than I thought. The education section of his resume was a big wad of bullets and buzzwords that I had somehow blended together into the misunderstanding that he had a CS degree from the "parent" school. Maybe sort of like seeing "University of Washington Extension Program" and thinking it was actually the UW proper.
This was a blunder on my part; my only excuse is that I didn't actually read his resume until just before the phone screen. After all, I didn't find the guy -- the screen just showed up as a surprise, like most of my interviews.
So he didn't have the degree I thought he had. His degree was really a degree in "Software Engineering", and I decided to steer the screen towards his fundamentals, to see what overlap there was between what he'd learned, and what they teach in comp sci programs.
Here's how the conversation went, more or less:
Me: What courses did you take in your CS degree?
Him: Well, I took all flavors of OOP, and distributed objects. object-oriented design, requirements engineering, web services, distributed objects, security principles, design patterns, practical software engineering. It was purely a software engineering degree.
Me: Did you take a data structures course?
Him: Ah, no, didn't have to take that one.
Me: What about any of the following: operating systems, compilers, theory of computation, discrete mathematics, hardware architecture, digital design, algorithms, compilers, or machine learning?
Him: Nope.
Me: Do you know big-Oh algorithm analysis?
Him: No, not off the top of my head.
Me: Do you know the difference in lookup performance between a linked list and a hashtable?
Him: Well, a hashtable is faster, since a linked list slows down as it gets bigger.
Me: Do you know how a Java hashtable is implemented?
Him: I haven't looked into that, no.
Me: Can you speculate?
Him: Maybe a binary search? I couldn't speculate further...
Me: How does a binary search work?
Him: You have a lookup object, and jump to the middle; if it happens to be greater than your object, then you jump right, otherwise left, and it keeps halving it.
A "lookup object." Great. I felt like saying, in my best Indiana Jones impersonation: "Objects. Why did it have to be objects?" I should have known this guy was going to be a disaster. Why should I have known? Because his resume looks astoundingly like the resume of the fictitious Ignatius P. Codswallop III, a fake person with a fake resume I invented as part of a resume-screening guide I wrote last year. Ignatius had a CS-ish masters degree, a jumbled-up language list just like this candidate's, and an equally dubious list of accomplishments. The resemblance really is remarkable.
I wonder how many Senior Technologist Distributed Feasibility Architects out there think that an algorithm is an object? Probably most of them. Especially if they took the Design Patterns course. It's like Ralph Johnson said: "After a while, everything starts to look like the Strategy Pattern." Here's a headline for ya: Gang of Four discovers first-class functions, years after they publish Design Patterns. Buncha racecar drivers, if you ask me.
But the interview really bugged me, because the guy is obviously smart, and he's producing useful things for his company. How is it that he hadn't taken any of the fifteen-odd courses we associate with an undergrad CS degree?
Inevitable Specialization
Joel Spolsky calls these folks Architecture Astronauts. He's got an interesting point, but I don't think he's got quite the right point. He's absolutely correct that there is a breed of programmer out there who likes to think and talk about "architecture", and they use a lot of heavyweight frameworks and buzzwords. And they seem like they're so far removed from "reality" (the reality of RPC call formats, to cite one of Joel's examples) that they appear not to be doing anything useful.
I think the point he's missing is that the programming industry has fragmented into different roles: this is an inevitable step in the evolution of every field. Back in the times of Da Vinci, a smart person could learn pretty much everything there is to know about a subject. A "Renaissance Man" could actually be a master of many disciplines, and a world-class one at that: someone who's actually expanding the boundaries of various discipline(s).
And up through maybe the 1970s, it was possible to know essentially everything about computers and computing.
At some point, though, every discipline becomes too complex for one person to master the whole thing. Here are some examples:
In the movie industry, you used to be able to create a competitive indie movie with a couple of people and a camera. Nowadays that's virtually impossible -- Blair Witch and Clerks being obvious exceptions. There are a bunch of different specialties now, from directors and actors through special-effects folks, stunt people, set designers, lighting technicians, and so on, all the way "down" to key grips and gaffers, whatever the hell they do. It must be an important function, since every movie has them, and I can assure you that Sean Astin doesn't ever step in to play the role of "gaffer" in a pinch. Well, bad example. But you get the idea.
In the game industry, you have programmers, artists, game designers, and a whole bunch of other roles that I described briefly in A Software Fable. God knows you don't ask the programmers to draw pictures, and you don't ask game designers to write code. But it's worth noting that there are at least two entirely separate classes of programmer: the folks who work on the physics engine and scripting engine, and the folks who program the game itself using the scripting language.
In the medical profession, you have doctors, dentists, pharmacists, nurses, technicians, orderlies, surgeons, specialists, malpractice lawyers -- in fact, it's become so specialized that you can go to a Toenail Extraction Specialist. I had to do that once, after a very unfortunate snowboarding accident, and I can assure you, she was quite adept with her scalpel and pliers. I'd never have allowed a pharmacist to cut my toenail off.
The fact is, every industry eventually fragments into specializations. When people in different specialty roles need to collaborate, each one needs at least a vague idea as to what the other one does. Doctors and pharmacists, for instance, have a bit of overlap, but for the most part, doctors know surprisingly little about drugs. In fact, an ordinary family-practice doctor may not know all that much about any particular sub-specialty.
Please don't misconstrue my point here. Doctors know a lot of stuff; it takes years and years to learn it, and I'm not taking anything away from them. And so do pharmacists, and so do (for that matter) malpractice lawyers. But none of them knows everything needed to run the medical industry we have today -- an industry which, despite its flaws, it a hell of a lot better than what they had during the Renaissance.
You might be tempted to think of it this way: each role needs to know the API for any collaborating roles. You don't need to know how the person in the other role does their magic; you just need to know who does what, how to find them, and how to interact with them.
However, doctors and pharmacists sometimes make mistakes that can kill people. To reduce the frequency of these mistakes, they check each others' work, and occasionally one will notice an error: an incorrect dosage, the wrong drug name, suspicious side-effects from using a drug. The more doctors and pharmacists know about each other's fields, the less likely they are to kill someone accidentally. In fact, both doctors and pharmacists are required to do a substantial amount of ongoing training throughout their careers, for exactly that reason.
So if you were nodding in agreement when I said that different roles can treat each other as opaque APIs, you might want to think twice about it. I think different roles may often get away with treating each other as opaque boxes. But I also think having a broader and deeper understanding of how all the roles work makes everyone better at their own role.
Racecar Drivers
When we're talking about engineers who love architecture, objects, APIs, and big, heavyweight frameworks, Joel calls them Architecture Astronauts, which makes them sound clownish. I think of them as Racecar Drivers, though. For one thing, lots of people would rather be racecar drivers than racecar engineers. It's more glamorous to drive the car around, smashing up other cars. (Or am I think of monster-truck drivers?) So the term adequately reflects the widespread J2EE itch to be a "senior technologist", which sounds a lot more glamorous (to them) than being a systems engineer.
But I also think of them as racecar drivers because the drivers are doing something that's pretty darn complex in its own right. They're competing in a system built by car engineers and racetrack engineers, but the actual engineers of those systems would make lousy racecar drivers. Racecar driving is a new skill that has emerged from a complex underlying system, and it's a skill that is mostly unrelated to engineering.
In some sense, it's similar to writing: you don't have to be able to implement a word processor in order to be a professional writer. It's simply not a requirement. Authors are doing different work than programmers, and it's not necessary for them to know how to program in order to know how to write well. It may help, but it's ultimately not necessary to achieve greatness in writing.
That might sound stupidly obvious, but consider: do you need to know how to write array-sorting algorithms in order to achieve greatness in enterprise architecture? I think a TURD's situation is analogous to a professional writer's: TURDs build real-world systems using heavyweight frameworks, and they often have only the dimmest understanding of what's going on under the hood. (Hence my claim that they're Turing-Unaware -- their base-level machine is J2EE. As far as they're concerned, the level directly below J2EE is quantum mechanics.)
Because TURDs are programming (as opposed to, say, writing, or driving cars), it may appear at least superficially that they're in the same Software Development Engineer (SDE) role as the people who built the frameworks. But frameworks are many-layered beasts, and it's not clear to me that someone who's really good at doing virtual machines or RPC wire-protocols is going to be any good at building, say, B2B systems using high-level frameworks. They're just not the same skill set.
Do we need racecar drivers?
Our organization has settled on a fairly narrow definition of SDE. You are one or you aren't one, and aside from a few quasi-contentious gray areas (e.g. SDE-UI, SDE-test, support SDE, QA engineer), we tend to think of them all as being this sort of Universal Soldier, a Buckaroo Banzai-type who's a combination neurosurgeon, racecar driver, rock star, and comic book hero. Everyone we hire needs to be the last hope for the human race.
I'm starting to wonder whether this is equivalent to saying we're only going to hire doctors for our hospital. Nobody else, just doctors. When I put it that way, it seems obvious that this would be a terrible mistake. You'd never find enough doctors to fill all the roles needed in a hospital.
But what if there were a glut of doctors on the market, and they all made about the same amount of money as pharmacists, nurses, technicians, and orderlies? Then would it be OK to hire only doctors, on the assumption that they're capable of doing the work of any of those other roles?
Absolutely not, and for two reasons. The reason we all think of immediately is that doctors want to practice medicine as doctors, not as orderlies. That's analogous to the argument that you need support engineers because superstar principal-level folks don't want to "waste" time on support.
But I don't think that's a valid reason. Putting good engineers on technical-maintenance roles usually works pretty well, because the engineers will automate away the maintenance.
The real reason you don't hire only doctors is that doctors would be terrible at the other roles. Doctors don't know enough about pharmacy to be competent pharmacists; that's why we have pharmacists in the first place, isn't it? And can you imagine a doctor playing nurse, clumsily sticking you with needles and yanking your I.V. tubes out by accident?
I visited a dentist once (once!) who didn't have a dental hygienist on his staff, and he cleaned my teeth himself. What a nightmare! It felt like he had 18 thumbs jammed in my mouth. At the end of it, my teeth hurt, and I was soaking wet and miserable; heck, my teeth weren't even very clean. I never went back to him again. My next-door neighbor, also a dentist, was shocked when I told him about it. He couldn't imagine a dentist without a hygienist. They're different skills!
It's clear that hospitals (and even clinics) need many different roles, and while some of them are more specialized than others, there are various roles that you might consider to be peers, in that they all require years of specialized and mostly non-overlapping training. The fact that a doctor could go learn how to perform competently in another role is utterly moot, because you need the role today, not 3 years from now.
So hospitals need to hire people in different roles. What about us, then? Is it OK to label every role we need as an "SDE position", on the assumption that SDEs can go learn any specializations they might need?
This is an important question and I don't know the answer. Right now, hiring only so-called "SDEs" has two very obvious effects on the way we do business:
We try to calibrate all of them against each other at review time, even though they have very different skill sets and are doing very different things.
We set our hiring bar very explicitly as "someone who has a Computer Science degree or 'equivalent'". I.e. we usually refuse to hire SDEs if they're weak with algorithms and data structures.
Interestingly, our hiring practices and our calibration process both receive endless attention and discussion. It never stops, which I think is an indication that we don't have either of them right yet. If all SDEs were equivalent, and we only needed to hire one kind of SDE, then we probably wouldn't still be arguing about it after almost nine years of refining our hiring and calibration techniques.
So I'm not sure I know what an SDE is anymore. If we're a bunch of racecar and racetrack engineers, and we're building this huge system, at what point do we decide we need actual racecar drivers? I.e., people who don't know much about physics, but they know a lot about applied physics -- e.g. exactly how much to ease up on the throttle so they don't spin out of control at a certain speed.
I think our actual organization aligns itself in such a way that people fill the roles we need, at least to some extent. We have principal engineers focused on architecture, for example. That seems like a good thing. But sometimes we have strong SDEs spending months or years in roles that are effectively support roles. That doesn't seem so good; it seems to imply that they aren't empowered to automate the support.
In fact, in the latter situation, what is the right thing? Having doctors be orderlies? Even in the best of scenarios, open-heart surgery patients aren't going to wheel themselves out of the operating room. I can't think of any man-made system in any domain that runs forever without needing support technicians of some sort. And I doubt we're anywhere near solving that problem in software systems.
It's clear that software and servware organizations need to be designed just as carefully as software and servware systems. You don't build a complete software system using only databases; you need different kinds of components, each with its own specialization. And I don't think you can build a high-quality software engineering organization purely out of "generic SDEs", or you run the risk of hiring a racecar driver when you really needed a racecar engineer, or vice-versa.
I don't think we have any idea how to figure out which roles we really need, and we'll never know unless we start by agreeing to split the term SDE into a set of different and mostly unrelated roles.
At this point in the evolution of our industry, the term SDE seems about as useful as calling someone a scientist. "Scientist" was a useful distinction when we were in preschool, to distinguish that job family from, say, firemen. But in the real world, nobody is a "scientist". It's way more complicated than that, and we hardly have a use for that term anymore.
Investment Inertia
The last part of the phone-screen I had with this candidate, whom I'll refer to as Mr. Earnhardt, was by far the most interesting. For all his lack of insight into his favorite programming language, he had tremendous insight into the cultural problems at his organization. When I asked him why he's looking for work, he told me he's been having trouble getting his company to adopt the technologies and practices that he's most interested in pursuing. I asked for clarification, and he told me an interesting and familiar-sounding story...
Mr. Earnhardt's company is a defense contractor (or so I gathered) somewhere in Europe. The company has a bunch of highly skilled programmers who write platform-dependent code for Unix machines from a specific vendor. Some write in C++, some write in C, and many write directly in assembly language, because it's sort of a badge of honor to be able to do so.
These engineers, Mr. Earnhardt explained, are absolute masters at their domain. He didn't want to take anything away from them -- some of them are absolute wizards at writing platform-dependent code. One time they found a bug in the vendor's compiler, so one of the guys took the compiler home, disassembled it, found the bug, patched it, and brought the working binary in the next day. The feats these folks can perform are nothing short of astounding.
Mr. Earnhardt asked them: "What if we need to port the system to Windows NT?" Fine, they replied, we'll port it. He pointed out that it would be essentially a copy-and-paste of the code base, and that they'd now have to maintain two different (and incompatible) versions of the same system. But it was like talking to a wall. As far as they're concerned, that's just how the universe works. If you need to port your code, you port your code.
Mr. Earnhardt gave them a bunch of credit -- in fact he lavished them with praise about their abilities with C++, C, and assembly language on these Unix machines. He thought they were extraordinary in their ability to diagnose strange linker problems, patch compilers, and write high-performing C++ string libraries.
He tried to tell them that these problems have all been solved, and that it's counter-productive to be solving them all over again. His argument is that they should move up to a higher level of abstraction, and start solving new problems. But the engineers were in their comfort zone, and heavily invested in it at that. He told me they saw absolutely no need to move up a rung on the abstraction ladder, and they made the usual noises about how a higher-level system could never possibly perform well -- all the usual excuses that programmers in their comfort zones have been making since the 1950s.
So they were going out of business. It was happening slowly, but their love for their low-level systems programming was keeping them from being competitive with companies that were simply solving the problems, rather than writing string libraries that would perform incredibly well if they ever got around to solving the business problem.
However, these folks were wizards at C and assembly, and they had delivered cool things in the past, so they had the ear of the management, and the C++ wizards said: "We don't need Java!" Hence, Mr. Earnhardt was leaving.
Although I felt he'd given a good summary, and by then I was starting to have doubts about my core interviewing philosophies, I still didn't hire him. This is in part because of a side-conversation we had, in which he explained that one other language these engineers really loved (and Earnhardt hated) was Ada. I can't do justice to how crazy he sounded in his dismissal of Ada, so I'll just copy that section out of my phone-screen notes:
Me: So they're writing C++ then.
Him: Yes, and some are even writing in assembler. Some of them are also using Ada. It works, and you can do OO in it, but it's a horrible language.
Me: What's horrible about it?
Him: Strong typing: horrible and great. Once you get your head around strong typing, it's fine, but most people who come to it find it so restrictive and painful that they give it up.
Me: Isn't Java strongly typed?
Him: Not as strongly as in Ada -- in Java you can turn anything into an Object. In Java you have these weird int and long and simple types; in Ada to use something you have to declare a type for it.
In Ada 95, the last one I used, they moved to object orientation. It sounds really silly, but the language didn't say "class" -- they used packages, not classes. You could write object oriented code in anything you liked, but it helps if the language makes it explicit.
But again, it's all bound by very strong types. When you strongly type something, you can't say "I have an int" -- you have to say "I have an int and here's its bounds". It's the reason you can write safety-critical in Ada, and you can't do it easily in other languages.
Me: Interesting! Does that mean Java programmers don't like to write safe code?
Him: No! But there's safe and then there's safe. You can achieve it in Java using good practices. In Ada there's this guarantee; in a weird way it doesn't mean the code will be good or bad; it just means it'll do what you told it to do.
Call me silly, but this is part of the reason I decided the guy was basically a crackpot. He described a powerful language with hard guarantees about program safety, already in use in production at his defense company, and decided he wanted to throw all that out the window because it's a "headache". And his claim that it wasn't true object-orientation just because they didn't use the word "class" -- well, that's just nuts.
Not to mention the fact that you can also achieve safety with good practices in C++. His double-standard was invisible to him because he was so enthralled with Java. To me, that means he's not capable of having an unbiased technology discussion, and someday if Java ever isn't the best choice for some problem, he'll be an obstruction to progress, just like his peers at his current company are today.
That mini-conversation wouldn't have disqualified him by itself, but it was definitely evidence I took into consideration in my final decision.
Bottom line: he understood how it feels when someone else is heavily invested in their skills, but he wasn't smart enough to realize how heavily he was invested in his own skills; he demonstrated this time and again in the screen. I consider it a fundamental failure: he failed the smartness test.
Should I hire a TURD?
I don't know. In fact, on this candidate, I voted not-inclined, but put in the notes that I'd be willing to let someone else screen him. The candidate expressed an interest in web services, so we passed it along to them. They declined to screen him a second time, which my gut tells me was the right decision... but only because he was so fanatical about his framework, and not necessarily because he had no idea how a hashtable works.
Willie L. once observed: "teams hire themselves". C++ teams hire C++ programmers, Java teams hire Java programmers, and so on. And they're all really suspicious of dangerous fringe-radicals like me who don't profess to be members of any one language religion. I think in the ideal world we'd find a way to assemble teams of people with vastly different (and complementary) skill sets, but that just doesn't seem to happen in practice.
I wondered briefly whether I'd be OK with hiring a Turing-Aware Racecar Driver: someone who does know how the underlying machine works, but who still wants to be a racecar driver (aka "Senior Technologist"), and who has been acting in that capacity for a long time. To be honest, I don't think I want any TARDs on my team either.
But I'm beginning to wonder if my hiring instincts are right. This is a hard problem.
(Published Jan 13 2005)