Back when I used to do technology consulting for banks and other financial firms, I found myself in the middle of a project with many of the characteristics that made the rollout of the Patient Protection and Affordable Care Act so difficult: hard deadlines fixed by lease expirations and some regulatory requirements, mission creep, and project requirements set by distant, hard-to-reach figures. For a month or so, I slept two to three hours a night, if at all, which has cognitive effects that I will save for a post on an even slower news day than this.
As you can imagine, not everything went smoothly on launch day. Amid the chaos, I got a call from the secretary of a very senior executive at the firm. His new voice-recognition software wasn’t working, and he needed me to come up right away.
I had servers that weren’t working right and a bunch of workstations that couldn’t access the network. “He should call the help desk,” I told her.
Her tone was arctic.
“He doesn’t deal with help desk personnel,” she said. “Please come up here right away.”
So I went to the office of Mr. Senior Executive. He was not at his desk. I played with his new software, which seemed to be working fine -- a bit slow, but in 1998, voice-recognition software took a while to become acclimated to your voice. I told the secretary it seemed to be working, and I left my pager number. It went off as I got to the elevator bank. I trekked wearily back to the office, where Mr. Senior Executive gestured at his computer. “It still doesn’t work right,” he said, and started to leave the office again.
“Hold on, please,” I said. “Can you show me exactly what’s not working?”
“It’s not doing what I want,” he said.
“What do you want?” I asked.
“I want it to be,” he replied, “like the computer on `Star Trek: The Next Generation.'”
“Sir, that’s an actor,” I replied evenly, despite being on the sleepless verge of hysteria. With even more heroic self-restraint, I did not add “We can get you an actor to sit under your desk. But we’d have to pay SAG rates.”
Now, when I used to tell this story to tech people, the moral was that executives are idiots. No, make that “users are idiots.” Tech people tend to regard their end-users as a sort of intermediate form of life between chimps and information-technology staffers: They’ve stopped throwing around their feces, but they can’t really be said to know how to use tools.
And, of course, users can do some idiotic things. But this particular executive was not an idiot. He was, in fact, a very smart man who had led financial institutions on two continents. None of the IT staffers laughing at his elementary mistake would have lasted for a week in his job.
Call it “the illusion of omnicompetence.” When you know a lot about one thing, you spend a lot of time watching the less knowledgeable make elementary errors. You can easily infer from this that you are very smart, and they are very stupid. Presumably, our bank executive knew that the phasers and replicators on "Star Trek" are fake; why did he think that the talking computer would be any more real?
But why should he have known that voice-recognition software, circa 1998, was sort of slow and ponderous? He was getting paid to think about financial issues, not the limits of computer learning. To be sure, he probably should have asked more questions before he bought that software. But as any journalist will tell you, the greatest danger of going into a new domain is the questions that you don’t know enough to ask.
I thought about "Star Trek: TNG" Man when I saw this weekend’s New York Times piece on what was going on behind the scenes during the final months of HealthCare.gov’s troubled development:
Eventually, Medicare agency officials began to suspect that staff members at CGI were intentionally trying to hide flaws in the system, to cover up for their inability to meet production deadlines. They ordered CGI technicians to drive from their offices near Dulles International Airport in Virginia to the agency headquarters near Baltimore to review their code with government supervisors…
Mr. Chao seemed to colleagues to be at his wit’s end. One evening last summer, he called Wallace Fung, who retired in 2008 as the Medicare agency’s chief technology officer. Mr. Fung said in an interview that he told Mr. Chao to greatly simplify the site’s functions. “Henry, this is not going to work. You cannot build this kind of system overnight,” Mr. Fung said he told him.
“I know,” Mr. Chao answered, according to Mr. Fung. “But I cannot talk them out of it.”
In the last week of September, the disastrous results of the project’s inept management and execution were becoming fully apparent. The agency pressed CGI to explain why a performance test showed that the site could not handle more than 500 simultaneous users. The response once again exhibited the blame-shifting that had plagued the project for months.
“We have not identified any inefficient and defective code,” a CGI executive responded in an email to federal project managers, pointing again to database technology that the Medicare agency had ordered it to use as the culprit, at least in part.
The technocratic idea is that you put a bunch of smart, competent people in government -- folks who really want the thing to work -- and they’ll make it happen. But “smart, competent people” are not a generic quantity; they’re incredibly domain-specific. Most academics couldn’t run a lemonade stand. Most successful entrepreneurs wouldn’t be able to muster the monomaniacal devotion needed to get a Ph.D. Neither group produces many folks who can consistently generate readable, engaging writing on a deadline. And none of us would be able to win a campaign for Congress.
Yet in my experience, the majority of people in these domains think that they could do everyone else’s job better, if they weren’t so busy with whatever it is they’re doing so well. It’s the illusion of omnicompetence, and in the case of HealthCare.gov, it seems to have been nearly fatal.
The policy people handed out impossible orders to the technical staff; when the technical staff couldn’t deliver their impossibility, they decided that the problem was incompetence. This percolated all the way down the line, and quite probably back up again -- why bother explaining things if the people you’re doing the explaining to are idiots? The supercilious tone that Henry Chao, the Medicare agency's deputy information officer, took toward congressmen -- particularly Republican congressmen -- in his recent testimony sounded a lot like I did 15 years ago when I told a client just how stupid they were being. (That was a mistake I made only once, thank God -- and my boss rightly ripped my head off after the meeting was over.) It sounded like a man who was fed up with all the fools who surround him.
But at the end of the day, when people are demanding that you do the impossible, your job is to explain why you can’t. The ability to manage the expectations of nontechnical users is actually an important piece of domain knowledge for technical people; if you flub that, you’ll fail just as surely as if you get the hardware or software wrong.
And, of course, on the other side, nontechnical stakeholders should make it their business to find out what is possible before, not after, they commit to designs and deadlines.
We like to think that being “smart and competent” makes you less likely to make mistakes. But when you’re out of your element, it may merely enable you to make more -- and larger -- mistakes.