The CIA and U.S. intelligence have gotten a lot of things wrong in recent years, at great cost to our national well-being. A significant part of the problem lies in “analysis,” where data is supposed to be interpreted but is all too often misinterpreted.
Gregory F. Treverton and C. Bryan Gabbard have written a new study of “analytic tradecraft,” published by RAND, that takes up the nature of the problem and looks at some of the solutions being put in place.
Some of the approaches to improving analysis they point to are technological. For example, there is a program called GENOA -II, designed to help intelligence analysts work better in groups. Among other things, it attempts to “automate team processes,” develop “cognitive aids that allow humans and machines to ‘think together’ in real-time about complicated problems,” and find ways to “overcome the biases and limitations of the human cognitive system.”
This sounds great. But count me deeply skeptical. Here’s why.
No technological solution can be better than the people running it. Consider a very simple “cognitive aid” like a computer spell-check program. These things have been around for a long time and everyone uses them. Treverton and Gabbard are smart men, who have every interest in producing a highly professional study. Treverton has handled all of Europe for the National Security Council and served as vice chair of the National Intelligence Council, overseeing the writing of America’s National Intelligence Estimates (NIEs). Gabbard is also an accomplished person, with a wealth of experience under his belt. But even so, and even with RAND editors poring over their study before it was released, the spell-check program was not fail-safe.
The Treverton-Gabbard study has:
“intellience” and “intellence” instead of intelligence;
“builiding” instead of building;
“proceess” instead of process;
“solftware” instead of software;
“uniue” instead of unique;
“syehtsis” instead of synthesis;
“coopertive” instead of cooperative;
“poential” instead of potential.
Why should there be nine such mistakes when the technology is in place to produce, almost effortlessly, a zero-error rate? The United States is not going fall victim to a surprise attack because of some typos in a RAND study. But we will fall victim to another surprise attack if don’t focus on the fact that the problem facing our intelligence community is not technology but severe shortcomings in the selection of analysts themselves.
See the case of Michael Scheuer, the kooky head of the CIA’s Osama bin Laden desk in the 1990′s, for one set of illustrations. See the case of Richard Immerman, the radical professor now in charge of analytic “integrity and standards” for the Intelligence Community, for another set of illustrations.
How many more illustrations do we need?