Engerman on Firth and Noren, 'Soviet Defense Spending: A History of CIA Estimates, 1950-1990'

Noel E. Firth, James H. Noren. Soviet Defense Spending: A History of CIA Estimates, 1950-1990. College Station: Texas A&M University Press, 1998. xix + 291 pp. $49.95 (cloth), ISBN 978-0-89096-805-5.

Reviewed by David Engerman (Department of History, University of California-Berkeley) Published on H-Diplo (April, 1999)

Military Intelligence

If anything about the Soviet Union fit Winston Churchill's famous line about "a riddle wrapped in a mystery inside an enigma," Soviet defense expenditures certainly must be high on the list. Release of only summary data (often simply one line in the government budget), bizarre and misleading categories for economic statistics, and, of course, outright deception made the task of estimating Soviet defense expenditures and economic capacity two of the major intellectual challenges of Cold-War Sovietology. Developing reliable data series and then estimating the size and structure of the Soviet economy, in fact, absorbed a good portion of the economists specializing in Soviet Studies.[1] These economists, based in universities and governmental and intergovernmental agencies as well as in the "intelligence community," devoted their professional lives to working out how to join economic tools, creative uses of proxies and substitutions, and expertise in the arcana of the Soviet economy. Deploying these tools, those economists in the intelligence agencies integrated information obtained by covert agents with the statistics published by Soviet agencies. In the end, these economists produced a series of estimates of Soviet economic strength and--even more important for U.S. budget debates--defense expenditures and capabilities.

It is these latter estimates, regarding defense, which preoccupied many of the CIA's economists. While the organizational structure changed over time, a dedicated group of Agency economists worked throughout the Cold War in Washington (and later Langley, Virginia) to compute and recompute estimates of Soviet military strength and its costs. Two of these economists, Noel E. Firth and James H. Noren, teamed up to write what amounts to a biography (or perhaps an autobiography) of a data series. Their Soviet Defense Spending: A History of CIA Estimates, 1950-1990, focuses not on the Agency of popular fascination--the Spy vs. Spy aspects of the Operations directorate--but instead on the abstract world of index-number effects, hidden inflation, and inferred costs. Yet the CIA's economic analysts, armed with calculators (!) and eventually computers rather than cloaks and daggers, produced what the authors plausibly call some of the most important information of the Cold War.

Soviet Defense Spending presents an analyst's-eye view of these crucial data, describing the organizational history of the Office of National Estimates, detailing the methods used to compute the estimates, and following the political uses of the data through the national security establishment and (occasionally) among elected officials. The mix of memoir and history allows the readers to learn from the authors' extensive experience (they total sixty-three years' experience with the CIA, almost fifty engaged in Soviet economic analysis) as well as their extensive analysis of the growing stock of declassified reports. The book is unabashedly an insider account--dedicated to those who "worked with integrity and professionalism to create and improve the important analytic capability ... that is described in this book"--which seeks to demonstrate the superiority of the CIA's methods over those of its many critics.

It should be said at the outset that some of the issues surrounding the estimates of Soviet defense spending are, literally speaking, irresolvable. For reasons unlikely to be of interest to diplomatic historians (and thus not dealt with here), the pricing method selected--whether to use rubles or dollars, which base year to use--can have an enormous impact on the final result. There is, furthermore, no simple answer to the questions of which year or which currency to use; different comparisons of defense spending require different methods. It is these issues about valuation which led to most of the political headaches about CIA estimates.

After a brief introduction, the first three chapters describe, in chronological order, key events in building the models of the Soviet economy and its defense-related sectors. The Soviet economic estimates were born at the CIA during Max Millikan's brief tenure as the founding director of the Office of Research and Reports (1951-1952). Millikan defined the strategy for such calculations as starting with an "inventory of ignorance" and then reducing the list of unknowns through "successive approximations" (p. 13). This broad directive was applied to the question of determining Soviet military expenditures through the development of the "building-block method," which began by estimating the number of physical units (whether ships, jeeps, or even soldiers) and then estimating the procurement and operating costs of each item. The legendary Sherman Kent endorsed this approach when he took charge of national estimates in 1952.[2] Building blocks had a number of key advantages: published data on physical units were deemed more accurate and in any case were easier to determine through covert means. At the same time, there remained plenty of room for "successive approximations": starting with about twenty-five spending categories, the reports of the 1990s included almost 1800 such categories (p. 15). Finally, the reports emphasized physical units--which were, after all, the most important element of Soviet defense; expenditures alone would not indicate what sort of military threat would be presented by the Red Army. Where costs were not known, analysts used analogs--using Soviet trucks or American tanks, for instance, to estimate the costs of Soviet tanks--and then adjusted for differences in known physical aspects from weight to cruising range. Analog-based data, far shakier than direct-cost data, accounted for over half of earlier estimates, dropping to about one-third by the late 1980s (p. 20).

The next chapter documents the increasing use of quantitative techniques in national security studies of the 1960s, especially as the Pentagon absorbed Secretary of Defense Robert McNamara's "whiz kids." The increasing demands on CIA resources, not least from the newly empowered Defense Intelligence Agency, kept CIA analysts running from project to project and often led them to neglect to update the price and cost figures in the CIA database (pp. 37, 41). Even more time-consuming was the first effort to establish the Strategic Cost Analysis Model (SCAM) on a computer system; the authors imply that analysts were too busy SCAMming to conduct basic research on military prices (p. 51). These resource-allocations decisions would come to haunt the Agency in the mid-1970s, as a combination of external pressures, new data (aided especially by a key defector) and internal works forced a major revision of the defense burden (that is, the percentage of the Soviet economy devoted to the military). This crisis comprises the climax of the book, as one might expect when the protagonist is a data series. The crisis sparked heated public debate when the CIA announced that their earlier estimates of Soviet defense spending at 6-8 procent of GNP lowballed the burden by as much as half; the revised estimated burden ranged from 11-13 procent.

The aftermath of this crisis is dealt with in Chapter Four, as the authors patiently defend the CIA's recalculations. The revisions, they note, had little to do with an increase in Soviet defense capability, only with expenditures. Yet the underestimates were substantial: the 1975 Soviet defense budget, earlier reported as 152 billion rubles, was re-estimated at 234 billion; the procurement estimate jumped from 56 to 94 billion (p. 60). In the heat of the moment, the criticisms of the CIA (especially from the hawks) exaggerated the impact of these calculations. Eugene Rostow compared the revisions to other "pathological phenomena" like Galileo's trial or Galen's theory of four bodily humors. Richard Nixon made the portentous if inaccurate prediction that, "Thanks in part to this intelligence blunder, we will find ourselves looking down the nuclear barrel in the 1980s" (p. 65). External review panels--nicknamed "the intelligence inquisition" by one of the besieged--made a number of suggestions to the CIA, endorsing the Agency's basic methodology while bemoaning the "truly amazing lack of understanding" among those using the defense burden estimates.[3] The chapter carries the story of the estimates through the Gorbachev era, which saw the demise of SCAM, the shrinking of the Agency's efforts in this field, and the first bilateral debates about the defense estimates themselves.

Chapter Five describes the state of the estimates in the 1980s, carrying on the fine tradition of Whiggish histories of scientific fields written by practitioners. The authors explain some of the newest innovations in the estimating methods and provide the latest summaries of CIA estimates of Soviet defense expenditures throughout the Cold War. This chapter will be especially useful for those interested in the data itself, as opposed to the computational models and interdepartmental disputes over territory and method.

The authors take a more explicit stance in defense of CIA methods in Chapter Six, in which they reply to some of the many criticisms of their work.[4] This chapter is unlikely to convince long-term critics, such as Franklyn Holzman (who has consistently criticized the CIA for overestimating Soviet defense spending) and William Lee and Steven Rosefield (who have maintained that the CIA massively underestimated Soviet defense spending). Yet the beleaguered authors make their case well, without resorting to the insinuations of some of their opponents.[5] This chapter, however, along with stray comments in the chronological sections (e.g., p. 88), implies the specious argument that anyone being criticized from all sides must be doing something right. Perhaps a more modest conclusion applies: the issues are exceptionally complex and deeply politicized.

The book concludes with an evaluation of the CIA's efforts. The authors are rightly proud of the significant accomplishments of their division: the estimates, they argue, "provided a generally accurate description of the dynamics of Soviet defense programs, and, to a lesser degree, their levels" (p. 196). They even outline two positive externalities--unintended consequences of the military-economic analysis methods: forcing a look at the day-to-day operation of the Soviet military, which allowed the Agency to assess how training levels and maintenance might affect overall war-readiness. At the same time, the analytics provided a framework for understanding the Soviet military operations as a whole, and pointed to gaps in the Agency's knowledge--Millikan's "inventory of ignorance" (pp. 199-200). Two appendices deal with subsidiary issues, such as costing improvements between the mid- 1970s controversy and the collapse of the USSR, as well as some of the issues revolving around the index-number problem.

Soviet Defense Spending is generally well-written, judicious (if also exhibiting a certain amount of professional pride), and clear. While Firth and Noren do make compelling rebuttals of many of the method-related criticisms, they are unlikely to convince their longstanding critics. The authors, furthermore, barely mention a major, and quite critical, evaluation of the CIA's work in Soviet economics. That report, conducted for the House Permanent Select Committee on Intelligence, written by leading American experts on the Soviet economy and chaired by Professor James Millar, defended the CIA against the criticism that it did not predict the great changes of 1985-91.[6] The panel also praised the CIA's methods used to estimate the Soviet military threat. At the same time, however, these scholars sharply criticized the Agency's use of related statistics, such as defense spending, defense burden, and comparative defense expenditures. Perhaps thinking back to the mid-1970s crisis, the expert panel concluded that the spending estimates were less reliable, less useful, and more frequently misunderstood (wilfully or otherwise) than data on physical units. A hypothetical Soviet attack, after all, would have come in the form of tanks, ships, missiles, or battalions--not in rubles. Firth and Noren make the general case at the start of the book that spending figures were necessary to add up all these different sorts of physical units. Yet the spending data, so easily used (and misused) in debates over the U.S. defense budget, contained theoretical problems in reporting the data, missing or inaccurate cost figures, incommensurable organizations or technologies, and other such problems. This rather fundamental question and the most recent report to raise the issue (i.e., the Millar report) are both given short shrift in Soviet Defense Spending. A related shortcoming is the very limited discussion of the defense burden's denominator, the Soviet GNP. Less fettered by secrecy concerns, a large literature on estimating the overall size of the Soviet economy (what one Russian critic called the "crafty figure") has emerged.[7]

Noel Firth and James Noren have performed a service for specialists in the Soviet economy as well as for historians interested in Cold War intelligence. Their extensive research (including a sixteen-page bibliography consisting primarily of declassified CIA reports), their insights into the history of the defense spending estimates, and their clear prose make the book essential reading for those who follow these narrow but important questions. Their thorough descriptions of the process of estimating Soviet defense bring a wealth of new information into the hands of specialists, while their tables summarizing the data will prove useful to a wider audience. Recent historical scholarship has taken a critical eye to American intelligence about specific Soviet foreign-policy decisions.[8] Soviet Defense Spending provides the sort of insider perspective necessary for scholarly analyses of economic intelligence on the USSR over the duration of the Cold War.

Notes:

[1]. For retrospectives, see James R. Millar, "Rethinking Soviet Economic Studies," in Beyond Soviet Studies, ed. Daniel Orlovsky (Baltimore: Johns Hopkins University Press, distributed for Woodrow Wilson Center Press, 1995); and Gertude E. Schroeder, "Reflections on Economic Sovietology," Post-Soviet Affairs 11 (1995): 197-234.

[2]. See especially Sherman Kent, "The Law and Custom of the National Intelligence Estimate: An Estimation of the Theory and Some Recollections Concerning the Practices of the Art" (1976), in Sherman Kent and the Board of National Estimates: Collected Essays, ed. Donald P. Steury (Washington: Center for the Study of Intelligence, CIA, 1994).

[3]. Former chief of the Defense Intelligence Agency Lt.Gen. Samuel V. Wilson, quoted in John Prados, The Soviet Estimate: U.S. Intelligence Analysis and Russian Military Strength (New York: Dial, 1982), xiii.

[4]. CIA economists have frequently made these defenses before, including one of the co-authors of Soviet Defense Spending: James H. Noren, "The Controversy over Western Measures of Soviet Defense Expenditures," Post-Soviet Affairs 11 (1995): 238-76.

[5]. One critic goes so far as to insinuate "Soviet influence" in underestimating defense expenditure levels: Steven Rosefielde, False Science: Underestimating the Soviet Arms Buildup: An Appraisal of the CIA's Direct Costing Effort, 1960-1980 (New Brunswick, N.J.: Transaction Press, 1982), 252-53. For other prominent criticisms, see Franklyn D. Holzman, "Politics and Guesswork: CIA and DIA Estimates of Soviet Military Spending," International Security 14 (fall 1989): 101-31 [with reply by James E. Steiner of the CIA and Holzman's rebuttal in International Security 14 (spring 1990): 185-98]; William T. Lee, The Estimation of Soviet Defense Expenditures, 1955-75: An Unconventional Approach (New York: Praeger, 1977).

[6]. "Survey Article: An Evaluation of the CIA's Analysis of Soviet Economic Performance, 1970-1990," Comparative Economic Studies 35 (summer 1993): 33-57. The survey was conducted for the House Permanent Select Committee on Intelligence by some of the leading figures in Soviet economics: Daniel M. Berkowitz, Joseph S. Berliner, Paul R. Gregory, Susan J. Linz, and James R. Millar (Chair).

[7]. The phrase is coined in Vasilii Seliunin and Grigorii Khanin, "Lukavaia Tsifra," Novyi Mir, February 1987, #2: 181-202; for background and an evaluation of Khanin, see Mark Harrison, "Soviet Economic Growth Statistics since 1928: The Alternative Statistics of G.I. Khanin," Europe-Asia Studies 45 (1993): 141- 67. Other criticisms of the CIA estimates include Khanin, Sovetskii ekonomicheskii rost: analiz zapadnykh otsenok (Novosibirsk: EKOR, 1993), ch. 3; U.S. General Accounting Office, Soviet Economy: Assessment of How Well the CIA Has Estimated the Size of the Economy (A Report to Honorable Daniel Patrick Moynihan, U.S. Senate, 1991); Michael Boretsky, "The Tenability of the CIA Estimates of Economic Growth," Journal of Comparative Economics 11 (December 1987): 517-42 [with a reply by CIA economist John S. Pitzer and Boretsky's rebuttal in Journal of Comparative Economics 14 (June 1990): 301-26]. One leading expert provides a judicious overview: Abraham C. Becker, "Intelligence Fiasco or Reasoned Accounting? CIA Estimates of Soviet GNP," Post-Soviet Affairs 10 (1994): 291-329.

[8]. Initial efforts to place CIA military (rather than military-economic) estimates in the diplomatic history of Cold-War crises include Matthew Evangelista, "Second-Guessing the Experts: Citizens' Group Criticism of the Central Intelligence Agency's Estimates of Soviet Military Policy," International History Review 19 (August 1997): 563-93; and Phillip A. Karber and Jerald A. Combs, "The United States, NATO, and the Soviet Threat to Western Europe: Military Estimates and Policy Options, 1945- 1963," Diplomatic History 22 (summer 1998): 399-429 [with commentaries by John S. Duffield and Matthew Evangelista in ibid. pp. 431-49]. An effort to understand the economic estimates' foreign-policy significance in non-crisis situations appears in Richard E. Kaufman, "CIA Economic Intelligence on the USSR, 1949-60," Post-Soviet Affairs, 11 (1995): 277-93.

Copyright (c) 1999 by H-Net, all rights reserved. This work may be copied for non-profit educational use if proper credit is given to the author and the list. For other permission, please contact hbooks@mail.h-net.org.

Printable Version: http://www.h-net.org/reviews/showpdf.php?id=2967

Citation: David Engerman. Review of Firth, Noel E.; Noren, James H., Soviet Defense Spending: A History of CIA Estimates, 1950-1990. H-Diplo, H-Net Reviews. April, 1999. URL: http://www.h-net.org/reviews/showrev.php?id=2967

Copyright © 1999 by H-Net, all rights reserved. H-Net permits the redistribution and reprinting of this work for nonprofit, educational purposes, with full and accurate attribution to the author, web location, date of publication, originating list, and H-Net: Humanities & Social Sciences Online. For any other proposed use, contact the Reviews editorial staff at hbooks@mail.h-net.org.