VON NEUMANN, John. & GOLDSTINE, Herman H.

‘Numerical Inverting of Matrices of High Order,’ pp. 1021-1099 in Bulletin of the American Mathematical Society, Vol. 53, No. 11, November, 1947. [Offered with:] ‘Numerical Inverting of Matrices of High Order II,’ pp. 188-202 in Proceedings of the American Mathematical Society, Vol. 2, No. 2, April, 1951.

Menasha, Wis. & New York: Published by the Society, 1947 [-1951].

First edition, journal issues in the original printed wrappers, of two of von Neumann’s major papers. “The 1947 paper by John von Neumann and Herman Goldstine, ‘Numerical Inverting of Matrices of High Order’ (Bulletin of the AMS, Nov. 1947), is considered as the birth certificate of numerical analysis. Since its publication, the evolution of this domain has been enormous” (Bultheel & Cools). “Just when modern computers were being invented (those digital, electronic, and programmable), John von Neumann and Herman Goldstine wrote a paper to illustrate the mathematical analyses that they believed would be needed to use the new machines effectively and to guide the development of still faster computers. Their foresight and the congruence of historical events made their work the first modern paper in numerical analysis. Von Neumann once remarked that to found a mathematical theory one had to prove the first theorem, which he and Goldstine did concerning the accuracy of mechanized Gaussian elimination – but their paper was about more than that. Von Neumann and Goldstine described what they surmised would be the significant questions once computers became available for computational science, and they suggested enduring ways to answer them” (Grcar, p. 607). “In sum, von Neumann’s paper contains much that is unappreciated or at least unattributed to him. The contents are so familiar, it is easy to forget von Neumann is not repeating what everyone knows. He anticipated many of the developments in the field he originated, and his theorems on the accuracy of Gaussian elimination have not been encompassed in half a century. The paper is among von Neumann's many firsts in computer science. It is the first paper in modern numerical analysis, and the most recent by a person of von Neumann’s genius” (Vuik). Von Neumann & Goldstine’s 1947 paper is here accompanied by its sequel (the 1947 paper comprises Chapters I-VII, the sequel Chapters VIII-IX), in which the authors reassess the error estimates proved in the first part from a probabilistic point of view. The only other copy of either paper listed on ABPC/RBH is the OOC copy of part I (both journal issue and offprint).

“Before computers, numerical analysis consisted of stopgap measures for the physical problems that could not be analytically reduced. The resulting hand computations were increasingly aided by mechanical tools which are comparatively well documented, but little was written about numerical algorithms because computing was not considered an archival contribution. “The state of numerical mathematics stayed pretty much the same as Gauss left it until World War II” [Goldstine, The Computer from Pascal to Von Neumann (1972), p. 287]. “Some astronomers and statisticians did computing as part of their research, but few other scientists were numerically oriented. Among mathematicians, numerical analysis had a poor reputation and attracted few specialists” [Aspray, John von Neumann and the Origins of Modern Computing (1999), pp. 49–50]. “As a branch of mathematics, it probably ranked the lowest, even below statistics, in terms of what most university mathematicians found interesting” [Hodges, Alan Turing: the Enigma (1983), p. 316].

“In this environment John von Neumann and Herman Goldstine wrote the first modern paper on numerical analysis, ‘Numerical Inverting of Matrices of High Order’, and they audaciously published the paper in the journal of record for the American Mathematical Society. The inversion paper was part of von Neumann’s efforts to create a mathematical discipline around the new computing machines. Gaussian elimination was chosen to focus the paper, but matrices were not its only subject. The paper was the first to distinguish between the stability of a mathematical problem and of its numerical approximation, to explain the significance in this context of the ‘Courant criterium’ (later CFL condition), to point out the advantages of computerized mixed precision arithmetic, to use a matrix decomposition to prove the accuracy of a calculation, to describe a ‘figure of merit’ for calculations that became the matrix condition number, and to explain the concept of inverse, or backward, error. The inversion paper thus marked the first appearance in print of many basic concepts in numerical analysis.

“The inversion paper may not be the source from which most people learn of von Neumann’s ideas, because he disseminated his work on computing almost exclusively outside refereed journals. Such communication occurred in meetings with the many researchers who visited him at Princeton and with the staff of the numerous industrial and government laboratories whom he advised, in the extemporaneous lectures that he gave during his almost continual travels around the country, and through his many research reports which were widely circulated, although they remained unpublished. As von Neumann’s only archival publication about computers, the inversion paper offers an integrated summary of his ideas about a rapidly developing field at a time when the field had no publication venues of its own.

“The inversion paper was a seminal work whose ideas became so fully accepted that today they may appear to lack novelty or to have originated with later authors who elaborated on them more fully. It is possible to trace many provenances to the paper by noting the sequence of events, similarities of presentation, and the context of von Neumann’s activities” (Grcar, pp. 609-610).

We are fortunate to have an account of the genesis and content of these two important papers in Goldstine’s own words. In the years immediately following the end of World War II, Von Neumann, Goldstine and others instituted the ‘electronic computer project’ at the Institute for Advanced Study at Princeton, NJ. One of the first topics discussed “was the solution of large systems of linear equations, since they arise almost everywhere in numerical work. V. Bargmann and D. Montgomery collaborated with von Neumann on a paper on this subject. Then H. Hotelling, the well-known statistician, wrote an interesting paper in 1943 in which he studied a number of numerical procedures, including the Gaussian method for inverting matrices. He pointed out in a heuristic and as it turned out, inaccurate, way that the Gaussian method for inverting statistical correlation matrices [of order n] would require about k + 0.6n digits during the computation to obtain k-digit accuracy. Thus to invert a matrix of order 100 would in his terms require 70 digits to be used if one wanted 10-digit accuracy.

“Johnny and I never quite believed that Gauss would have used a procedure so lacking in elegance, given his great love for computation. Indeed, his collected works contain a considerable amount of material on both astronomy and geodesy that shows his love for and great skill in computation. As some partial evidence of this, we know he certainly used the so-called Cooley-Tukey method to handle Fourier transforms. Taking his skill as given, we looked closely at the procedure and wrote a paper on the subject that we used as an elaborate introduction to errors in numerical calculation [the first offered paper]. We tried in that paper to alert the practitioners in the field to a phenomenon that had not been particularly relevant in the past but was to be a constant source of anxiety in the future: numerical instability. In the course of the analysis we also brought to the fore the now obvious notion of well- and ill-conditioned matrices …

“In a second paper we raised a question that we thought might become more important than in fact it ever became [second offered paper]. We said, let us not worry so much about what might happen in a very small number of pathological cases; instead let us see what occurs on the average, what we can expect if we need to do this same task many times. To achieve this probabilistic result I had to develop proofs for several theorems in probability theory, which I did with considerable difficulty, only to receive a letter from a statistician named Mulholland after the paper appeared in which he showed me how to do one part with the slightest work: A mere flip of the wrist sufficed to demonstrate some obvious thing. My only consolation was that Johnny had not seen how to do it simply either. In the even, I suppose that our second paper sacred practitioners of the subject away from the field of probabilistic estimates instead of bringing them in” (Goldstine, pp. 10-11).

“Von Neumann and Goldstine’s paper [I] has been called the first in this ‘modern’ numerical analysis because it is the first to study rounding error and because much of the paper is an essay on scientific computing (albeit with an emphasis on numerical linear algebra). The list of error sources in Chapter 1 is clearer and more authoritative than any since. The axiomatic treatment of rounding error in Chapter 2 inspired later analyses. The discussion of linear algebra in Chapters 3 to 5 contains original material, including the invention of triangular factoring … The rounding error analysis in Chapter 6 accounts for just one-quarter of the paper, with the analysis of triangular factoring a fraction of that … The bulk of Chapter 6 then bounds the residual of inverting symmetric positive definite matrices … The concluding Chapter 7 interprets the rounding error analysis … The paper closes by evaluating the residual bound for "random" matrices, and by counting arithmetic operations” (Vuik).

Hook & Norman, Origins of Cyberspace 957 (first part only). Bultheel & Cools (eds.), The Birth of Numerical Analysis, 2009; Goldstine, ‘Remembrance of things past,’ in A History of Scientific Computing, Stephen G. Nash (ed.), pp. 5-16; DSB XIV, 92; Grcar, ‘John von Neumann’s Analysis of Gaussian Elimination: the Founding of Modern Numerical Analysis,’ SIAM Review 53 (2011), 607-682; Kees Vuik, Birthday of Modern Numerical Analysis, (ta.twi.tudelft.nl/users/vuik/wi211/num_anal.html).

Two vols., 8vo (242 x 152mm). Part I: pp. 1021-1140; Part II: pp. 175-334. Original printed wrappers. Book label of Erwin Tomash inside front cover of each issue. Fine copies.

Item #5481

Price: $2,250.00