Probability Theory: The Logic of Science

Author : E. T. Jaynes, Wayman Crow Professor of Physics, Washington University
Publication Date : June 1994

From the Preface:

The following material is addressed to readers who are already familiar with applied mathematics at the advanced undergraduate level or preferably higher; and with some field, such as physics, chemistry, biology, geology, medicine, economics, sociology, engineering, operations research. etc., where inference is needed. A previous acquaintance with probability and statistics is not necessary; indeed, a certain amount of innocence in this area may be desirable, because there will be less to unlearn.

We are concerned with probability theory and all of its conventional mathematics, but now viewed in a wider context than that of the standard textbooks. Every chapter after the first has "new" (i.e.. not previously published) results that we think will be found interesting and useful. Many of our applications lie outside the scope of conventional probability theory as currently taught. But we think that the results will speak for themselves, and that something like the theory expounded here will become the conventional probability theory of the future.

Strict adherence to the mathematical rules of probability theory often simplifies the computations in two ways: (A) The problem of determining the sampling distribution of a "statistic" is eliminated; the evidence of tire data is displayed fully in the likelihood function, which can be written down immediately. (B) One can eliminate nuisance parameters at the beginning of a calculation, thus reducing the dirneusionality of a search algorithm. This can mean orders of magnitude reduction in computation over what would be needed with a least squares or maximum likelihood algorithm. The Bayesian computer programs of Bretthorst (1988) demonstrate these advantages impressively, leading in sortie cases to major improvements in the ability to extract information from data, over previously used methods.

Some readers should be warned not to look for hidden subtleties of meaning which are not present. We shall, of course. explain and use all the standard technical jargon of probability and statistics - because that is our topic. But although our concern with the nature of logical inference leads us to discuss many of the same issues, our language differs greatly from the stilted jargon of logicians and philosophers. There are no linguistic tricks and there is no "meta language" gobbledygook; only plain English. We think that this will convey our message clearly enough to anyone who seriously wants to understand it. In any event, wee feel sure that no further clarity would be achieved by taking the first few steps down that infinite regress that starts with: "What do you mean by 'exists'" ?

View/Download Probability Theory: The Logic of Science