3, Sandy Rise,
February 18, 1944.
Professor Fowler has sent me a copy of his letter to Dr. Bartlett, in which he writes of his discussion with you and Dr. Jeffreys regarding the possibilities of a statistical basis for quantum mechanics.
He suggests I should discuss the matter with you sometime, and I should be glad to do so if you can spare the time.
You will remember no doubt we talked about this in December 1940, when I was beginning to consider these ideas.
7 Cavendish Avenue,
I should be glad to meet you any week-end.
On Saturdays I have a lecture from 12–1 and a fire watching in the evening, but apart from that I could meet you any time on Saturday or Sunday. So choose any week-end you like. The most convenient time for me would be Saturday morning at about 10.30 or 11, when I am in the Arts School, but if this is too early for you, would you come round to my house Saturday afternoon or Sunday?
7 Cavendish Avenue,
I should be glad to see you Saturday afternoon the 11tth. If you come around 2.30 it would do very well.
I have enclosed a reprint I have just received from Whittaker, which you may care to read as it deals with the point at issue.
June 26th, 1944.
On thinking over the objection you raised when I last saw you, to my statistical treatment of quantum Mechanics, it has occurred to me that the difficulties are chiefly a question of interpretation. I think the theory can be rendered acceptable by abandoning the idea, taken over from the original (Bohr) quantum mechanics, that eigenstates have an objective reality.
One of the difficulties of the theory is that the probability distributions obtained for p and q from single eigenfunctions, can take negative values except perhaps for the ground state. Only linear superpositions of eigenfunctions lead to defined positive probability distributions in phase-space. Now, as I explained in my paper, I consider the form I obtained for the phase-space distribution F(p,q) as in a way an extension, or rather, an exact form of Heisenberg’s principle of uncertainty, in the sense that it imposes not only the well-known inequality for the dispersions of p and q, but a special form for their whole probability distribution. Perhaps then, the fact that phase-space distributions corresponding to single eigenstates can take negative values may be interpreted as meaning that an isolated conservative atomic or molecular system in a single eigenstate is a thing that cannot generally be observed without contradicting this generalised principle of uncertainty. I think this can be conceded, and no doubt physical arguments could be brought forward to support such a view. Only statistical assemblies and distributions corresponding to linear superpositions of eigenfunctions such that F(p,q,t) is always greater than zero would be observable, and would have an objective reality.
If this is accepted, it then ceases to have a meaning — to talk about a system having exact values of energy, momentum etc. in a given eigenstate, so that the second difficulty, i.e. the fact that the theory does not necessarily lead to such values, also disappears. The only thing that has a physical meaning is the working out of the final statistical distributions over a number of states, representing the results of experiments. I think that in this way my theory may be reconciled with the usual form of quantum mechanics, and may possibly lead to new results capable of experimental verifications.
The interpretation of spectra, for example, would be obtained in the usual manner from the mean values of electric dipole moments, leading to the same results as the ordinary theory. The physical notation of quantum jumps must be abandoned. The possible frequencies of the spectral lines are exhibited in the expansion of phase-space distribution at time t in terms of the phase-space eigenfunctions for fik(p,q)
The forbidden lines drop out, of course, on averaging of F(p,q,t). A more refined interpretation would involve extending the theory to radiation and its inter-action with matter.
With regard to the Stern-Gerlach experiments, I should like to quote from C.G. Darwin’s paper 'Free Motions in Wave Mechanics', Proc. Roy. Soc. A. 117 (1928) p. 260: 'in the Stern-Gerlach experiment, we do not say that the field splits the atom into two groups and then separates these. We say that a wave goes through the field, and when we calculate its intensity at the terminal plate, we find that it has two maxima which we then interpret as two patches of atoms.' This shows that the theory of the Stern-Gerlach experiment may be tackled by ordinary wave methods, without necessarily postulating exact eigenvalues for the angular momenta, and in fact, Darwin gives this theory in the same paper, on page 284. Actually, the treatment of such dynamical problems involving the evolution with time of wave packets may be simplified by the use of the methods developed in my paper, as I have shown for Darwin’s treatment of the free and uniformly accelerated electron, where in addition to his results, I also obtained the joint distribution for p and q.
In fact, I regard such dynamical problems as one case where the methods outlined may have an advantage over the usual methods. Furthermore, as the theory leads to the distributions at phase-space, and also to correlations at two instants of time, there is a possibility it may lead to experimental verifications in the field of electron and molecular beams. Another field where I think the theory may be of some value is in the study of statistical assemblies, since it leads to phase-space distributions for p and q, (equivalent to the Maxwell-Boltzmann distribution) for Fermi-Dirac and Bose-Einstein assemblies. This may be of value in the kinetic theory of non-uniform fluids.
I should like now to submit to you a few ideas of a more speculative nature. In the theory as I have developed it in my paper, a combination of the transformation equations for ψ(p), φ(p) with Newtonian mechanics, leads to Schrödinger’s equation and ordinary quantum mechanics. As I mentioned in the course of our conversation, substantially the same transformation equations combined with the mechanical equations of special relativity lead to the Klein-Gordon equation. One would expect new forms of quantum mechanics (such as your spinor equations for the electron) to appear from the combination of new transformation equations with the mechanical equations. As long, however, as these mechanical equations, whether classical or relativistic, are deterministic, the form of quantum mechanics obtained will be deterministic for isolated systems, and therefore unsatisfactory for nuclear theory. This is, I think, a further argument in favour of the idea that a satisfactory quantum theory of the nucleus must be based on some form of unitary theory involving the electro-magnetic field in a fundamental manner, since one would expect then the mechanical equations for a particle to be non-deterministic because it would never be isolated from the infinity of degrees of freedom of the radiation field.
Some work I have been doing lately is connected with your work on a joint probability distribution F (p q t) and has led me to think that there may be a limited region of validity for the use of a joint probability distribution. However, I have rather forgotten the details of your work and would be glad if you could let me see again the part of it dealing with F (p q t). I may get a more favourable opinion of it this time. Have you done any more work on it since our previous correspondence?
3, Sandy Rise,
March 22nd, 1945.
Unfortunately, my paper is in the hands of Professor Chapman of the Imperial College, and I only have the one typescript. However, I have sent your letter to him with a request that he should forward you the paper as soon as he has finished with it. On the other hand, I have just heard from M.S. Bartlett, that he is back at Queen’s; he is pretty familiar with my work, and I feel sure he will give you any explanation that you may require, if you care to get in touch with him, especially as he has worked out a new and improved method for obtaining the joint distributions.
I notice you have used Fock’s operators in your paper on 'Quantum Electrodynamics'. I have been wondering whether the work to which you refer in your letter is connected with this, as these operators imply in a way eigenfunctions in phase-space. I thought I could see a way of tying it up with my work when I was reading your paper, but I did not get very far with it.
I am afraid I have not done very much since I last wrote to you, as my engineering work is keeping me pretty busy. However, I have worked out the relativistic extension for scalar wave-functions, which leads to the wave equation
where s is the local time of the particle. This is a 'time dependent' extension of the Klein-Gordon equation; I do not know whether it has been considered before. There is a difference in the interpretation, however. I take s as the independent variable and the space time co-ordinates, and the momentum energy vector as random functions of s. The ordinary probability distribution which is then a scalar in space-time, is given as in the non-relativistic theory by ρ(qi,s)=ψψ*. The joint-phase space distribution for co-ordinate-time and momentum-energy is obtained in terms of ψ as in the non-relativistic theory, and gives in the same way for the space-time conditional means of the momentum-energy vector
This is normal for the current density, but connects the charge density with the space-time conditional mean value of the energy, rather than with probability, giving thus an immediate interpretation of the negative values obtained when the energy eigenvalues are negative.
I think this interpretation of probability as a scalar in space-time is perhaps more satisfactory than as the time-component of a ψ-vector, though there is a conceptual difficulty, since ρ must then be considered as variable with the local time of the particle. Another difficulty is connected with the relation
which would restrict the phase-space probability distribution to a 7-dimensional hyper surface. One way of turning this difficulty would be to consider m0 itself as a random variable, perhaps capable of taking a number of eigenvalues — but all this is purely speculative. I am not really clear about the last part.
In collaboration with M.S. Bartlett, I have also carried further the treatment of the harmonic oscillator in phase-space. Some of the results are rather reminiscent of those you obtain with the ξ-operator. This work is fairly complete, and I should be able to let you have a typescript of it shortly, if you are interested.
I have also been considering applications to statistical mechanics, which, since they require distributions in phase-space, would seem to offer an obvious field to the theory. But apart from equilibrium distributions, I rather hope that the application of the theory of random functions, will also lead to methods generally suitable for non-uniform states and fluctuation problems.
3, Sandy Rise,
March 23rd, 1945
Dear Professor Dirac,
My letter to Professor Chapman yesterday, crossed with his, returning my typescript which I therefore enclose.
1. General Validity
The practical issue here seems to be simply this:-
It also seems clear now that the analysis into eigenstates is a matter of mathematical technique. This is supported by:-
3. Discrete energy levels
The remarks under 1. are in sympathy with your view that here it is meaningless to ask whether the energy levels are really discrete, but to ask whether the theoretical spectra are correct. Incidentally one might note that while there is no objection to a conceptual discrete energy level existing over infinite time (as I pointed out in my reply to a previous comment of yours), it is true that in practice the observation of a spectra over a finite time implies a blurring of the lines. This is recognized and a theory has been worked out (see, for example the early chapters of Rosseland’s Theoretical Astrophysics). This observational fact may tend to obscure any finer points on the energy level distributions.
Similarly with the Stern-Gerlach effect — it is a matter for agreement with experiment — though here I shall not try to comment since I believe this effect involves electron spin, with which your theory does not deal.
4. Interference and diffraction
Similarly also with these phenomena. There is a word of caution here. When I looked at this a little while ago in an attempt to determine as precisely as possible from observe[d] results the form of the uncertainty principle, I satisfied myself that the interference of protons and electrons after passing through two narrow slits will not arise if the latter are merely passively filtering a statistical assembly of particles with an initial distribution of position and momentum; it is essential to allow the uncertainty principle to imply an actual change in the momentum possibility distribution consequent on the positional probability distribution at the slits.
Compare the discussion by Whittaker (Proc. Phys. Soc. 55, p. 464, 1933) of polarisation of Nicol prisms. He asserted that this phenomenon was impossible to explain by any what he called `crypto-deterministic’ mechanism, citing an alleged proof by von Neumann of this. But it was clear that he was referring to a deterministic behaviour of the protons without interaction with the prism; and this point has been taken up by Pelzer (Proc. Phys. Soc. 56, p. 195, 1944), who shows that with such interaction Whittaker’s assertion is not necessarily true.
This means, however, in connection with your suggestion of experimental verification with electron beams, that in successive measurements taken on a beam of photons or electrons, the effect of each measurement must be allowed for, and this will presumably affect the observed correlations at two instants of time.
The reference in the last paragraph of your letter to Dirac to nuclear theory was extremely interesting, though I think that a completely satisfactory extra-nuclear theory will not be possible either until radiation is satisfactorily incorporated. It is pointed out in the attached Notes* that irreversible changes appear excluded in the standard wave-vector technique (this is surprising in view of the common claim that the processes covered are non-deterministic). There is presumably the possibility, however, as apparently envisaged in your treatment of the electromagnetic field, of introducing irreversible changes in the well-known statistical way from reversible ones by averaging over a large number of irrelevant degrees of freedom after the complete equations have been set up.
Thanks for sending me your manuscript again. The situation with regard to join[t] probability distributions is as follows, as I understand it.
A joint distribution function F(p,q) should enable one to calculate the mean value of any function f(p,q) in accordance with the formula
I think it is obvious that there cannot be any distribution function F(p,q) which would give correctly the mean value of any f(p,q), since formula (1) would always give the same mean value for pq and for qp and we want their means to differ by iħ. However one can set up a d.f. F(p,q) which gives the correct means values for a certain class of functions f(p,q). The d.f. that you propose gives the correct mean value for , for τ and θ any numbers, but would not give the correct mean value for other quantities, e.g. it would give the same mean value for , whereas we want this second quantity to be times the first. In some work of my own I was led to consider a d.f. which gives correctly the mean value of any quantities of the form , i.e. all the p’s to the left of all the q’s in every product. My d.f. is not a real number in general, so it is worse than yours, which is real but not always positive, but mine is connected with a general theory of functions of non-commuting observables.
I am writing up my work for publication and I propose to refer to your work somewhat in these terms:-
'The possibility of setting up a probability for non-commuting observables in quantum mechanics to have specified values has been previously considered by J.E. Moyal, who obtained a probability for a coordinate q and a momentum p at any time to have specified values, which probability gives correctly the averages of any quantity of the form , where τ and θ are real numbers. Moyal’s probability is always real, though not always positive, and in this respect is more physical than the probability of the present paper, but its region of applicability is rather restricted and it does not seem to be connected with a general theory of functions like the present one.'
Do you think this reference would correctly describe your work and do you have any objection to such a reference?
There may be other d.f.’s which are worth considering and there is a field of research open here. Will you be able to work on it?
18 Ambrose Avenue
London N.W. 11.
April 29th, 1945.
Many thanks for your letter. I was most interested by your remarks concerning your work on a general theory of functions of non-commuting observables, and should be very glad to see it. Are you acquainted with the work of Whittaker, and Kermack and McCrea on this subject? The references are: E.T. Whittaker, Proc. Ed. Math. Soc. Ser. 2, v. 2 (1931) 189–204; W.O. Kermack and W.H. McCrea, ibid. ser. 2, v. 2. (1931), 205–219 and 220–239.
If I understand correctly your remarks concerning joint probability distributions, you consider them as functions of the non-commuting variables P, Q, which will give correct averages for certain classes of functions of the latter. (I shall use hereafter P, Q for the non-commuting quantities, and p, q, for the corresponding commuting variables.) Such functions may of course prove extremely useful mathematically, but they can hardly be called probability distributions in any ordinary sense.
My approach to this problem has been entirely different. I have looked for a probability distribution in the ordinary sense, which will be a function of the ordinary, commuting variables p, q. Its connection with functions of the corresponding non-commuting operators P, Q of quantum mechanics, is that it should give correct means for such of these functions (i.e. Hermitian operators) as are formed to represent physical quantities. If a physical quantity is given in classical mechanics by a function M(p,q), (i.e. a Hamiltonian, or an angular momentum) a Hermitian operator M(P,Q) is formed to represent it according to certain rules. I have looked for an F(p,q) such that it will always give
It is obvious that such a function F(p,q) should be connected with a unique method of forming the quantum mechanics operators from the corresponding classical mechanics functions if p and q (I am speaking of course, of the classical quantum mechanics for particles without spin). A first test for the correctness of such an F(p,q), will therefore be that the corresponding method for forming operators should give correctly at least all the known Hermitian operators of the theory, (since a general method for forming these operators is not generally agreed upon in the standard theory).
The F(p,q) which I propose in my paper fulfils these conditions. It can be expressed either as a series development in ψ(p) and φ(p) or as an integral expression in terms of the ψ’s alone or φ’s alone (the latter is due to M.S. Bartlett) as follows
I have shown that it corresponds univocally to the following method of forming operators (already proposed by McCrea). Let M(p,q) be an ordinary function of p and q (e.g. some constant of the motion in classical mechanics). To form the corresponding operator M(P,Q) we write first a function Mp(P,Q) of the non-commuting operators P, Q, which is obtained from M(p,q) by placing all the P’s to the right of the Q’s, i.e. by replacing all polynomial terms qmpk in M(p,q) by QmPk. The correct operator M(P,Q)is then obtained as
Form (2) for F(p,q) will give correct averages for all operators formed as in (3) by averaging in p-q space over the corresponding ordinary function M(p,q), i.e.
It is consequently incorrect in my view to say that the F(p,q) in my paper will give correct averages only for functions of the form . Actually, it will give the right averages for all operators formed as in (3), and in particular, for all the Hermitian operators considered in the classical quantum mechanics of particles without spin, e.g. Hamiltonian, angular momentum, total angular momentum, radial momentum, etc. It is easy to check that (3) does give the usual operator form for all these quantities. In the case of the example quoted in your letter, it will give correct average for . (I may mention here that this form of F(p,q) and method of forming operators is valid for rectilinear coordinates only.)
Furthermore, the F(p,q) in my paper leads to certain forms for the space-conditional averages of the powers of p (i.e., averages of pmfor a given value of q), the first two being
Early in my work (Sect. II) I obtained a set of partial differential equations for probability distributions, which have the form of the hydrodynamic equations of continuity and motion and express conservation of probability. These are of quite general validity, and are not connected with any special form of F(p,q) or any physical assumption. Substitution in these general equations of the expressions above for the space-conditional means of p, p2, taken in conjunction with the equations of classical mechanics, lead to the Schrödinger equation, as I have shown in my paper. The Schrödinger equation is thus shown to result from this special form for F(p,q), the laws of classical mechanics, and the general properties of probability distributions for dynamical variables. I think this is the other essential condition for a correct F(p,q): that it should be consistent both with the Schrödinger equation and the equations for conservation of probability.
Regarding the range of validity of form (2) for F(p,q), and the fact that it leads to negative values for single eigenstates, I have already mentioned in my last letter that this may possibly mean, reverting to your view, that joint measurement of p and q is inconceivable in pure states, but only in a combination of states that leads to a defined positive F(p,q). I think possibly this may be a general feature for any possible F(p,q) in quantum mechanics, because of the necessary orthogonality properties of the phase-space eigenfunctions corresponding to pure states. Such (possibly) negative eigenfunctions, which must be compounded to give a positive probability function, occur in the classical calculus of probabilities in the theory of chain probabilities. However, as was pointed out by M.S. Bartlett, even the possibly negative f(p,q) corresponding to a pure state will still lead to correct averages for operators of form (3), so that the theory retains its usefulness even in this connection. I pointed out in my last letter for example, how it could be used to calculate transition probabilities.
In conclusion, my view is that this form (2) of F(p,q) has quite general validity, and that the theory it leads to, is entirely equivalent to the classical quantum mechanics of particles without spin.
I have considered the connection of this theory with the general theory of functions of non-commuting variables. From this point of view, the theory starts with , and leads to the general method (3) for forming observables. One might conceivably take another starting point, which would be connected with some other method for forming observables. However, apart from other considerations (cf. Hermann Weyl, “Theory of Groups and Quantum Mechanics” p. 275) all the other forms of F(p,q) I tried, taken in conjunction with classical mechanics and the equations of conservation of probability, did not lead to the Schrödinger equation, but to some different wave equation. They correspond thus to some scheme different from the classical quantum mechanics. In particular I discarded for this reason the first F(p,q) I tried, which was connected with the general operator form
which gave the exponential form and consequently had the form
I believe I showed you these attempts in 1940.
One of the problems in the theory of non-commuting variables, which I have not been able to solve is: what general transformation will leave form (2) for F(p,q) and (3) for operators invariant? It is easy to see that this is the case for linear transformations from Cartesian coordinates, and also for the dynamical-contact transformation of classical mechanics; but it is not maintained e.g. for a transformation to polar coordinates and their conjugate momenta. An allied problem is to find a general form for F(p,q) for any canonical coordinates corresponding to form (2) for rectilinear coordinates. I am hoping your work will give me a lead in this connection.
With regards to your query, I do not, for the reasons mentioned above, think that your reference to my work gives a correct description of it. It is certainly not correct in my view to say that form (2) for F(p,q) is limited to giving correctly averages for quantities of the form ; in fact, it will give averages for all observables formed as in (3), and this includes as far as I know, all the observables ordinarily considered in classical quantum theory. This would not perhaps matter a great deal, if my work was already published, since readers could then refer to the original. I have not however been able so far to arrange for its publication, due largely, as you will no doubt remember, to your veto, which made the late Professor Fowler hesitate about presenting it to the Royal Society. Your criticism is thus left without an answer. Your objection at the time, if I remember rightly, was chiefly that joint distributions for p and q had no physical meaning and consequently no validity or usefulness. I am glad to notice that you now think they open an interesting field of research.
Regarding your query to whether I shall be able to do further work on this subject, my main difficulty is again the fact that my existing work is not yet published. For one thing, I shall want to base future work, at least partly, on the papers now in your hands. It is also discouraging to accumulate for years unpublished results, as I have been doing. Finally, there are material difficulties: the papers you have seen, represent my first real effort at research in pure mathematics and theoretical physics; I was hoping that their publication would eventually enable me to transfer my activities entirely from the field of research in engineering and applied physics to that of pure science, and to do some serious work on theoretical physics. Failure to obtain publication has forced me to adjourn such plans sine die, and my present work is leaving me less and less time for pure research.
c/o. Goscote Hotel,
Goscote Hall Road,
April 25th, 1945.
There are a few points in the paper I sent you which I should like to amplify.
First, regarding the range of validity of the F(p,q) distributions, I have been considering the possibility of a modified interpretation of the mathematical formalism. You will have noticed that one of the difficulties of the theory is that the method of forming F(p,q) does not lead to functions that are defined positive for all p and q when applied to a system in a single eigenstate. This might be interpreted, reverting partly to the point of view expressed in your book, as indicating that simultaneous probability distributions for p and q have no precise meaning for a system in a single eigenstate, or again, that a classical particle picture is not valid for a system in a pure state, and that the hypothesis of pure state is incompatible with the simultaneous measurement of p and q. The classical particle amongst a number of states in such a manner is to make F(p,q) positive.
This would limit the possibility of giving the probabilities of simultaneous values for p and q. However, as M.S. Bartlett points out in his paper, it does not necessarily upset the mathematical structure of the theory or its equivalence to classical wave mechanics. If, as I think, this equivalence is correct, then the theory should lead to correct results for the various quantities obtained by wave mechanics, such as frequencies and transition probabilities, even when dealing with negative functions F(p,q). The appearance of the latter should then be taken to mean that the situation is such that simultaneous prediction of the values of p and q is impossible, but would not impair the calculation of other experimentally determinable quantities.
It would be possible to use the formalism of this theory to supplement in certain cases, the perturbation method in the calculation of transition coefficients. This can be done as follows: if the system is originally in the unperturbed eigenstate k, with the phase space eigenfunction fkk(p0,q0)corresponding to the q-space eigenfunction uk(p0,q0)
the phase space distribution F(p,q,t) at time t would be obtained by substituting in fkk(p0,q0) the classical solution p(t), q(t), in terms of the initial values p0 , q0, for the system under the action of the perturbing forces (when it is possible to find such solutions). In other words, one would apply to fkk(p0,q0) the contact transformation in time of classical mechanics to obtain F(p,q,t) at time t; one could then expand the latter in terms of unperturbed phase space eigenfunction
and obtain thus directly the transition coefficients a*kn(t)akn(t)=Akn from state k to state n.
Applied, for example, to the schematic case of an oscillator of change e, following the application of a perturbing electric force of large wavelength, this method leads for the transition coefficient from the ground state to the k-th state to the exact expression
(calling ΔE the increase in mean energy). The first term of the expansion of A0kin power of ΔE coincides then with the first approximation by the perturbation method
I have been considering the application of this method to radiation oscillators, in view of the possibility that some of the divergences may be due to a mathematical breakdown of the perturbation method.
P.S. I have just received your letter but must defer answering for a few days, as I am moving to London. My new address will be 18. Ambrose Ave. N.W. 11.
Thanks for your letter and your references to Whittaker and others. These papers are very interesting, though not directly connected with the subject under discussion.
I still do not agree that your d.f. gives correctly the average values of all Hermitian operators considered in classical mechanics. It is true that it works alright for , but it goes wrong as soon as one applies it to more complicated examples. For example your d.f. would give the same average for the two Hermitian operators QP2Qand PQ2P, whereas they ought to differ by 2ħ2. You may answer that these two Hermitian operators do not correspond to classical quantities. To anticipate this answer I have worked out another example, which certainly is of practical importance. Take a harmonic oscillator of energy . Its average energy when it is at a temperature T is the average value of the Hermitian operator . I have checked that this average value is not given correctly by your d.f. Your d.f. gives the correct average for quantities of the form and for quantities expressible linearly in terms of such quantities, e.g. for any f(a,b) or , but is not more general than this. Do you not agree?
I have enclosed a copy of my paper. I should be glad if you would send it back in two or three weeks time, as I do not have another copy.
Do you want me to send you back your work now? I would be willing to help you publish it if you would change it so that it does not contain any general statements which I think to be wrong. I would suggest it would be better to publish the quantum theory part separately from the rest, because it is on rather a different footing (according to my view).
Your theory gives correctly the average energy when the system is in a given state, (i.e. represented by a given wave function) but not when the system is at a given temperature. Take a harmonic oscillator with energy . The probability of its being in the n-th state is proportional to the average value An of . According to your theory
When the An’s have been calculated, we can get the average energy by
It is not very easy to calculate An, but is quite easy to calculate from the known property of wave functions
We now get
which is the classical result and not the quantum one.
In Bartlett’s paper which you just sent me, the quantum values for the energy of the harmonic oscillator are assumed and the correct value for was obtained because of this assumption. You can always get the right answer by borrowing sufficient results from the ordinary quantum theory. The true test of a theory is whether it always gives consistent results whichever way it is applied, and my way of evaluating given above shows that your theory does not always given consistent results. The discrepancy in this case arises because I use your d.f. for calculating the average of , and this quantity is not expressible linearly in terms of .
You say your theory gives a different value for , and this can only mean that your theory is not consistent with the usual quantum values for the energy, otherwise there is no room for any uncertainty in the value of . Your theory gives a value for greater than the usual one by an amount > (with ). Thus for a harmonic oscillator in its state of lowest energy your theory will give fluctuations in energy corresponding to , instead of a constant energy. Surely you must agree that your theory is wrong in this case, and that therefore it has limitations.
The general statement in your work that I disagree with is the one (given in your last letter) that dynamical variables must be of the form . The square of the energy of a harmonic oscillator, namely is not of this form, and if you replace it by something that is of this form you get energy fluctuations in the state of lowest energy, which I this is a self-contradiction.
18 Ambrose Avenue
May 15th, 1945.
Many thanks for your letter and enclosed paper. I have not yet had time to read the latter, but I shall do so as soon as possible.
I am not quite clear as to how you worked out the average energy for an oscillator at temperature T. The theory in my paper gives correctly the average energy for a Maxwell-Boltzmann assembly of N oscillators. I enclose the draft of an unfinished paper by M.S. Bartlett and myself which gives the relevant calculations in §4 (you may also find §2 & §3 of some interest). A difference with the orthodox method is found not in the expression for the average energy , but in the standard deviation, which comes out as instead of (not neglecting the ground state energy). I have always found so far that my treatment leads to the same average values as the usual methods, but shows difference in the fluctuations: this may lead to an experimental test of the theory.
I agree that my d.f. yields correct averages for quantities expressible linearly in terms of expressions such as
but this includes quite a wide class of functions. In fact, it can be shown (c.f. McCoy, Proc. Mat. Acad. Sc., 18 (1932) 634) that (1) is equivalent to the form for Hermitian operators mentioned in my last letter.
For a polynomial term p2q2 the corresponding operator (P2Qs)0 obtained by (1) or (2) can be cast in a more symmetrical form
In particular, for the term (P2Q2)0 mentioned in your letter, (2) and (3) lead to
(by the way, surely QP2Q–PQ2P=0 !).
The hypothesis on which I base my derivation of the d.f. (and therefore the rest of the theory) is equivalent to the assumption in the standard (matrix) theory that dynamical observables must be of the form (1) (non-dynamical operators might be construed in the statistical theory as symmetry, etc. conditions on the d.f.). Relation (1) is obviously more restrictive than Heisenberg’s exchange relations alone: it might be considered as the basic postulate of a well-defined form of quantum kinematics. In this form, it has been given by H. Weyl, who bases his arguments in its favour on group-theoretical considerations: iP, iQ generate a unitary Abelian group in 'ray'-space; the hypothesis is then that dynamical observables are the matrices of the representation of this group’s algebra, which are given by (1) if the group is supposed irreducible. My argument is, that it leads to a theory that is consistent both with the Schrödinger equation and the usual statistical interpretation. I think it should be possible to prove that it is the only form of quantum kinematics that does so, and that a different form would necessitate revising either the statistical interpretation, or the wave-equation — but this is only a conjecture so far.
Summarizing, I think it would be fair to say that my paper gives a derivation of classical quantum mechanics on a purely statistical basis, (plus Newtonian mechanics) which is equivalent to the standard matrix theory with the addition of Weyl’s postulate for a quantum kinetics and furthermore that it shows the consequences such a theory entails with regards to the problems of determinism, probability distributions, fluctuations, quantum statistics, etc. Would you agree to this character; and the controversial issue it raises? I am not clear, however, as to exactly what general statements you think are wrong.
I shall not need my typescript until there is a need of revising it for publication, so that you can return it whenever you have finished with the problems of determinism, fluctuations, quantum statistics, etc. Would you agree to this statement of the position?
I thank you for your (conditional) offer to help me publish my papers. I have no objection to publishing the quantum theory part separately; I agree, it is on a different footing from the rest, because of its more tentative character; and the controversial issue it raises. I am not clear, however, as to exactly what general statements you think are wrong.
I shall not need my typescript until there is a need of revising it for publication, so that you can return it whenever you have finished with it.
18 Ambrose Avenue
May 26th, 1945.
I thank you for your letter of the 18th. With regards to your derivation of the average energy for an oscillator at fixed temperature, I don’t know how this method works out in the standard theory, but the reason for the result you obtained on the basis of my theory is fairly obvious. You start with a Maxwell d.f. for p and q
You then work out the coefficient
Since the fnm(p,q)
form an orthogonal set in phase-space, the coefficient An is merely the Fourier coefficient ann in the expansion
(It is possible to show that in (4) ann=0 for n≠m). You then proceed to show through the An that for (1)
but this is of course obvious by a direct calculation
I don’t think your remark on getting the right answer 'by borrowing sufficient results from the ordinary quantum theory' quite fair: in so far as my theory is equivalent to the ordinary theory, it leads to the same eigenvalues for the mean of the energy, as I have shown in my paper. In order to prove an inherent inconsistency in my theory one would have to show that the method you use follows necessarily from my basic postulates, but this is not the case. My method on the other hand is based on a theory for statistical assemblies resulting from these postulates (c.f. my paper, §10). As such, it is quite consistent with the rest of the theory, and also appears to lead to correct results.
The difficulty regarding the dispersion for the energy of the oscillator in a single eigenstate is more serious. I think it is connected with the fact that f(p,q) can be negative: if the conclusion is (in accordance with your views) that a joint d.f. for p and q is impossible in a single eigenstate, then the probability distribution for , and consequently the dispersion, have no direct physical meaning. This could be interpreted through the fact that it is impossible to measure the energy in a single eigenstate in a finite time. Only a d.f. giving the band-width and intensity distribution of the spectrum lines would have a physical meaning, and could be compared with experiment. This would involve, however, extending the theory to include radiation.
I am prepared to mention your objections concerning the operator forms in the body of my paper (do you agree that with the imposition of this restriction on operators for dynamical variables in the usual matrix theory, the latter becomes equivalent with my theory?)
I do not think there are any inherent inconsistencies in my theory, but I agree that this restriction leads to results that do not tally with certain hitherto accepted features of the usual theory, and may possibly clash with experimental results. Should the latter prove to be the case, then in my view the conclusion to be drawn from my work would be, that the usual statistical interpretation of classical quantum mechanics must be revised. Comparison with the experiment of such differences with the usual theory might perhaps be sought for in the fluctuations for statistical assemblies, the intensity distributions of spectral lines, or the calculation of transition probabilities.
If you agree to the above, then I should be glad to know if you are still prepared to help me in publishing my work and what form of publication you would suggest. I think I could condense the mathematical part into a paper in two parts of 15–20 pages each, and the quantum mechanics part into 20–25 pages.
I return your typescript, which I read with great interest, especially as I have treated the same subjects in my paper and arrived at different conclusions. For example, the operator form I use constitutes a general method for forming functions of observables which (as compared with yours) is unambiguous when the latter are non-commuting, and does not depend on their order. We have already discussed the d.f. for p and q at one instant of time, but I have also given an expression for their distribution at two instants of time, in terms of the phase-space eigenfunctions in my main paper (§14), and in terms of the transformation function in §2 of the paper on the oscillator I sent you, which it is interesting to compare with your results on the same subject. My conclusion regarding trajectories in my theory is that for a conservative and unperturbed system they reduce to those of classical mechanics, I discussed the resulting implications with regards to the principle of uncertainty and the problem of determinism in §15, and showed in the succeeding paragraphs, that it leads to correct results in examples on the free and uniformly accelerated particle, and the oscillator. I have also worked out in collaboration with Bartlett an alternative method of calculating from Hamilton’s principal function in classical mechanics based on Whittaker’s work.
I expect to be going abroad in a few days time and not to be back till the end of July, so I am returning your papers herewith in case you should need them in the meantime. Thanks for returning my paper.
It now appears that the dispersion of the energy in a stationary state is the simplest example which shows the limitations of your theory. This dispersion will be pretty general on your theory, and will probably occur with all stationary states and all dynamical systems. This is not a difficulty that can be got around in any way, because it contradicts the whole idea of sharp energy levels — it would imply a lack of sharpness in the energy levels much too great to be reconciled with experimental evidence. It shows therefore that the joint d.f. does not work in the case of E2. Also it does not work for higher powers of E.
If the limitations in the applicability of the joint d.f. are clear[ly] stated, which would mean partly rewriting it, I would be glad to help you publish your work. The quantum theory part of your work could form a paper which I could communicate to a scientific journal. With regard to the remainder, I do not know how much of it represents new research work and how much is an exposition of known results. Do you have any suggestion about where it should be published? What did Fowler say about it?
18 Ambrose Avenue,
LONDON N.W. 11,
17th June, 1945
I was sorry to see in the press that your visit to the U.S.S.R. was cancelled at the last moment: I expect you must be very annoyed at the whole incident.
Your letter and my papers reached me only on Tuesday: the delay was apparently due to the fact that the envelope had broken open during transit; fortunately nothing seems to be missing.
I agree that the occurrence of non-zero dispersions in eigenstates is the main difficulty or limitation in my theory. I did point it out and discuss it at some length in the paper I sent you, and will of course do so again as clearly as I can when I redraft it for publication (which I intend to do in any case in order to produce a condensed version.)
My work on Random Functions is new. Professor Fowler’s original suggestion was to present the whole work for publication in the Proc. Roy. Soc. (including the part on Quantum Mechanics) as three separate papers. My intention was then to rewrite it in a more condensed form, cutting out appendices, some of the examples, etc., so as to have three papers of 15 to 20 pages each. Would you consider this now as a suitable arrangement?
Bartlett has told me that you are holding colloquiums on Quantum Mechanics in Cambridge. Would it be possible for me to attend some of these? I shall be visiting Cambridge fairly regularly in connexion with my present duties, and it may prove possible to arrange for these visits to coincide with the date of your colloquium.
7 Cavendish Avenue,
The quantum theory part of your work could be written up as one paper, and the remainder as two more, provided it divides naturally into two parts. If it does not divide it might be better to keep it as one long paper. Probably the Proc. Roy. Soc. is the best journal for them.
We have been having Colloquiums, usually on Friday afternoons but sometimes on Monday afternoons. They will probably be resumed in October and we would be glad if you could come to any of them.
18 Ambrose Avenue,
London N.W. 11
10th July, 1945.
Many thanks for your letter of the 26th. As you suggest, I am now rewriting the part of my work on quantum mechanics as a separate paper. As regards the rest, I am rewriting it as a paper in two parts, which could then appear either separately or together, whichever is more convenient.
Thank you for your invitation to the colloquiums; I am looking forward to attending them.
I enclose some notes in which I have tried to develop a method which would overcome the difficulty about non-zero-dispersions for eigenvalues in my theory and also extend it to generalized canonical coordinates. This is still tentative in character, and there are several things I still want to clear up, but I should be glad in the meantime to have your opinion on this development. I also enclose some notes comparing the results in your paper with mine.*
18 Ambrose Avenue,
London N.W. 11,
21st August 
With my best wishes for a pleasant journey.
17 Cavendish Avenue,
Your new version is more in accordance with the standard quantum mechanics, but it is considerably more complicated as you need a different joint prob. distr. for each system of coordinates. You are definitely departing from classical statistics when you make the joint prob. distr. depend on the system of coordinates, and if you depart so much from the usual classical ideas is there any point in trying to fit things into a classical framework? What advantages does your system have over the usual statistical interpretation of quantum mechanics? Any results that you get from your system must either conform to the usual quantum mechanics or else be incorrect. I think your kind of work would be valuable only if you can put it in a very neat form.
I am returning your paper herewith,
P.S. The Colloquium time has been changed to Friday 3 pm.
I heard from Bartlett that you would be wiling to talk about your quantum theory work at our colloquium, and I think it would be a good idea to have it discussed if you don’t mind possible heavy criticism. Would Friday the 25th Jan at 3 pm suit you? If this does not leave you sufficient time we could make it a week later. If you cannot conveniently deal with it in one afternoon there is no objection to your carrying on the following week.
7, Cavendish Avenue,
Many thanks for your letter of the 9th and your invitation to speak at your colloquium. I have now been able to arrange to be free on two successive Fridays, i.e. the 25th January and the 1st of February. I shall be able to take advantage of your offer to speak at two successive colloquiums as I think this will be necessary if there is going to be a long discussion of the paper.