Large Systems and Entropy

For two Einstein solids with a few hundred oscillators and energy units, the multiplicity is peaked, but perhaps 20% of the macrostates are still fairly probable. With a few thousand, the peak is already too sharp to draw, and real systems have more than \(10^{20}\) particles. To quantify this we need the mathematics of very large numbers.

There are small numbers like 23, large numbers like \(10^{23}\) (the order of Avogadro’s number), and very large numbers like \(10^{10^{23}}\), each the exponential of the one before. Adding a small number to a large one doesn’t change it, \(10^{23} + 23 \approx 10^{23}\) (unless you later subtract the same large number off), and multiplying a very large number by a large one doesn’t change it either:

The common trick is to take the logarithm, which turns a very large number into an ordinary large one, and exponentiate at the end. With \(e^x = y \Leftrightarrow x = \ln y\), \(\ln(ab) = \ln a + \ln b\), and for \(|x| \ll 1\), \(\ln(1+x) \approx x\) (for example \(\ln 1.1 = 0.0953\ldots\)).

Multiplicities involve factorials, and \(N! = N(N-1)\cdots 2\cdot 1\) gets too large fast. Replacing every factor by \(N\) gives \(N^N\), a gross overestimate; on average each factor is effectively smaller by a factor of \(e\), so \(N! \approx (N/e)^N = N^Ne^{-N}\). A further correction gives

for \(N \gg 1\); if all we care about is the logarithm, the second is usually good enough.

Apply Stirling to the Einstein solid, \(\Omega(N,q) \approx (q+N)!/q!\,N!\) for large \(N\). In the high-temperature limit \(q \gg N\), using \(\ln(q+N) \approx \ln q + N/q\),

(and \(\Omega \approx (eN/q)^q\) for \(q \ll N\), since \(\ln\Omega\) is symmetric in \(q\) and \(N\)). Now take two such solids with \(N\) oscillators each, sharing \(q = q_A + q_B\): \(\Omega = (e/N)^{2N}(q_Aq_B)^N\), peaked by symmetry at \(q_A = q/2\). Write \(q_A = q/2 + x\) and \(q_B = q/2 - x\); with \(\ln(1 - \epsilon) \approx -\epsilon\),

a Gaussian, which falls to \(1/e\) of its maximum at \(|x| = q/2\sqrt{N}\). (This is also an instance of the central limit theorem, and of the de Moivre–Laplace theorem, by which the binomial distribution approaches a Gaussian.) Slide \(N\) up to see the peak against the full range \(0 \le q_A \le q\):

For \(N = 10^{20}\), the width is one part in ten billion of the axis: drawn 1 cm wide, the axis would be \(10^5\) km long, more than twice around the Earth. Ten widths away from the peak, the multiplicity is down by \(e^{-100} \approx 10^{-44}\). Random fluctuations away from the most likely macrostate are utterly unmeasurable: the thermodynamic limit. In a well-known quip, it all works because Avogadro’s number is closer to infinity than to 10.

How many microstates does a monatomic ideal gas have with energy \(U\) in volume \(V\)? For one atom, the multiplicity is proportional to the volume of ordinary space (doubling the container doubles it) and to the area of the surface \(p_x^2 + p_y^2 + p_z^2 = 2mU\) in momentum space, a sphere of radius \(\sqrt{2mU}\). To count states we need a bit of quantum mechanics: by the uncertainty principle \(\Delta x\,\Delta p_x \approx h\), so a one-dimensional box of length \(L\) with momentum range \(L_p\) has \(LL_p/h\) distinct states, and in three dimensions \(\Omega_1 = VV_p/h^3\).

For two atoms, the constraint \(p_{1x}^2 + \cdots + p_{2z}^2 = 2mU\) is the surface of a sphere in six dimensions, and since the atoms are indistinguishable, swapping them gives the same state: divide by \(2!\). For \(N\) atoms, using the area of a \(d\)-dimensional sphere, \(2\pi^{d/2}r^{d-1}/(d/2 - 1)!\), with \(d = 3N\):

The exponent \(3N/2\) is half the number of degrees of freedom, just as \(\Omega \approx (eq/N)^N\) for the Einstein solid with its \(2N\). In general, a system with only quadratic degrees of freedom and enough energy that quantization is unnoticeable has \(\Omega \propto U^{Nf/2}\).

Two samples of the same gas, each with \(N\) molecules, separated by a movable partition that also passes energy, have \(\Omega_{\text{total}} = [f(N)]^2(V_AV_B)^N(U_AU_B)^{3N/2}\). This is the same form as for two Einstein solids, so the peak is a Gaussian in each direction, with widths \(U_{\text{total}}/\sqrt{3N/2}\) and \(V_{\text{total}}/\sqrt{N}\): very sharp in both \(U_A\) and \(V_A\).

If molecules can move between the halves too, a sharp peak fixes the density, \(N_A/V_A = N_B/V_B\). The chance of finding every molecule in the left half is the multiplicity with \(V \to V/2\), \(\Omega/2^N\), over essentially \(\Omega\): a probability of \(1/2^N\). Below, molecules fly freely around a box; the program counts how often they are all on the left.

For all these systems, particles and energy tend to rearrange themselves until the multiplicity is at, or very near, its maximum. Any large system in equilibrium will be found in the macrostate with the greatest multiplicity, aside from fluctuations normally too small to measure. Multiplicity tends to increase: a more general statement of the second law of thermodynamics. Since multiplicities are very large numbers, it is convenient to work with their logarithm, and for historical reasons to multiply by Boltzmann’s constant:

the entropy, an ordinary large number with units of energy per temperature (J/K). For a large Einstein solid with \(q \gg N\), \(S = Nk_{\mathrm{B}}[\ln(q/N) + 1]\); with \(N = 10^{22}\) and \(q = 10^{24}\), \(S \approx 0.77\ \mathrm{J/K}\). More particles, more energy, more room, breaking molecules apart, or mixing substances all increase the number of possible arrangements, and so the entropy: intuitively, a measure of molecular-level disorder (though crushed ice has much less entropy than the same amount of water). Since \(\ln(ab) = \ln a + \ln b\), the entropy of a composite system is the sum of its parts’, \(S_{\text{total}} = S_A + S_B\), an extensive quantity. And because the logarithm increases monotonically, entropy tends to increase.

Could a tiny “demon” that lets fast molecules through a trapdoor one way and slow ones the other spontaneously decrease entropy? Maxwell’s demon has to be operated, which takes energy, and the overall entropy still increases.

Taking the logarithm of \(\Omega_N\) with Stirling’s approximation gives the Sackur–Tetrode equation:

At fixed \(U\) and \(N\), \(S\) depends on volume only through \(Nk_{\mathrm{B}}\ln V\), so expanding from \(V_i\) to \(V_f\) gives \(\Delta S = Nk_{\mathrm{B}}\ln(V_f/V_i)\). In a quasistatic isothermal expansion, heat \(Q = Nk_{\mathrm{B}}T\ln(V_f/V_i)\) flows in, so \(\Delta S = Q/T\), valid for any quasistatic process. In a free expansion into vacuum, there is no work and no heat (\(\Delta U = 0\)), yet the same \(\Delta S\) appears: \(\Delta S = Q/T\) fails because free expansion is not quasistatic. Remove a partition between two different gases and each expands into the whole chamber, giving the entropy of mixing, \(\Delta S = 2Nk_{\mathrm{B}}\ln 2\). But if both sides hold the same gas, nothing really mixes, and \(\Delta S\) must be zero. The key is distinguishability, carried by the \(1/N!\) in \(\Omega_N\). Try each case:

Without the \(N!\), the entropy would be \(Nk_{\mathrm{B}}[\ln(V(4\pi mU/3Nh^2)^{3/2}) + \tfrac32]\), and simply sliding a partition into a box of one gas would lower it by \(Nk_{\mathrm{B}}\ln 2\), violating the second law. This is the Gibbs paradox, resolved by the indistinguishability of molecules. Processes that create no new entropy (of the universe), such as quasistatic ones, are reversible; those that create entropy, such as free expansion or any heat flow between objects at different temperatures, are irreversible, though a very slow heat flow approaches the reversible limit. More on this in the next chapter.