An interactive demo created by Claude Opus 5.5, based on the lecture notes by Sang Hoon Lee (Statistical Physics 1, Chapter 2, The Second Law, Sections 2.1 to 2.3)
Why does heat flow from hot to cold, and never the other way? Not because it must, but because it is overwhelmingly probable. To see why, learn to count all the ways a system’s energy can be arranged.
Many thermodynamic processes happen in one direction but never in reverse: not inevitably, but because the reverse is overwhelmingly improbable. To make that precise we need probability and statistics, and the art of counting, combinatorics. Start by flipping three different coins: a penny, a nickel, and a dime.
Each of the 8 possible outcomes is a microstate. Classify them by something coarser, such as how many heads came up, and each class is a macrostate. The number of microstates in a macrostate is its multiplicity, \(\Omega\). If the coins are fair, every microstate is equally probable, so
Flip 100 coins and there are \(2^{100} \approx 10^{30}\) microstates but only 101 macrostates (0 to 100 heads). One head can be in any of 100 places, so \(\Omega(1) = 100\); a pair of heads in \(100 \cdot 99/2\) ways, since the pair can come in either order; in general, for \(N\) coins,
read “\(N\) choose \(n\).” Flip the coins below, over and over, and watch the counts pile up where the multiplicity is largest.
Why flip coins in physics? Because the same mathematics describes a two-state paramagnet: \(N\) dipoles, each pointing up or down, that act like tiny compass needles tending to align with an external magnetic field (switch the view to “Dipoles”). With \(N_\uparrow\) up and \(N_\downarrow = N - N_\uparrow\) down, \(\Omega(N_\uparrow) = \binom{N}{N_\uparrow}\). In a field, an up dipole has less energy than a down one, so (neglecting interactions between dipoles) the total energy is fixed by \(N_\uparrow\): specifying the macrostate is the same as specifying the total energy, as it is, at least in part, for nearly all physical systems.
Unlike fair coins, the field breaks the symmetry between up and down, so the microstates are no longer all equally probable, and the probability of \(N_\uparrow\) is not simply \(\Omega(N_\uparrow)/\Omega(\text{all})\). That case is taken up in Section 3.3.
A quantum harmonic oscillator with potential energy \(\tfrac12 k_s x^2\) has energy levels \(E_n = (n + \tfrac12)\hbar\omega\). Using \(\hbar\omega = hf\) and ignoring the zero-point energy, the allowed energies are \(0, hf, 2hf, \ldots\): any number of energy “units,” all the same size. Atoms in a solid oscillate like this in three independent directions, so \(N\) oscillators describe \(N/3\) atoms. Modeling a solid as a collection of identical quantum oscillators was first proposed by Albert Einstein in 1907: an Einstein solid.
With \(q\) units of energy among \(N\) oscillators, how many microstates are there? Represent each unit as a dot and separate the oscillators with \(N-1\) bars. Every arrangement of \(q\) dots and \(N-1\) bars is a microstate, so
Each dot is one unit of energy; each bar separates one oscillator from the next.
For a very small Einstein solid, \(N = 3\), the microstates can be listed: \(\Omega(0) = 1\), \(\Omega(1) = 3\), \(\Omega(2) = 6\), \(\Omega(3) = 10\).
Is any one particular microstate special? No. Picking 1, 2, 3, 4, 5, 6 as lottery numbers seems absurdly unlikely to come up, but it is exactly as likely as any other set of numbers you might choose. Every microstate is equally improbable; what differs is how many of them make up each macrostate.
Now let two Einstein solids, A and B, share energy, “weakly” coupled so the exchange between them is much slower than within each. Over short times \(U_A\) and \(U_B\) are essentially fixed, and the macrostate of the composite system is specified by them. Only the total \(q_{\text{total}} = q_A + q_B\) is fixed overall. For each \(q_A\), solid B has \(q_B = q_{\text{total}} - q_A\), and since the two solids are independent,
Assume that over long times the energy gets passed around randomly, so that every microstate is equally probable. That is the fundamental assumption of statistical mechanics: in an isolated system in thermal equilibrium, all accessible microstates are equally probable. Macrostates are not: the most likely one is the one with the most microstates.
With 3 + 3 oscillators and 6 units there are 462 microstates; \(q_A = 3\) has 100 of them, and all the energy in B has only 28. With 300 + 200 oscillators and 100 units there are about \(9.3 \times 10^{115}\) microstates, and nearly all of them sit near \(q_A = 60\): the energy divides in the ratio \(N_A : N_B = 3 : 2\). The probability of finding \(q_A < 10\) is below \(10^{-20}\). The universe is about \(10^{18}\) seconds old, so you would need to check a hundred times a second for its whole age to have a decent chance of seeing it. In layman’s terms: it never happens.
Here the 300 + 200 oscillator system starts with all 100 units in solid B. At each step, a random oscillator passes one unit (if it has one) to another random oscillator anywhere in the system. Any such step can be undone by the reverse step, which is equally likely, so no microstate is preferred over another: the principle of detailed balance, which is exactly what makes the fundamental assumption work. Yet watch what happens to \(q_A\).
Each square is an oscillator: 300 in A, 200 in B. Darker squares hold more energy units. The time-spent histogram includes the initial climb, so its low tail fades as the run goes on.
Energy flows spontaneously from B to A, and never back (aside from small fluctuations around \(q_A = 60\)): the system behaves irreversibly. That is the physical explanation of heat. It is a probabilistic phenomenon, not absolutely certain but extremely likely. The flow stops when the system reaches, or comes very near, its most likely macrostate, the one with the greatest multiplicity. This law of increase of multiplicity is one version of the famous second law of thermodynamics: not a fundamental law, but a very strong statement about probabilities.
And this is “just” 300 + 200 oscillators. Real solids have around \(10^{23}\), far beyond what even a computer can count directly, which calls for some good approximations, the subject of the next section.