Introduction to Statistical Physics

Introduction to Statistical Physics¶

Imagine you have a bottle of air sitting on your desk. Inside that bottle are roughly 10²² molecules — nitrogen, oxygen, a little argon — all flying around, bouncing off each other and off the walls, changing direction billions of times per second. If you tried to write down the position and velocity of every single molecule and track how each one evolves according to Newton’s laws, you would need more computer memory than exists on Earth, and more time than the age of the universe.

And yet, with just three numbers — temperature, pressure, and volume — you can describe that bottle of air with extraordinary precision. How is that possible?

The answer is statistical physics: the art of extracting exact macroscopic predictions from microscopic chaos, not despite the enormous number of particles, but because of it. When 10²² particles are involved, statistical fluctuations become negligibly small, and average behaviour becomes essentially certain. This is the deep reason thermodynamics works so well.

This chapter introduces the ideas, vocabulary, and first results of statistical physics. We will:

1. Understand the difference between microstates and macrostates.

2. State and explore the fundamental postulate of statistical mechanics.

3. Introduce phase space as the arena where statistical mechanics lives.

4. Define the Boltzmann entropy and understand what it really means.

5. Derive temperature, pressure, and the first law from statistics alone.

6. Apply everything to the classical ideal gas.

7. Resolve the Gibbs paradox and understand why particles must be treated as indistinguishable.

Microstates and Macrostates¶

The Gap Between Micro and Macro¶

The most fundamental distinction in statistical physics is the difference between a microstate and a macrostate.

A microstate is the complete, precise description of a system at the atomic level — the exact position and velocity of every single particle, at every instant. For a gas of $N$ particles in three dimensions, this is a list of $6N$ numbers: three position coordinates and three momentum components per particle. The microstate changes billions of times per second as particles collide and move.

A macrostate is what we actually measure in the lab — quantities like temperature $T$, pressure $P$, volume $V$, and total energy $E$. A macrostate is specified by just a handful of numbers, regardless of whether the system contains 10 or 10²³ particles.

A Warm-Up: Coins and Counting¶

Before diving into atoms and gases, let us build intuition with the simplest possible example: flipping coins.

Imagine you flip $N = 4$ coins. Each coin can be Heads (H) or Tails (T). The microstate is the full outcome of every coin — something like HHTH. There are $2^4 = 16$ equally probable microstates in total.

Now suppose you only care about the total number of heads — this is the macrostate. There are only 5 possible macrostates: 0, 1, 2, 3, or 4 heads.

How many microstates correspond to each macrostate?

The macrostate with 2 heads is 6 times more likely than the macrostate with 0 heads, simply because it has 6 times as many corresponding microstates. If we observed the system at a random moment, we would almost certainly find it in the most probable macrostate.

Now scale this up: for $N = 100$ coins, the number of microstates for 50 heads is roughly 10²⁹, while 100 heads has just 1. For $N = 10^{23}$ coins (the scale of real physics), the most probable macrostate is so overwhelmingly more likely than any other that it is, for all practical purposes, the only macrostate we ever observe. This is why thermodynamics is so precise.

The Fundamental Postulate¶

All Microstates Are Equally Likely¶

The entire edifice of statistical mechanics rests on one foundational assumption:

This might seem like a very bold claim. Why should we believe it? There are several reasons:

It is the most unbiased assumption. If we have no information that distinguishes one microstate from another, it is logically most honest to assign them equal probability. Any other choice would require us to explain why some microstates are preferred.

It is consistent with the equations of motion. Liouville’s theorem in classical mechanics — and its quantum analogue — shows that the density of phase-space trajectories is conserved as the system evolves. A uniform distribution over accessible microstates is therefore a stationary (time-independent) distribution, consistent with equilibrium.

It works. Every prediction derived from this postulate agrees with experiment to extraordinary precision. That is ultimately its strongest justification.

Why Systems Settle in the Most Probable Macrostate¶

Given the fundamental postulate, why do we always observe the most probable macrostate?

The answer is pure arithmetic. In a system of 10²³ particles, the most probable macrostate has so many more corresponding microstates than any other macrostate that the probability of ever observing a different macrostate is not just small — it is essentially zero.

To put it concretely: if you released a gas into one half of a box, the probability that all molecules spontaneously return to the left half is $2^{-10^{23}}$ — a number so tiny that even if you waited for a trillion times the age of the universe, it would almost certainly never happen.

This is the statistical origin of the second law of thermodynamics: systems evolve towards states of higher probability (higher entropy) and never spontaneously return to states of lower probability.

Phase Space: The Arena of Statistical Mechanics¶

What is Phase Space?¶

To count microstates for a continuous system (like a gas of real atoms), we need a mathematical arena. That arena is phase space.

For a single particle moving in three dimensions, we need 6 numbers to specify its state completely: three position coordinates $(x, y, z)$ and three momentum components $(p_x, p_y, p_z)$. Together, these define a single point in a 6-dimensional space — the phase space of one particle.

For a system of $N$ particles, the phase space is $6N$-dimensional. A single point in this high-dimensional space specifies the complete microstate of the entire system — where every particle is and how fast it is moving.

The Energy Surface and the Energy Shell¶

For an isolated system with fixed total energy $E$, all accessible microstates must satisfy:

where $H$ is the Hamiltonian (total energy function). This condition defines a surface in phase space — the energy surface — and the system’s trajectory is forever confined to it.

In practice, we always allow a tiny energy uncertainty $\Delta$ (much smaller than $E$), so we work with a thin shell between energies $E$ and $E + \Delta$ rather than an infinitely thin surface. The volume of this energy shell is the key quantity we need:

This volume $\Gamma$ counts (in a precise sense) how many microstates the system has access to. The larger $\Gamma$, the more microstates are accessible, and — as we will see — the larger the entropy.

Entropy: Counting Disorder¶

The Boltzmann Definition¶

We have seen that some macrostates are more probable than others simply because they correspond to more microstates. Boltzmann had the brilliant idea of capturing this in a single number. In 1877, he proposed:

where:

• $S$ is the entropy of the macrostate,

• $\Omega$ is the number of microstates compatible with that macrostate (proportional to the phase-space volume $\Gamma$),

• $k_B = 1.381 \times 10^{-23}$ J/K is Boltzmann’s constant, which sets the scale.

This formula is engraved on Boltzmann’s tombstone in Vienna. It is one of the most important equations in all of physics.

Why the Logarithm?¶

The logarithm in $S = k_B \ln \Omega$ is not arbitrary. It is chosen for a crucial physical reason: entropy must be additive.

If you have two independent systems A and B, their combined entropy should be $S_{A+B} = S_A + S_B$ (total entropy is the sum of parts). But the total number of microstates for two independent systems is the product: $\Omega_{A+B} = \Omega_A \times \Omega_B$.

The logarithm converts this product into a sum:

Any other function of $\Omega$ would fail this requirement.

Entropy in Everyday Terms¶

Let us build some intuition with a few everyday examples before turning to physics.

Temperature: What Does It Really Mean?¶

Heat Flow Between Two Systems¶

We now ask: if two systems are placed in thermal contact and allowed to exchange energy, what determines how they share that energy at equilibrium?

Consider two systems, 1 and 2, with energies $E_1$ and $E_2$ and entropies $S_1$ and $S_2$. Their total energy is fixed: $E_1 + E_2 = E = \text{const}$. Energy can flow back and forth, but the total is conserved.

The combined system will settle into the most probable state, which means the state with the maximum total number of microstates, i.e., the state that maximises the total entropy $S_1 + S_2$.

When we work out the mathematics of this maximisation, we find that at equilibrium:

In other words, equilibrium requires that both systems have the same rate of change of entropy with energy. This common value is what we define as the inverse temperature:

The Physical Meaning of Temperature¶

This definition might look abstract, but it has a beautifully simple physical interpretation.

$\partial S / \partial E$ measures how rapidly the number of accessible microstates increases when you add a little energy to the system. Think of it as the system’s appetite for energy:

• A system with a large $\partial S/\partial E$ (low $T$) has a strong appetite — adding energy dramatically increases the number of accessible microstates.

• A system with a small $\partial S/\partial E$ (high $T$) has a weak appetite — it already has so many accessible microstates that adding more energy barely changes the count.

Heat flows from high $T$ to low $T because it increases the total entropy:**

When a hot system (small $\partial S/\partial E$) transfers energy to a cold system (large $\partial S/\partial E$), the cold system gains more entropy than the hot system loses. The total entropy increases. This process is spontaneous. The reverse — heat flowing from cold to hot — would decrease total entropy and never happens spontaneously.

The Zeroth Law of Thermodynamics¶

The definition of temperature as the quantity equalised at thermal equilibrium immediately implies the zeroth law of thermodynamics:

Pressure, Chemical Potential, and the First Law¶

Where Does Pressure Come From?¶

Just as temperature was defined by asking what is equalised when two systems can exchange energy, we can define pressure by asking what is equalised when two systems can exchange volume.

Imagine two chambers separated by a movable, thermally insulating piston. The total volume $V_1 + V_2 = V$ is fixed, but the piston can slide, exchanging volume between the two sides. The system maximises total entropy, which gives the equilibrium condition:

The common value of this derivative defines pressure through $P/T = (\partial S/\partial V)_{E,N}$. Microscopically, pressure arises because moving the wall gives the particles more space to occupy — more accessible microstates — and the system pushes in the direction that increases its entropy.

Chemical Potential¶

Similarly, if two systems can exchange particles, the equilibrium condition involves the chemical potential $\mu$, defined through:

Particles flow from regions of high chemical potential to low chemical potential, just as heat flows from high temperature to low temperature.

The First Law of Thermodynamics¶

Putting together the definitions of temperature, pressure, and chemical potential, we can write a single equation that expresses how the internal energy $U$ changes when the macroscopic state of the system changes:

This is the first law of thermodynamics — the conservation of energy — but written in a form that makes explicit the three ways energy can change:

The Second Law: Entropy Always Increases¶

The Statement¶

The second law of thermodynamics is one of the most profound statements in all of science:

From the statistical viewpoint, this is not mysterious at all. An isolated system evolves randomly through its accessible microstates. There are enormously many more microstates corresponding to the high-entropy macrostate than to any low-entropy macrostate. So the system almost always moves towards higher entropy — not because of any driving force, but simply because of the overwhelming weight of numbers.

The Arrow of Time¶

The second law gives time its direction. The microscopic laws of physics (Newton’s laws, quantum mechanics) are perfectly symmetric in time — if you filmed a single particle bouncing, you could not tell if the film was playing forwards or backwards. But if you filmed a gas expanding into a vacuum and played it backwards, the absurdity would be immediately obvious.

This asymmetry is entirely statistical. The expanding gas visits ever more probable macrostates. The reversed film would show a journey towards an astronomically improbable macrostate — possible in principle, but never observed in practice.

Free Expansion as an Example¶

A gas occupies the left half of a box. The right half is empty (vacuum). The partition is suddenly removed.

• Before: all $N$ molecules confined to volume $V/2$. Few accessible microstates.

• After: all $N$ molecules free to move in volume $V$. Each molecule has twice as many positions available. The number of accessible microstates increases by $(2)^N = 2^{10^{23}}$ — an unimaginably enormous factor.

The entropy increase is:

For one mole of gas, this is about 5.8 J/K — a macroscopically measurable entropy increase. The gas never spontaneously contracts back to the left half.

The Classical Ideal Gas¶

Now we apply the full statistical mechanics framework to the most important example: the classical ideal gas. This is a gas of $N$ identical, non-interacting particles in a box of volume $V$, with total energy $E$.

“Non-interacting” means the particles do not push or pull on each other — they only feel the walls of the box. This is an excellent approximation for low-density gases at high temperature (like ordinary air at room conditions).

Counting Microstates¶

For the ideal gas, the total energy is just the sum of kinetic energies of all particles:

where $\mathbf{p}_i$ is the momentum of particle $i$ and $m$ is its mass.

The constraint $H = E$ defines a hypersphere in the $3N$-dimensional momentum space with radius $R = \sqrt{2mE}$. Multiplying by the position space volume $V^N$ (each of the $N$ particles can be anywhere in the box), the total accessible phase-space volume is:

The volume of a hypersphere in $n$ dimensions with radius $R$ is $\frac{\pi^{n/2}}{\Gamma(n/2+1)} R^n$, giving:

The Sackur-Tetrode Equation¶

Taking the logarithm and using the correct normalisation $\Gamma_0 = h^{3N} N!$ (more on the $N!$ shortly), the entropy of the ideal gas works out to be:

This is the Sackur-Tetrode equation (1912), independently derived by Hugo Sackur and Otto Tetrode. It is the complete entropy of a monatomic classical ideal gas.

From this single equation, all thermodynamic properties of the ideal gas follow by differentiation.

Everything Follows from Entropy¶

The Gibbs Paradox and Indistinguishability¶

A Surprising Puzzle¶

Here is a thought experiment that reveals something deep about the nature of identical particles.

Take a box divided by a partition. On the left: $N/2$ molecules of gas A at temperature $T$ and pressure $P$. On the right: $N/2$ molecules of gas B, also at temperature $T$ and pressure $P$. Now remove the partition.

If A and B are different gases (say, nitrogen and oxygen), the gases mix, and the entropy increases. This is the entropy of mixing — a real, measurable effect. You can verify it by noting that you now need to do work to separate them again.

But what if A and B are the same gas — say, nitrogen on both sides? When you remove the partition, nothing observable happens. The gas on both sides was already at the same temperature, pressure, and density. The system is in equilibrium both before and after.

The entropy should not change. And yet if you naively count microstates by treating the molecules as distinguishable labelled particles (as classical mechanics suggests), you find a spurious entropy increase of $\Delta S = Nk_B \ln 2 > 0$ — as if mixing two identical gases were somehow different from having one big container of gas. This is the Gibbs paradox, noticed by Josiah Willard Gibbs in 1875.

The Resolution: Particles Are Indistinguishable¶

The resolution is both simple and profound: identical particles are not distinguishable.

In quantum mechanics, two electrons, two nitrogen molecules, or any two identical particles are not merely similar — they are in principle completely indistinguishable. There is no experiment, even in principle, that can tell “molecule number 347” from “molecule number 8,193” if both are nitrogen molecules in the same quantum state.

This means that if you swap two identical particles, you have not created a new microstate — you have the same microstate. To correctly count microstates, we must divide by the number of ways we can permute $N$ identical particles, which is $N!$ (N factorial):

With this correction, the entropy becomes:

Notice that the volume appears as $V/N$ (volume per particle), not $V$. This makes the entropy extensive — proportional to $N$ when $E/N$ and $V/N$ are held fixed — and the mixing entropy for two identical gases is exactly zero.

What Happens for Different Gases?¶

When genuinely different gases mix, the entropy increase is real and is given by:

where $x_1 = N_1/N$ and $x_2 = N_2/N$ are the mole fractions. For equal amounts ($x_1 = x_2 = 1/2$), this gives $\Delta S = Nk_B \ln 2 > 0$. This entropy of mixing is the thermodynamic reason why spontaneous mixing of different gases is irreversible.

A Preview: The Three Ensembles¶

The microcanonical ensemble — the framework we have built throughout this chapter — describes a system that is completely isolated: fixed $E$, $V$, and $N$. It is the most fundamental ensemble, but it is often the most difficult to work with in practice, because real experiments rarely fix the energy exactly.

Statistical mechanics has developed two other ensembles, each suited to a different experimental situation. All three give the same thermodynamic results in the limit of large $N$.

The microcanonical ensemble answers the question: given fixed energy, what are the equilibrium properties?

The canonical ensemble answers: given fixed temperature (heat bath), what are the equilibrium properties?

The grand canonical ensemble answers: given fixed temperature and chemical potential (heat and particle reservoir), what are the equilibrium properties?

Summary¶

In this chapter we have laid the foundations of statistical physics. The central ideas to take away are:

Exercises¶