Thermodynamics 2: the statistics

Last week I started a quick look at the most mind-boggling bit of 'classical' physics, the second law of thermodynamics, one of the stars of my book Dice World.  This week I'm exploring how the second law can both be true and not true at the same time.

Thinking of the traditional approach of 'heat flows from hot to cooler', this seemed solid and unbeatable in the Victorian world of brass and iron. And once scientists began to take the idea of atoms and molecules as real entities seriously (something that happened surprisingly late), it also made a lot of sense at the level of individual particles - but there was a twist in the tail.

In the beginning was order

A favourite thought experiment for thermodynamicists is a box that is split in two. (They don't get out much, and consider the Large Hadron Collider to be showy and unnecessary.) On one side of the box is a hot gas. On the other side is a cool gas. Life doesn't get much more exciting than this. But wait, it can - because there is a door in the partition separating the two halves.

Let's imagine we could see the individual molecules of the gas, zooming around. (At least a small subset of them.) on the hot side they would be flying about a lot faster than on the cool side. That's what temperature is all about - the energy levels of the molecules. So we open the door and molecules start to swap between sides of the box. After a while, instead of all hot on the left (say) and all cool on the right we will have a mix of hot and cool on the left and a mix of hot and cool on the right. What has happened in macro terms? The hot side got cooler, the cool side got hotter. Heat moved from the hot bit to the cool bit. 'Result!' as our over-excited thermodynamicist might shout.

But soon disorder ruled

It's worth also looking at this from the point of view of entropy. Entropy, you may remember, is the measure of disorder that is a key to deep thinking about the second law. You could work the change in entropy mathematically, because there is a formula for entropy that is designed for this kind of situation. It's a very simple formula by the standard of modern physics, so I won't apologise for mentioning it. It's:

S = k log W

S is the entropy (E was already used up for energy when this equation came along), k is a constant called Boltzmann’s constant and W is the number of ways a system can be arranged to achieve the particular result. (Log is short for 'logarithm'. If you're old enough, you'll know what this is. If you aren't, look it up.) Think of the example of the letters on this web page. If you imagined there are a series of slots on the screen that you can put letters in (think of the old moveable type printing press), then it’s easy to see that there is one way to arrange the letters to get a specific page, but by trying each letter in each slot you could (very slowly) work out W for randomly distributing all the letters and would get a much higher value.

Mostly entropy isn’t about letters on a page but about stuff, and particularly the atoms or molecules that make that matter up. There again, in principle, we can imagine different values for entropy for, say, a crystal where all the atoms have to slot into specific positions and a gas where they’re bouncing all over the place. We couldn’t do the sums exactly – and would have to resort to statistics to get anywhere – but it’s entirely possible to see how entropy applies this way in theory.

As it happens, we don't need to do the maths to see what is happening in terms of entropy with our box. To start with it was relatively ordered because most of the hot (high speed) molecules were on the left and most of the cool (lower speed) molecules were on the right. After a while it is more disordered because each side is a mix of the two. There are more ways to have them in this mixed state than in a state where they are separated. W is bigger. Disorder - entropy - has increased.

Physicists were so impressed with the inevitability of this process that they were prepared to call it a law, and to say that either heat would flow from hot to cold or nothing would happen at all - but that this process would never reverse. Never ever. Not once. And this is where they got a bit of shock when they started to think of the implications of that simple divided box.

Once we are dealing with billions of molecules, flying around randomly, it isn't possible to make a practical prediction of exactly what will happen from moment to moment. Instead we have to rely on statistics, the branch of mathematics that allows us to take an overview of a lot of items simultaneously. And that tells us that, on the whole, the molecules will balance out and we will end up with a mix on both sides. On the whole, W will increase and the entropy will rise. Disorder will rule. But note that 'on the whole' - not every time. Not an unbreakable law. Just a statistical likelihood.

It is entirely possible - though extremely unlikely - that all the hot molecules will happen to head for the left side at the same time, and all the cooler ones to the right. So the system could go from being all nicely mixed up to being separated again. This would mean heat flowing from a cooler region to a hotter one. Entropy would have spontaneously decreased. Remarkably, the second law, despite being so fundamental to the universe working the way we expect it to, is only statistical. It works most of the time. Almost all the time. But just occasionally, over a long enough timescale, it is bound to fail.

The last episode, coming soon, will wrap up the second law with some demonic action.