I can't say the new book has won me over (and I ought to stress that, like Not Even Wrong, it's not an easy read), but what I do now understand is the puzzle many onlookers face as to how physicists can end up in what appears to be such an abstruse and disconnected mathematical world to be able to insist with a straight face and counter to all observation that we need at least 10 and probably 11 dimensions to make the universe work.
It seems that string theory emerged from an attempt to explain the strong force back in the late sixties, early seventies. The idea of particles as tiny strings, rather than point particles, seemed to provide an explanation for the strong force, however the only way to make it work required the universe to have 26 dimensions (25 spatial, one of time). This was all looking quite good (if weird, but quantum theory has showed us that weird is okay), until the new collider experiments showed the sort of scattering you'd expect from particles, not strings - and along came quantum chromodynamics, requiring only the standard 4 dimensions, blowing string theory out of the water.
However, the more mathematically-driven physicists loved string theory because it was elegant and seemed to hold together unnaturally well, even if it didn't match the real world. They continued to play around with it and eventually massaged it from what was intended as a description of the strong interaction into a mechanism for quantum gravity (or more precisely several mathematical mechanisms). The good news was that this did away with the 26 dimensions, though the bad news was it still required at least 10. Again, there was no experimental justification for the mathematics, but in its new form, mathematical things started to click into place. There was a surprising effectiveness and fit to other mathematical structures. The approach even fitted a number of oddities of the observed particle families. So the abstruse mathematics felt right - and that, essentially is why so many theoretical physicists have clung onto string theory even though it has yet to make new experimentally verifiable predictions, and has so many possible outcomes and all the other problems those books identify with it.
What Why String Theory? isn't very good at, is giving a feel for what is going on in the brains of the physicists in the way ordinary folk can understand (the author is himself a theoretical physicist), so I thought it might be useful to share an analogy that seemed to fit well for me. We're going to do a thought experiment featuring a civilisation that does mathematics to base 5, rather than the familiar base 10. So they count 1, 2, 3, 4, 10, 11, 12, 13, 14, 20, 21... For some obscure reason they use the same numbers as us, but only have 0, 1, 2, 3 and 4. Now these people have come across some textbooks from our civilisation. And they see all those numbers, which make a kind of sense, except there's some weird extra symbols.
Before I go into what they do, I ought to defend the base 5 idea, in case you're wondering why any civilisation would not sensibly realise they could count on the digits of both hands, but rather stuck to the 5 fingers and a thumb of a single hand. This isn't because the civilisation has a strange one armed mutation, it's because they were cleverer than us. How many can you count to on your two hands? Ten. But my civilisation can count to 30. This is because they don't regard their left and right hands as equivalent, but as two totally separate things with different names. The left hand has five digits. But the right hand has five handits. (Bear with me.) When they count on their fingers, they go up the digits of the left hand just as we do. But when the pinkie goes up, they close the whole left hand and raise the pinkie of their right hand, representing five. They then count up on the left again, but when they get a full hand they raise the second finger on their right hand, and so on. Instead of just working linearly across their fingers and thumbs, by working to base 5 their hands become a simple abacus.
So, back to interpreting our base 10 documents. Some rather wacky mathematicians in this society start playing with using bigger bases than base 5. There's no reason why, no application. It's just interesting. And when they happen on base 10 - so they're counting 1, 2, 3, 4, A, B, C, D, E, 10, 11, 12, 13, 14, 1A, 1B... they get a strange frisson of excitement. This isn't the same as the system used in our documents, where the 12th character in the list is 7, rather than C. But suddenly the two kinds of mathematics start to align. Calculations that didn't make any sense suddenly start to click.
In a hugely simplified analogy, this seems a bit like the string lovers' reason for sticking with their theory. It has that same kind of neat mathematical fit. It seems to work too well to be just coincidence. All those extra dimensions and intricate mathematical manipulation don't seem natural, any more than working to base 10 seems natural when you think of left and right hands as totally different things. But it doesn't mean there's not something behind it. I hope the analogy helps you - it certainly helped me to devise it!