I've just reviewed a very interesting book - Pythagoras' Revenge by Arturo Sangalli.
It's a novel... but its aim is more that of a popular maths book. Although there is a storyline and characters, it explains quite a lot of maths along the way.
In previous posts I've looked at books that were science books without being science fiction - whether it was the lab lit of Experimental Heart and Tangled Roots or the nuclear energy thriller Rad Decision. This book is slightly different - where they have a science setting and incidentally get across some of the science, this explicitly sets out to be a vehicle to educate as well as entertain.
It's a mixed bag - I wasn't particularly impressed with it as a novel, and occasionally it just goes into information dump mode - yet despite this, I'd say it was a success. The fact is, it was a lighter read than a traditional popular science/maths book. It did make me want to read on, like fiction does.
I think there are some real opportunities for doing this kind of thing even better in the future.