J. Lin

On reading the same textbook twice

Every few years I pick up an old textbook I thought I already understood, and discover that I did not. This is not, I think, because the book changed. It is because the part of me that is doing the reading changed, and the book is patient enough to wait.

The first time I read Bishop's Pattern Recognition and Machine Learning, I was a graduate student with a deadline. I skimmed for the tricks I needed and skipped the parts that looked like scaffolding. The derivations that I did work through felt like a performance: I could reproduce each step if asked, but I could not have told you why any particular step was the right one to take. I passed the relevant exam. The book went on a shelf.

I came back to it almost ten years later, for no particular reason other than having an afternoon free and the book nearby. The chapter on the EM algorithm, which I had once treated as a set of equations to memorize, now read as a small piece of writing about patience. You begin with a guess. You do the best you can given the guess. You update the guess. You do this until the guess stops moving. The equations had not changed; I had finally slowed down enough to read them as prose.

I say this not to recommend any particular textbook — the best one is usually whichever one you have already bought — but to argue against the idea that technical reading is a thing you finish. The books I return to most often are not the ones I understood the first time. They are the ones I understood badly enough, the first time, that there was something left to come back for.

There is a related, quieter point. A lot of the best work in machine learning over the last several years has felt, to me, like a slow return to ideas the field had already written down and set aside: scaling behavior, kernel methods in new clothing, the geometry of probability distributions. The people who noticed first were, often, the people who had been willing to read the old books a second time.

Below are two of the books I find myself returning to, in case they are useful to anyone else. Both are distributed, legally and at no cost, by their authors. A machine-readable copy of this list lives at /api/books.