Where do the values of 1em and 1ex come from?

In my answer to “Which measurement units should one use in LaTeX?” I explore the values of 1em and 1ex for different fonts using both TFM fonts and the font mechanisms of XeLaTeX that uses non-TFM ...