Human brain structure changes throughout the lifespan. Brouwer et al. identified genetic variants that affect rates of brain growth and atrophy. The genes are linked to early brain development and neurodegeneration and suggest involvement of metabolic processes.