Background: Telomeres are crucial parts of chromosomes that protect the genome. They shorten every time the cell replicates, and shorter telomeres have been associated with increasing age and with many health behaviours. There is inconclusive evidence on the association between physical activity (PA) and telomere length. Objectives: To examine how leisure-time PA (LTPA) is associated with telomere length and telomere attrition during 10 years of follow-up in elderly people. Design: This study is a 10-year prospective follow-up study. Method: For this prospective study, we examined 1,014 subjects (mean age at baseline 60.8 years) from the Helsinki Birth Cohort Study (HBCS). Relative leukocyte telomere length (LTL) was measured with a quantitative real-time PCR and LTPA with a validated questionnaire. Multiple linear regression analyses were used to assess the association between sex-specific LTPA quartiles and LTL at baseline and change in LTL over 10 years. The analyses were adjusted for age, educational attainment, smoking, body fat percentage, oestrogen exposure in women and for follow-up time when applicable. Results: At baseline, volume of LTPA was not associated with LTL in men (p = 0.66) or in women (p = 0.33). Among women, however, higher volume of LTPA at baseline was associated with greater shortening of LTL (p for linearity 0.040) during the 10-year follow-up. No association was found among men (p for linearity 0.75). Conclusions: Our findings suggest that PA has a sex-specific role in regulation of telomere length in the aging process as in our study a high volume of LTPA in elderly women, but not in men, was associated with more rapid telomere attrition.