Background: Dementia, stroke, depression, and disability are frequent in late life and are major causes of quality of life disruption and family burden. Even though each of these disorders relies on specific pathogenic processes, a common clinical manifestation is psychomotor slowing. Objective: We assessed the relevance of a simple marker of low psychomotor speed in predicting several brain outcomes: dementia, Alzheimer’s disease (AD), Parkinson’s disease (PD), stroke, depressive symptoms, and disability in activities of daily living (ADL) and instrumental ADL (IADL). Methods: PAQUID is a population-based study involving 3,777 individuals aged 65 or older prospectively followed-up with repeated clinical evaluations. After 10 years, 437 participants developed dementia, 333 developed AD, 71 developed PD, 207 reported incident stroke, 404 developed disability in ADL, 994 in IADL, and 494 developed depressive symptomology. Psychomotor speed was measured with the digit symbol substitution test (DSST). Cox proportional hazards models controlled for several confounders assessed the risk of incident outcomes. Results: Participants with low DSST performance had increased risk of incident all-type dementia (hazard ratio [HR] 3.41, p < 0.0001) and AD-type dementia (HR 3.18, p < 0.0001). Higher risk for PD (HR 2.98, p = 0.04), IADL (HR 1.82, p < 0.0001), ADL disability (HR 1.95, p = 0.001), depressive symptoms (HR 1.53, p = 0.03), and a statistical trend for stroke (HR 1.88, p = 0.09) was also found. Conclusion: Low psychomotor speed is associated with an increased risk of developing various brain outcomes: dementia, AD, PD, disability, depressive symptoms, and marginally stroke. Low psychomotor speed may be the consequence of a number of discrete cerebral abnormalities and could be considered as a marker of brain vulnerability. In clinical practice, a low score in DSST should be seen as a warning sign of possible negative evolution.