There are conflicting reports about the potential role of vitamin antioxidants in preventing and/or slowing the progression of various forms of cognitive impairment including Alzheimer’s disease (AD). We examined longitudinal data from the Canadian Study of Health and Aging, a population-based, prospective 5-year investigation of the epidemiology of dementia among Canadians aged 65+ years. Our primary objective was to examine the association between supplemental use of antioxidant vitamins and subsequent risk of significant cognitive decline (decrease in 3MS score of 10 points or more) among subjects with no evidence of dementia at baseline (n = 894). We also explored the relationship between vitamin supplement use and incident vascular cognitive impairment (VCI; including a diagnosis of vascular dementia, possible AD with vascular components and VCI but not dementia), dementia (all cases) and AD. After adjusting for potential confounding factors assessed at baseline, subjects reporting a combined use of vitamin E and C supplements and/or multivitamin consumption at baseline were significantly less likely (adjusted OR 0.51; 95% CI 0.29–0.90) to experience significant cognitive decline during a 5-year follow-up period. Subjects reporting any antioxidant vitamin use at baseline also showed a significantly lower risk for incident VCI (adjusted OR 0.34, 95% CI 0.13–0.89). A reduced risk for incident dementia or AD was not observed. Our findings suggest a possible protective effect for antioxidant vitamins in relation to cognitive decline but randomized controlled trials are required for confirmation.