Background and Purpose: The understanding of the contribution of genetic factors to cognitive impairment after stroke is incomplete. The aim of the study was to examine whether the apolipoprotein E ε4 allele (ApoE ε4) is a risk factor for cognitive impairment in the early phase after stroke. Methods: The sample comprised 152 Norwegian stroke rehabilitation inpatients (mean age 76.8 years, SD 10.5) examined at a mean of 18.3 days (SD 13.4) after hospital admission. Post-stroke cognitive impairment was assessed with the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). The following proposed risk factors were analysed: ApoE genotype, demographics (age, sex, education), pre-stroke cognitive reduction [Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE)], pre-stroke vascular factors (including previous stroke), stroke characteristics (type, location), and neurological stroke-related impairment [National Institutes of Health Stroke Scale (NIHSS)]. Cognitive impairment was defined as an RBANS total index score ≤1.5 SD below the mean. Multiple logistic regression analyses were performed to find risk factors for post-stroke cognitive impairment. Results: Four variables were found to be independent risk factors for cognitive impairment after stroke: ApoE ε4 (OR = 3.7; 95% CI = 1.2–11.6), IQCODE score ≥3.44 (OR = 9.2; 95% = CI 2.3–37.2), total or partial anterior stroke syndromes (OR = 3.2; 95% CI = 1.3–8.0), and NIHSS total score >5 (OR = 7.3; 95% CI = 2.7–19.7). No association between ApoE ε4 and pre-stroke cognitive reduction (IQCODE) was found. Conclusions: The presence of one or two ApoE ε4 alleles may be a significant independent risk factor for cognitive impairment in the early phase after stroke.