Background: Reduced engagement with habitual activity (HA) is associated with greater risk and progression of cognitive decline and falls in older adults and people with dementia. Understanding external and intrinsic factors that affect HA may provide novel targets for non-pharmacologic interventions. Objective: This study primarily aims to identify factors that influence HA in normal ageing and cognitive impairment, such as cognitive and motor problems and disease subtype. Methods: 108 older adults participated in this study; 36 with cognitive impairment due to Alzheimer’s disease (AD), 30 with dementia with Lewy bodies (DLB), 16 with Parkinson’s disease dementia (PDD), and 26 controls. A tri-axial accelerometer recorded continuous data of volume, variability, and pattern of HA over 7 days. Participants undertook a battery of cognitive and neuropsychological assessments. Results: One-way analysis of variance controlling for age and gender shows that people with DLB and PDD engage less with HA than controls (p ≤ 0.01), but there were no significant differences between AD and controls (p ≥ 0.01). Multivariate analysis demonstrated motor disease and impairments in activities of daily living (ADLs) independently explained 10–26% of volume, variability, and pattern of HA in people with cognitive impairment. Conclusion: People with cognitive impairment have reduced HA engagement compared to controls. Motor disease and impairments in ADLs most strongly contribute to these findings and may be important to consider for disease management. Wearable technology can provide a personalised picture of an individual’s daily behaviours and may be a useful tool for person-centred care.