We studied the validity of the aphasia item of a widely used stroke scale – the Scandinavian Stroke Scale (SSS) – in discriminating between aphasia and normal language function in 33 stroke patients of an acute stroke unit. They were assessed by a nurse using the aphasia item from the SSS and by a speech and language therapist carrying out a full evaluation of the language function. The latter served as the ‘gold standard’. The agreement between the nurses’ and the speech and language therapist’s scoring was good (weighted kappa = 0.74, 95% CI 0.51–0.97), and the sensitivity and specificity of the SSS aphasia item were also satisfactory. However, the predictive value of a positive test was as low as 0.55 (95% CI 0.23–0.83), indicating nearly every second of the positives being false positive. Using the aphasia score of the SSS as a diagnostic aid for aphasia after stroke results in a high rate of false positives and inflates the prevalence figures for aphasia in epidemiological studies of stroke.