To survey and describe the quality and breadth of the literature surrounding the use of machine learning on drawing tests of cognition.

Dementia affects approximately 6.4% of the North American population over 60 years of age, and costs an estimated US $226 Billion per year.  Future treatments of dementia are likely to require early detection of cognitive impairment that is reliable, cost-effective and efficient.  Machine learning has shown promise as an outstanding tool in the areas of data and image classification.

The Embase, Medline and Cochrane Central Library databases were searched for a combination of keywords used to describe cognitive tests of drawing AND machine learning.  Inclusion criteria for full text review were papers describing machine learning using features of cognitive drawing tasks for classification of cognitive deficits.  Title/abstract and full text review were undertaken by four reviewers, with discrepancies resolved by the primary investigator.  Papers selected from the full text review process were then adjudicated by the grading of recommedations, assessment, development and evaluations (GRADE) methodology.

4620 titles/abstracts were screened, out of which 64 were selected for full text review.  21 studies were selected for qualitative synthesis, out of which 18 were selected for reporting.  The GRADE assessment provided very low quality evidence for the use of machine learning for classifying cognitive deficits.  Quality was primarily limited by poor study design, lack of clear gold standard comparators, small study size and inconsistency in outcomes, techniques, and study design.  Mean accuracy was 70%, mean sensitivity was 73% and mean specificity was 75%.

Machine learning has been applied on cognitive drawing tasks but the quality of the evidence for its use is very low.  For future work, machine learning models must first be optimized on large, diverse datasets, and then validated using randomized clinical trials that compare outcomes against standard methods.