Automated Scoring of Tablet-Administered Expressive Language Tests.pdf

Speech and language impairments are common pediatric conditions, with as many as 10% of children experiencing one or both at some point during development. Expressive language disorders in particular often go undiagnosed, underscoring the immediate need for assessments of expressive language that can be administered and scored reliably and objectively. In this paper, we present: (1) our tablet-based child language assessment instrument; (2) a dataset we havecollected with this instrument; and
more » ... ) a set of highly accurate computational models we havedeveloped for scoring responses to several expressive language tasks administered with thisinstrument. In our assessment framework, instructions and stimuli are presented to the childon a tablet computer, which records the child's responses in real time, while a clinician controlsthe pace and presentation of the tasks using a second tablet. The recorded responses for fourdistinct expressive language tasks (expressive vocabulary, word structure, recalling sentences,and formulated sentences) are then scored using traditional paper-and-pencil scoring and usingmachine learning methods relying on a deep neural network-based language representationmodel. All four tasks can be scored automatically from both clean and verbatim speech transcriptswith very high accuracy at the item level (83􀀀99%). In addition, these automated scores correlatestrongly and significantly (r = 0:78 􀀀 0:99, p < 0:001) with manual item-level, raw, and scaledscores. These results point to the utility and potential of automated computationally-driven methods of both administering and scoring expressive language tasks for pediatric developmental language evaluation.
doi:10.6084/m9.figshare.14043941.v1 fatcat:hznbrwgv2zeqzkm7c2hry4jb3q