Purpose: Multiple methods have been suggested for quantifying syntactic com- plexity in speech. We compared eight automated syntactic complexity metrics to determine which best captured verified syntactic differences between old and young adults.

Method: We used natural speech samples produced in a picture description task by younger (n = 76, ages 18–22 years) and older (n = 36, ages 53–89 years) healthy participants, manually transcribed and segmented into sentences. We manually verified that older participants produced fewer complex structures. We developed a metric of syntactic complexity using automatically extracted syntac- tic structures as features in a multidimensional metric. We compared our metric to seven other metrics: Yngve score, Frazier score, Frazier–Roark score, develop- mental level, syntactic frequency, mean dependency distance, and sentence length. We examined the success of each metric in identifying the age group using logistic regression models. We repeated the analysis with automatic tran- scription and segmentation using an automatic speech recognition (ASR) system.

Results: Our multidimensional metric was successful in predicting age group (area under the curve [AUC] = 0.87), and it performed better than the other met- rics. High AUCs were also achieved by the Yngve score (0.84) and sentence length (0.84). However, in a fully automated pipeline with ASR, the performance of these two metrics dropped (to 0.73 and 0.46, respectively), while the perfor- mance of the multidimensional metric remained relatively high (0.81).

Conclusions: Syntactic complexity in spontaneous speech can be quantified by directly assessing syntactic structures and considering them in a multivariable manner. It can be derived automatically, saving considerable time and effort compared to manually analyzing large-scale corpora, while maintaining high face validity and robustness.