中文标题#
语音作为多任务基于大语言模型的心理健康预测的多模态数字表型
英文标题#
Speech as a Multimodal Digital Phenotype for Multi-Task LLM-based Mental Health Prediction
中文摘要#
语音是一种非侵入性的数字表型，可以为心理健康状况提供有价值的见解，但通常被当作单一模态处理。 相比之下，我们提出将患者语音数据视为用于抑郁症检测的三模态多媒体数据源。 本研究探讨了基于大型语言模型架构在融合语音生成文本、声学特征点和声音生物标志物的多模态环境中进行语音基础抑郁症预测的潜力。 青少年抑郁症是一个重大挑战，且常与其他多种疾病共存，例如自杀意念和睡眠障碍。 这为我们研究中同时预测抑郁症、自杀意念和睡眠障碍的多任务学习（MTL）提供了额外的机会。 我们还提出了一种纵向分析策略，该策略对多个临床互动中的时间变化进行建模，从而全面理解病情的发展。 我们提出的三模态、纵向多任务学习方法在 Depression Early Warning 数据集上进行了评估。 它实现了 70.8% 的平衡准确率，高于每种单模态、单任务和非纵向方法。
英文摘要#
Speech is a noninvasive digital phenotype that can offer valuable insights into mental health conditions, but it is often treated as a single modality. In contrast, we propose the treatment of patient speech data as a trimodal multimedia data source for depression detection. This study explores the potential of large language model-based architectures for speech-based depression prediction in a multimodal regime that integrates speech-derived text, acoustic landmarks, and vocal biomarkers. Adolescent depression presents a significant challenge and is often comorbid with multiple disorders, such as suicidal ideation and sleep disturbances. This presents an additional opportunity to integrate multi-task learning (MTL) into our study by simultaneously predicting depression, suicidal ideation, and sleep disturbances using the multimodal formulation. We also propose a longitudinal analysis strategy that models temporal changes across multiple clinical interactions, allowing for a comprehensive understanding of the conditions' progression. Our proposed approach, featuring trimodal, longitudinal MTL is evaluated on the Depression Early Warning dataset. It achieves a balanced accuracy of 70.8%, which is higher than each of the unimodal, single-task, and non-longitudinal methods.
PDF 获取#
抖音扫码查看更多精彩内容