中文标题#
語音作為多任務基於大語言模型的心理健康預測的多模態數字表型
英文标题#
Speech as a Multimodal Digital Phenotype for Multi-Task LLM-based Mental Health Prediction
中文摘要#
語音是一種非侵入性的數字表型，可以為心理健康狀況提供有價值的見解，但通常被當作單一模態處理。 相比之下，我們提出將患者語音數據視為用於抑鬱症檢測的三模態多媒體數據源。 本研究探討了基於大型語言模型架構在融合語音生成文本、聲學特徵點和聲音生物標誌物的多模態環境中進行語音基礎抑鬱症預測的潛力。 青少年抑鬱症是一個重大挑戰，且常與其他多種疾病共存，例如自殺意念和睡眠障礙。 這為我們研究中同時預測抑鬱症、自殺意念和睡眠障礙的多任務學習（MTL）提供了額外的機會。 我們還提出了一種縱向分析策略，該策略對多個臨床互動中的時間變化進行建模，從而全面理解病情的發展。 我們提出的三模態、縱向多任務學習方法在 Depression Early Warning 數據集上進行了評估。 它實現了 70.8% 的平衡準確率，高於每種單模態、單任務和非縱向方法。
英文摘要#
Speech is a noninvasive digital phenotype that can offer valuable insights into mental health conditions, but it is often treated as a single modality. In contrast, we propose the treatment of patient speech data as a trimodal multimedia data source for depression detection. This study explores the potential of large language model-based architectures for speech-based depression prediction in a multimodal regime that integrates speech-derived text, acoustic landmarks, and vocal biomarkers. Adolescent depression presents a significant challenge and is often comorbid with multiple disorders, such as suicidal ideation and sleep disturbances. This presents an additional opportunity to integrate multi-task learning (MTL) into our study by simultaneously predicting depression, suicidal ideation, and sleep disturbances using the multimodal formulation. We also propose a longitudinal analysis strategy that models temporal changes across multiple clinical interactions, allowing for a comprehensive understanding of the conditions' progression. Our proposed approach, featuring trimodal, longitudinal MTL is evaluated on the Depression Early Warning dataset. It achieves a balanced accuracy of 70.8%, which is higher than each of the unimodal, single-task, and non-longitudinal methods.
PDF 获取#
抖音掃碼查看更多精彩內容