Title

题目

Deep learning detection of acute and sub-acute lesion activity from single-timepoint conventional brain MRI in multiple sclerosis

基于单次常规脑MRI的深度学习检测多发性硬化症急性和亚急性病变活动性

01

文献速递介绍

多发性硬化症（MS）是一种中枢神经系统的慢性炎症性和神经退行性疾病。其病理特征为脱髓鞘病变的形成，在MRI上表现为相对于周围正常白质（NAWM）的T2加权（T2w）高信号（Traboulsee和Li，2006）。在随访稀疏的纵向研究中，通过比较连续T2w扫描可检测到新病变的出现，这是急性炎症性疾病活动的实用标志物（Altay等人，2013）。因此，新病变活动的测量通常被用作临床试验的终点（Calabresi等人，2014），并在临床环境中指导治疗决策和预后（Wattjes等人，2021）。   新病变形成的急性期可分为两个阶段（Rovira等人，2013）：急性期和亚急性期。初始急性期的特征是血脑屏障短暂破坏，在静脉注射钆基造影剂后的T1w扫描中表现为局灶性对比增强区域（Kappos等人，1999），平均持续3-6周（Cotton等人，2003）。随后的亚急性期表现为未增强T1w和T2w图像上病变大小和信号强度的显著变化，通常持续3-6个月（Rovira等人，2013）（可能反映炎症水肿的吸收和包括修复在内的非炎症过程的复杂平衡）。亚急性期最终会进入慢性期（Rovira等人，2013）。   重要的是，病变演变是一个连续且异质的过程。尽管血脑屏障破坏后短时间内病变大小或信号强度的快速变化可能反映亚急性炎症活动及急性病理的逐渐消退，但这种变化也可能发生在所谓的慢性病变期，包括缓慢扩展的慢性活动性表型。本研究中常用的6个月阈值提供了亚急性与慢性状态的实用定义，应将其视为对未知生物学真相的近似。   虽然急性期病变可通过单次扫描（对比增强T1w扫描）识别，但亚急性期病变检测目前需要比较两个时间点（间隔不超过6个月），这带来了特殊挑战。在MS患者的临床管理中，对先前近期参考扫描的需求可能会延误治疗。在临床试验中，患者筛查通常在单一时间点进行，无需先前参考，因此急性病变检测仅限于钆增强（GdE）病变。这降低了活动性MS患者的试验 eligibility（入选资格）。[1]   为应对这些挑战，我们提出了一系列深度学习（DL）方法，以从单次MRI量化脑水平的急性炎症性病变活动。我们的贡献如下：   1. 定义新任务：从单次MRI量化脑水平的急性和亚急性MS病变活动。该任务超越了传统的GdE病变检测，旨在检测所有小于24周的病变，无论是否存在钆增强。我们在预测未来6个月急性病变活动的背景下，证明了新任务的临床实用性。   2. 多序列MRI建模：使用多个常规序列（对比前后T1加权、T2加权和质子密度加权）的MRI数据开发和评估DL模型。通过比较不同模型，我们为新任务建立了强有力的基准，并提出整合全脑特征的潜在优势，为未来研究指明方向。   3. 临床应用验证：我们的最佳性能模型（2D-UNet）通过单次扫描估算急性病变活动，当与传统GdE病变活动测量结合使用时，可显著改善对未来急性病变活动的预后预测，证明了其临床实用性。   本文其余部分结构如下：第2节总结先前工作，第3节介绍材料和方法（即数据/模型），第4节展示模型基准测试结果，第5节讨论每个模型的优势和局限性。

Abatract

摘要

Multiple sclerosis (MS) is a chronic inflammatory disease characterized by demyelinating lesions in the centralnervous system. Cross-sectional measurements of acute inflammatory lesion activity are typically obtained bydetecting the presence of gadolinium enhancement in lesions, which typically lasts 3-6 weeks. We formulatethe novel and clinically relevant task of quantification of recent acute lesion activity from the past 24 weeks(6 months) using single-timepoint conventional brain magnetic resonance imaging (MRI). We develop andcompare several deep learning (DL) methods for estimating this brain-level acuteness score and show that a2D-UNet can accurately predict acute disease activity at the patient-level while outperforming transformersand ensemble approaches. In the context of identifying subjects with acute (less than 6 months-old) lesionactivity, our 2D-UNet achieves an area under the receiver-operating curve in the range 80−84% on independentrelapsing-remitting MS cohorts. When used in conjunction with measurements of gadolinium-enhancing lesionactivity, our model significantly improves the prognostication of future acute lesion activity (over the next6 months). This model could thus be leveraged for population recruitment in clinical trials to identify ahigher number of patients with acute inflammatory activity than current standard approaches (e.g., gadoliniumpositivity) with a predictable precision/recall trade-off.

多发性硬化症（MS）是一种慢性炎症性疾病，其特征为中枢神经系统中的脱髓鞘病变。急性炎症性病变活动的横断面测量通常通过检测病变中钆增强的存在来实现，这种增强通常持续3-6周。我们提出了一项新颖且具有临床相关性的任务：利用单次常规脑磁共振成像（MRI）量化过去24周（6个月）内的近期急性病变活动。我们开发并比较了几种用于估算脑水平急性评分的深度学习（DL）方法，结果表明，2D-UNet能够在患者水平上准确预测急性疾病活动，其性能优于Transformer和集成方法。在识别具有急性（小于6个月）病变活动的受试者方面，我们的2D-UNet在独立的复发缓解型MS队列中实现了80-84%的受试者工作特征曲线下面积。当与钆增强病变活动的测量结合使用时，我们的模型显著改善了对未来急性病变活动（未来6个月内）的预后预测。因此，该模型可用于临床试验中的人群招募，以比当前标准方法（如钆阳性）以可预测的精确率/召回率权衡识别出更多具有急性炎症活动的患者。

Method

方法

3.1. Population overview

Brain MRI scans from 3 phase-III pivotal trials were retrospectively analyzed: ADVANCE (Calabresi et al., 2014) (1512 subjects withrelapsing-remitting MS; NCT00906399), ASCEND (Polman et al., 2006)(886 subjects with secondary progressive MS; NCT01416181) and DECIDE (Kappos et al., 2015) (1841 subjects with relapsing-remittingMS; NCT01064401). Inclusion criteria and MRI acquisition protocolshave been previously described; Table 1 summarizes relevant statistics.Briefly, patients participated in a baseline session and a series of followups, during which the following MRI modalities were acquired: T1w(pre- and post-gadolinium injection), T2w and proton density-weighted(PDw). Follow-up MRI scans were acquired 24, 48 and 96 weeks postbaseline in ADVANCE, after 24, 48, 72, 96, 108 and 156 weeks inASCEND, and after 24 and 96 weeks in DECIDE. Patient inclusion inthis study was constrained by MRI data availability.

3.1 人群概述   本研究回顾性分析了三项III期关键试验的脑MRI扫描数据：ADVANCE（Calabresi等，2014）（1512例复发缓解型MS患者；NCT00906399）、ASCEND（Polman等，2006）（886例继发进展型MS患者；NCT01416181）和DECIDE（Kappos等，2015）（1841例复发缓解型MS患者；NCT01064401）。纳入标准和MRI采集协议已在既往研究中描述；表1总结了相关统计数据。简而言之，患者在基线期及一系列随访期间完成MRI扫描，采集序列包括：T1加权（钆注射前后）、T2加权和质子密度加权（PDw）。ADVANCE随访MRI在基线后24、48和96周采集，ASCEND在基线后24、48、72、96、108和156周采集，DECIDE在基线后24和96周采集。本研究的患者纳入受限于MRI数据的可获得性。

Conclusion

结论

In this paper, we have focused on the estimation of acute MSlesion activity from single-timepoint conventional MRI. While this taskhas traditionally been tackled via the detection of GdE lesions, wehave proposed the new task of recent (less than 24-weeks old) T2lesion detection. We have demonstrated the clinical relevance of thisnewly proposed task by showing that quantification of recent MSlesion activity that is no longer GdE improves the prediction of futureinflammatory lesion activity, when used in conjunction with traditionalmeasures of GdE lesion activity. Moreover, we have developed andvalidated several models to establish a benchmark of achievable performances on our newly proposed task. In particular, our UNet-2D canidentify gadolinium-negative subjects with recent (less than 24-weeks)acute inflammatory lesion activity with high accuracy (ROC AUC inthe range 80–84% on independent RRMS datasets) and improves theprognostication of future inflammatory lesion activity. This could beexploited as a population enrichment strategy for identifying patientsmost or least likely to show disease activity during the course of aclinical trial.In the wake of recent major advances of the MS therapy armamentarium where 3 FDA-approved anti-CD20 treatments are associatedwith a complete silencing of acute inflammation, the next generation ofdrug development need is turning towards CNS pathways susceptible tocausally drive disability progression independent of acute inflammationalso called smoldering-associated worsening. As such, methods able toidentify subjects at risk of future acute inflammatory disease activity(such as models proposed in this paper) are likely to be valuable indesigning inclusion/exclusion criteria and/or enrichment strategies forfuture trials (e.g., via exclusion of subjects at risk of acute inflammation). In this respect, the output of our models could be used as a drugdevelopment tool as defined by the U. S. Food and Drug Administration(2024), i.e. methods, materials, or measures that have the potentialto facilitate drug development, such as a biomarker used for betterpatient stratification at baseline or trial enrichment. Such models mayprove to be a necessary step to augment and de-risk the probability oftechnical and regulatory success for novel drugs to be evaluated as totheir capacity to stop the ‘true MS progression’ related to intra-CNScompartmentalized pathobiology, unconfounded by a potential drugeffect on peripheral acute inflammation.

在本文中，我们聚焦于通过单次常规MRI估算多发性硬化症（MS）的急性病变活动。尽管传统上该任务通过检测钆增强（GdE）病变来解决，但我们提出了新的任务：检测近期（小于24周）的T2病变。我们已证明这一新任务的临床相关性——当与传统的GdE病变活动测量结合使用时，对不再具有GdE的近期MS病变活动进行量化，可改善对未来炎症性病变活动的预测。此外，我们开发并验证了多个模型，为这一新任务确立了可实现的性能基准。特别是，我们的2D-UNet能够高精度识别具有近期（小于24周）急性炎症性病变活动的钆阴性受试者（在独立的复发缓解型MS数据集上，ROC曲线下面积[AUC]为80-84%），并改善对未来炎症性病变活动的预后预测。这可作为一种人群富集策略，用于在临床试验过程中识别最可能或最不可能出现疾病活动的患者。   鉴于MS治疗领域的最新重大进展——3种FDA批准的抗CD20疗法可完全抑制急性炎症，下一代药物开发需求正转向中枢神经系统（CNS）通路，这些通路可能在不依赖急性炎症的情况下驱动残疾进展（也称为“缓慢进展相关恶化”）。因此，能够识别未来有急性炎症性疾病活动风险的受试者的方法（如本文提出的模型），可能在设计未来试验的纳入/排除标准和/或富集策略中具有重要价值（例如，通过排除有急性炎症风险的受试者）。在这方面，我们模型的输出可作为美国食品药品监督管理局（2024年）定义的药物开发工具，即有可能促进药物开发的方法、材料或措施，例如用于在基线时更好地对患者进行分层或试验富集的生物标志物。这类模型可能是一个必要步骤，以增强和降低新型药物的技术和监管成功概率，从而评估其阻止与CNS内区室化病理生物学相关的“真正MS进展”的能力，避免被药物对外周急性炎症的潜在影响所干扰。

Results

结果

4.1. Performance evaluation

We report subject-level classification results stratified by trial usingpredictions from the testing set, on the task of identifying subjects withacute MS lesions, defined as new T2 lesions that are less than 24-weeksold. Since GdE lesions are clearly visible on post-contrast T1w MRI,all models achieved near-perfect classification on GdE-positive subjects,with sensitivity exceeding 95%. We thus focus on subjects that did notshow GdE lesions such as to investigate the ability of our proposedmodels to specifically detect sub-acute MS lesion activity. Classificationmetrics are presented in Fig. 6 and Receiver Operating Characteristic(ROC) curves are shown in Fig. 7.

4.1 性能评估   我们使用测试集的预测结果，按试验分层报告受试者水平的分类结果，任务为识别具有急性MS病变（定义为小于24周的新发T2病变）的受试者。由于钆增强（GdE）病变在对比剂注射后的T1w MRI上清晰可见，所有模型在GdE阳性受试者中均实现了近乎完美的分类，敏感性超过95%。因此，我们聚焦于无GdE病变的受试者，以探究所提出模型特异性检测亚急性MS病变活动的能力。分类指标如图6所示，受试者工作特征（ROC）曲线如图7所示。

Figure

图

Fig. 1. Illustration of contrast-enhanced T1w and T2w brain MRI scans acquired atbaseline, week 4 and week 24 from a subject from ADVANCE. The cross-hair shows aT2 lesion that is new at week 24 relative to baseline and which only shows gadoliniumenhancement at week 4. This lesion is considered sub-acute at week 24 (less than 24-weeks old yet no longer GdE).

图1. 来自ADVANCE队列某受试者在基线、第4周和第24周采集的对比增强T1w和T2w脑MRI扫描示意图。十字标记显示了一个在第24周相对于基线的新发T2病变，该病变仅在第4周出现钆增强。该病变在第24周被视为亚急性（年龄小于24周但不再有钆增强）。

Fig. 2. Example of brain MRI scans and lesion segmentation masks for a representative participant of the ADVANCE cohort

图2. ADVANCE队列中一名代表性参与者的脑MRI扫描及病变分割掩码示例

Fig. 3. Task overview: predict whole-brain acuteness score from single-timepoint MRI.

图3. 任务概述：基于单次MRI预测全脑急性评分

Fig. 4. Network architecture of the full-brain transformer. It is composed of one ResNetdown-sampling block followed by 10 ResNet blocks. A class attention layer aggregatesinformation across patches and a fully-connected layer generates the final score

图4. 全脑Transformer的网络架构。该架构由一个ResNet下采样模块和10个ResNet模块组成，通过类别注意力层聚合各图像块的信息，最终由全连接层生成评分。

Fig. 5. Architecture of the lesion-based transformer. Patches are extracted around each lesion bounding box and resized to a target shape. A ResNet-like network produces patchembeddings. Dependencies between patches are modeled through a transformer block. A classifier head produces scores at the patch-level, while a class attention transformer blockcombines the patch embeddings to generate a prediction at the brain-level. The figure is represented in 2D here for clarity, but the volumes are processed in 3D in practice.

图5. 基于病变的Transformer架构示意图。围绕每个病变边界框提取图像块并调整为目标尺寸，通过类ResNet网络生成块嵌入。块间依赖关系由Transformer模块建模，分类头生成块级评分，同时通过类别注意力Transformer模块融合块嵌入以生成脑级预测。为清晰起见图中以2D形式展示，实际处理时采用3D体积数据。

Fig. 6. Classification metrics for single-timepoint identification of patients with acutelesions (<24 weeks-old) for the subset of patients without GdE lesions. The ensemblecombines the naive classifier, the UNet-2D, the full-brain transformer and the lesionbased transformer. Error bars show the 95% confidence interval across the 10 folds.See 11 for results of the lesion-based radiomics classifier

图6. 针对无钆增强（GdE）病变患者亚组，基于单次扫描识别急性病变（年龄<24周）的分类指标。集成模型结合了朴素分类器、2D-UNet、全脑Transformer和基于病变的Transformer。误差棒表示10折交叉验证的95%置信区间。基于病变的放射组学分类器结果见补充材料11。

Fig. 7. ROC curves for different models, computed among gadolinium-negative subjectsfrom the test set and grouped by trial. The area under the ROC curve (AUC) of thebest model (UNet-2D) is 84% in ADVANCE, 73% in the placebo arm of ASCEND, and80% in DECIDE

图7. 不同模型在测试集钆阴性受试者中的ROC曲线（按试验分组）。最佳模型（2D-UNet）的ROC曲线下面积（AUC）在ADVANCE中为84%，ASCEND安慰剂组中为73%，DECIDE中为80%。

Fig. 8. Balanced accuracy (and 95% CI) among gadolinium-negative subjects fromDECIDE using various combinations of input sequences. Results are given for the first ofthe 10 folds. Abbreviations: WMH: white matter hyperintensity map (i.e., T2 lesions);T1w: T1-weighted MRI; T2w: T2-weighted MRI; T1c: T1-weighted MRI post-gadolinium;PDw: proton density-weighted MRI

图8. 使用不同输入序列组合时，DECIDE队列中钆阴性受试者的平衡准确率（及95%置信区间）。结果为10折交叉验证中第一折的结果。缩写： WMH：白质高信号图（即T2病变）；T1w：T1加权MRI；T2w：T2加权MRI；T1c：钆增强后T1加权MRI；PDw：质子密度加权MRI。

Fig. 9. Visual interpretability maps, showing the segmentation map from the UNet-2D as well as the class attention maps and local predicted acuteness scores from the full-brainand lesion-based transformer models. Case 6 illustrates the robustness of the lesion-based transformer against local false positives: while several lesions were incorrectly predictedas acute, attention mechanisms were able to ignore incorrect local predictions and correctly classify the brain as non-acute

图9. 可视化解释图，展示了2D-UNet的分割图，以及全脑Transformer和基于病变的Transformer模型的类别注意力图和局部预测急性评分。案例6显示了基于病变的Transformer对局部假阳性的鲁棒性：尽管多个病变被错误预测为急性，但注意力机制能够忽略错误的局部预测，并正确将全脑分类为非急性。

Table

表

Table 1Demographics and lesion statistics of the participants from the ADVANCE, ASCEND and DECIDE cohorts analyzed in this paper. For the age and the brain volumes, we report themean standard deviation computed across subjects and visits, respectively

表1 本研究分析的ADVANCE、ASCEND和DECIDE队列参与者的人口统计学和病变统计数据。对于年龄和脑容量，分别报告了跨受试者和随访时间点计算的均值±标准差。

Table 2Classification results for the 6 cases shown in Fig. 9. TP, TN, FP and FN stand for truepositive, true negative, false positive and false negative, respectively.

表2 图9所示6个案例的分类结果。其中TP、TN、FP和FN分别代表真阳性、真阴性、假阳性和假阴性。

Table 3Regression results on the full model of predicting the NET2 lesion count at week 48relative to week 24 with both GdE and non-enhancing portion of the predicted NET2lesion volume as a predictor

表3 以预测的NET2病变体积中钆增强（GdE）和非增强部分为预测因子，对第48周相对于第24周NET2病变计数的完整模型回归结果