External validation of the Chicago Chiari Outcome Scale.

Introduction: Historically, assessment of clinical outcomes following surgical management of Chiari malformation type 1 (CM-1) has been challenging due to the lack of a validated instrument for widespread use. The Chicago Chiari Outcome Scale (CCOS) is a novel system intended to provide a less subjective evaluation of outcomes for patients with CM-1. The goal of this study was to externally validate the performance of the CCOS.

Methods: Patients undergoing surgery for CM-1 from 2001 to 2012 were reviewed (n=292). Inclusion criteria for this study were: 1) patients receiving primary posterior fossa decompression; 2) at least 5.5 months of post-operative clinical follow-up; and 3) patients ≤ 18 years of age at the time of surgery. Outcomes were evaluated using the CCOS, along with a “gestalt” impression of whether patients experienced significant improvement after surgery. A subgroup of consecutive patients undergoing operations from 2008 to 2010 (n=118) was selected for analysis of interrater reliability (n=73 meeting inclusion/exclusion criteria). In this subgroup, gestalt and CCOS were independently scored by two reviewers, and interrater reliability was assessed using the intraclass correlation coefficient (ICC) and kappa (κ) statistic.

Results: The median CCOS was 14, and 67% of patients had improved gestalt scores after surgery. Overall, the CCOS was effective at identifying patients with improved outcome after surgery (AUC=0.951). The interrater reliability of the CCOS (ICC=0.71) was high, though the reliability of the component scores ranged from poor to good (ICC 0.23 to 0.89). The functionality subscore demonstrated a low ICC and did not add to the predictive ability of the logistic regression model (Likelihood Rate = 1.8, p=0.18). When analyzing gestalt outcome, there was moderate agreement between raters (κ=0.56).

Conclusions: In this external validation study, the CCOS was effective at identifying patients with improved outcomes and proved more reliable than our gestalt impression of outcome. However, certain component subscores (functionality and non-pain symptoms) were found to be less reliable, and may benefit from further definition in score assignment. In particular, the functionality subscore does not add to the predictive ability of the CCOS, and may be unnecessary. Overall, we found the CCOS to be an improvement over the previously utilized assessment of outcome at our institution.

Full Publication