Oral Presentation NSW State Cancer Conference 2023

Extracting Performance Status from Clinical Notes: Application Across Multiple Cancer Sites (#53)

Farhannah Aly 1 2 3 , Gui Xiong 2 3 , Tim Churches 1 3 , Mahbuba Sharmin 2 3 , Angela Berthelsen 2 3 , Nasreen Kaadan 2 3 , Lois Holloway 1 2 3 4 , Shalini Vinod 1 2 , Merran Findlay 1 5 6 7 8 , Georgina Kennedy 1 3 5
  1. South Western Sydney Clinical School, University of NSW, Sydney, NSW, Australia
  2. Liverpool and Macarthur Cancer Therapy Centres, South Western Sydney Local Health District, Liverpool, NSW, Australia
  3. Ingham Institute, Liverpool, NSW, Australia
  4. Institute of Medical Physics, School of Physics, University of Sydney, Sydney, NSW, Australia
  5. Maridulu Budyari Gumal (SPHERE) Cancer Clinical Academic Group, Sydney, NSW, Australia
  6. Cancer Services, Royal Prince Alfred Hospital, Sydney Local Health District, Sydney, NSW, Australia
  7. Chris O'Brien Lifehouse, Sydney, NSW, Australia
  8. The Daffodil Centre, The University of Sydney, a joint venture with Cancer Council, Sydney, NSW, Australia

Background: The Eastern Cooperative Oncology Group (ECOG) Performance Status Score is an important measure of a cancer patient’s functional status. It is useful for prognosis, treatment decisions, and it plays a vital role in determining eligibility for clinical trials. Despite its significance, it is not universally recorded in structured data fields.

 

Aim: To assess feasibility of accurately extracting ECOG scores from free-text clinical notes using Natural Language Processing (NLP) in a head and neck cancer (HNC) cohort, evaluating transferability of these NLP tools to a lung cancer cohort.

 

Methods: Clinical notes and structured data were extracted for a cohort of 529 patients with primary HNC and a comparison cohort of 1547 lung cancer (LC) patients, (diagnosis 2010-2018). To address class imbalance an optimistic text filter was applied to identify potential mentions. From this, a balanced set of notes was randomly selected for annotation, then abstracted by expert annotators and checked for inter-annotator agreement.

 

A spaCy matcher pipeline was created using a subset of HNC notes and then evaluated in the remainder HNC and full LC sample. This pipeline was evaluated for its accuracy in identifying the exact tokens required to extract the full ECOG status, and performance when extracting the numeric ECOG value.

 

Results: For the combined HNC and LC cohorts, 1490 (72%) had ECOG score recorded in the structured fields. These patients had an average of 100 individual clinical notes from all sources (range 12 to 389), varying from short scheduling comments to full clinical histories (mean length 431 characters).  ECOG scores could be extracted for an additional 257 patients, increasing total data availability to 84%. ECOG score (text or structured) was unrecorded for more HNC (43%) than LC (7%) patients.

 

Inter-annotator agreement was perfect (Cohen’s Kappa score=1.0 for all subtasks). Accuracy of extracted tokens and values were also nearly perfect. ECOG score identification had an F1 score of 0.999 (HNC) and 0.993 (LC). For extracted ECOG value in range 0-4, micro-F1 scores were 0.998 and 0.989 for HNC and LC cohorts respectively.

Conclusion: ECOG is a straightforward yet high value target for NLP extraction. Using this pipeline, we increased ECOG status availability with a high degree of accuracy. Results are transferrable across clinical sites and fit for use in both clinical and quality-tracking use-cases. This pipeline will be extended to capture similar nutrition outcome-related scores, as well as more complex targets such as smoking status and treatment toxicities.