Transformers for cardiac patient mortality risk prediction from heterogeneous electronic health records

Emmi Antikainen (Corresponding Author), Joonas Linnosmaa, Adil Umer, Niku Oksala, Markku Eskola, Mark van Gils, Jussi Hernesniemi, Moncef Gabbouj

Research output: Contribution to journalArticleScientificpeer-review

2 Citations (Scopus)
41 Downloads (Pure)


With over 17 million annual deaths, cardiovascular diseases (CVDs) dominate the cause of death statistics. CVDs can deteriorate the quality of life drastically and even cause sudden death, all the while inducing massive healthcare costs. This work studied state-of-the-art deep learning techniques to predict increased risk of death in CVD patients, building on the electronic health records (EHR) of over 23,000 cardiac patients. Taking into account the usefulness of the prediction for chronic disease patients, a prediction period of six months was selected. Two major transformer models that rely on learning bidirectional dependencies in sequential data, BERT and XLNet, were trained and compared. To our knowledge, the presented work is the first to apply XLNet on EHR data to predict mortality. The patient histories were formulated as time series consisting of varying types of clinical events, thus enabling the model to learn increasingly complex temporal dependencies. BERT and XLNet achieved an average area under the receiver operating characteristic curve (AUC) of 75.5% and 76.0%, respectively. XLNet surpassed BERT in recall by 9.8%, suggesting that it captures more positive cases than BERT, which is the main focus of recent research on EHRs and transformers.
Original languageEnglish
Article number3517
Pages (from-to)3517
Number of pages1
JournalScientific Reports
Issue number1
Publication statusPublished - Dec 2023
MoE publication typeA1 Journal article-refereed


  • Humans
  • Electronic Health Records
  • Quality of Life
  • Cardiovascular Diseases
  • Death, Sudden
  • Electric Power Supplies


Dive into the research topics of 'Transformers for cardiac patient mortality risk prediction from heterogeneous electronic health records'. Together they form a unique fingerprint.

Cite this