Heart Disease Analysis from Medical Reports with DistilRoBERTa & BETO

Introduction

Heart disease is one of the leading causes of death worldwide. At the same time, a lot of useful information hospitals already collect sits locked away inside written test reports that are rarely analysed in bulk.

Echocardiogram reports are full of valuable clues about the heart, but reading every one by hand is slow and easy to slip up on. Language-understanding software can read this text automatically, helping flag concerning findings early and supporting faster decisions.

This project uses modern language models to sort heart-condition severity directly from echocardiogram report text, working with the everyday medical writing clinicians already produce.

Problem Statement

Most traditional tools expect neat, number-based inputs like age or blood pressure. Real hospitals produce something messier: long written reports full of medical terms, shorthand, and context. Key challenges include:

1. The information lives in free-form text: not tidy tables.
2. Some severity levels appear far more often than others.
3. Reports contain missing values: inconsistent formatting, and varied wording.
4. Any prediction has to be trustworthy enough to assist clinicians: not replace them.

Objectives

1. Sort heart-condition categories from echocardiogram report text.
2. Compare a few modern language-understanding models on the same data.
3. Prepare the medical text carefully so models can read it well.
4. Judge the models on measures that matter in a diagnostic setting.
5. Make the best model usable through a simple web tool for clinicians.

Dataset

The work is built around a set of echocardiogram reports. Depending on the client, this can be licensed or private hospital records, a public medical collection, data from other trusted sources, or a custom set gathered for the project. Reports are cleaned of missing and inconsistent entries and prepared for the models, with patient privacy respected throughout.

Methodology

The report text is cleaned and prepared, then three language models are trained to read it and assign a severity category: DistilRoBERTa, DistilBERT, and BETO. Each is trained and tested the same way so the comparison stays fair.

The novelty is applying context-aware language models directly to real medical report writing, rather than reducing everything to a handful of numbers. Because these models grasp meaning from wording, telling a mild note apart from a serious finding, they make better use of the detail clinicians already record, turning existing reports into an early-warning aid.