Evaluation of PORT / PSI and SOFA scores in predicting in-hospital mortality of patients with COVID-19 1

Introduction: There is limited information analyzing the utility of different prognostic scores in predicting inhospital mortality among patients with COVID-19. This study aimed to evaluate the performance of PORT/PSI and SOFA scores in predicting the in-hospital mortality of patients with COVID-19. Material and methods: This was an observational, analytical, and retrospective study that included consecutive patients hospitalized for COVID-19 from April 1, 2020, to May 31, 2020. The study population was characterized, and ROC analysis was performed and used to calculate the area under the curve of PORT/PSI and SOFA scores as well as the sensitivity, specificity, and predictive values. Results: A total of 151 patients were included, with a median age of 52 years (IQR 45-64); 69.5% 2 were men, with a median BMI of 29.3 kg/m (IQR 25.534.7). Of the total, 102 patients died during hospitalization (67.5%). The areas under the ROC curves for predicting inhospital mortality were 0.74 (95% CI 0.67-0.81) for the SOFA score and 0.85 (95% CI 0.78-0.90) for the PORT/PSI score. When compared, the PORT/PSI score predicted mortality significantly better than the SOFA score (p: 0.01). Conclusions: The PORT/PSI score is a good tool to predict in-hospital mortality in patients with COVID-19.


Introduction
Since the emergence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in Wuhan, China, in December 2019, viral pneumonia has become a significant public health issue (1). The disease caused by this virus, called coronavirus disease 2019 (COVID-19), is characterized by a wide range of symptoms, from mild to severe, with acute respiratory distress syndrome being its main complication (2). Therefore, assessing the severity of COVID-19 at the time of hospital admission acquired fundamental importance during the pandemic (1).
Identifying patients with a poor prognosis at hospital admission is essential to help guide rapid treatment and optimize the use of a medical unit's resources. Several prognostic scales have been used to identify a high risk of mortality in patients with community-acquired pneumonia [CAP] (2).
The PORT/PSI score (Pneumonia Severity Index) is a prediction score that determines the prognosis of CAP. Patients are stratified into five classes according to their score. All patients with a score > 90 should be hospitalized, while patients with a score > 130 require intensive care to obtain better outcomes [ Evaluation of PORT/PSI and SOFA scores in predicting in-hospital mortality of patients with COVID-19 Table 1. PORT/ PSI score The SOFA (Sequential Organ Failure Assessment) score assesses the level of dysfunction of six organ systems: the respiratory, circulatory, renal, hematological, hepatic, and nervous systems. The tool uses six criteria that reflect the function of each system, and scores from 0 to 4 are assigned [ Table 2] (4-8). A SOFA score 2 reflects significant organ dysfunction and an increased risk of adverse outcomes in patients with sepsis (9).
In a literature review, information is limited on the application of prognostic scores to predict in-hospital mortality in patients with COVID-19 or other pneumonias of viral etiology. Therefore, in this study, our objective was to compare the accuracy of the PORT/PSI and SOFA scores to predict mortality in hospitalized patients with COVID-19.

Material and methods
This is a retrospective cohort study from a hospital in Mexico City. Our hospital has operated as a COVID-19 hospital since March 2020. During the study period, the ICU capacity was only seven beds, but the capacity of the hospital's units was expanded from 36 to 72 beds (all equipped for invasive mechanical ventilation). In our unit, any patient suspected of h a v i n g C O V I D -1 9 w i t h r e s p i r a t o r y d i s t r e s s ( > 3 0 breaths/minute) or oxygen saturation lower than <90% in ambient air was hospitalized.
Consecutive hospitalized patients diagnosed with SARS-CoV-2 pneumonia between April 1, 2020, and May 31, 2020, were included in this study. The inclusion criteria were as follows: (1) patients older than 18 years; (2) patients with SARS-CoV-2 infection confirmed by PCR and with alterations in radiographic studies (infiltrates and/or ground glass pattern); (3) patients treated exclusively by the internal medicine service; and (4) patients discharged from the hospital due to either death or clinical improvement. The exclusion criteria were as follows: (1) pregnancy, (2) incomplete clinical records, and (3) incomplete information for calculating the SOFA and PORT/PSI scores. The following were collected from the medical records: demographic characteristics, comorbidities, days of hospitalization, and time until discharge. We retrospectively calculated the PORT/PSI and SOFA scores and compiled the results in a database. The study's primary outcome was in-hospital mortality, defined as documented death from any cause during hospitalization. In addition, the patients were classified into subgroups: survivors vs. nonsurvivors.
Statistical analysis was performed using the IBM SPSS Statistics 22 system. Categorical data were reported as proportions and counts, and continuous data were presented as medians and interquartile ranges (IQRs) unless the data were normally distributed. We compared the differences between the categorical variables using the chi-square test, and we evaluated the continuous variables using Student's ttest or analysis of variance.
The ROC curve was used to assess the predictive value of each scoring system, and the Z test was used for the area under the ROC curve (AUC). An AUC > 0.8 was considered a good performance. To evaluate the precision of each prognostic score, the cutoff point with the best results of the ROC curves was selected to determine the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). A value of p <0.05 was accepted as statistically significant.
The study was conducted following the Declaration of Helsinki. The Institutional Ethics Committee approved the study protocol (501-010-01-21, CEI-1-2021), with an exemption from the requirement of informed consent, as it was a risk-free and retrospective investigation. Furthermore, patients' privacy and personally identifiable information were protected, and data collection did not harm the patient.

Results
A total of 151 patients were included in this study. Epidemiolo-gical and clinical data are shown in Table 3.

Treatments, complications and clinical outcomes
Of this population (n = 151), 49 patients were discharged due to clinical improvement (33%), and 102 (67%) died during hospitalization. The patients had a median hospital stay of 7 days (IQR: 4-11). As expected, the patients who died were older, less educated, more likely to have diabetes, and experienced more complications. The values of the prognostic scores were higher in the deceased patients, as shown in Table 3. The patients who died also had shorter hospital stays than the survivors (6 (IQR 4-9) vs. 10 (IQR 7-15), p <0.001).

SOFA score
Of the study population, 15 patients (9.9%) had a SOFA score of 0 to 1, 110 patients (72.8%) had a score of 2-3, 19 patients (12.6%) had a score of 4-5 and 7 patients (4.7%) had a score 6. Mortality was 20% among those with a score of 0 to 1, 62.3% among those with a score of 2-3, 94.7% among those with a score of 4-5 and 100% among those with a score 6.
To calculate the sensitivity, specificity, and predictive values, a cutoff point of >2 points was used (the best results were obtained with this cutoff point). A SOFA score >2 at hospital admission had an average performance in predicting in-hospital mortality, with a sensitivity of 55.8%, specificity 81.6%, PPV of 43.2% and NPV of 88.1% (AUC: 0.74, CI 95% 0.67-0.81; p <0.001) ( Table 4).

PORT/PSI score
In this population, 22 patients (14.6%) were placed in group I, 39 patients (25.8%) in group II, 26 patients (17.2%) in group III, 46 patients (30.5%) in group IV, and 18 patients (11.9%) in group V. Mortality was 13.6% for those in group I, 48.7% for those in group II, 84.6% for those in group III, 91.3% for those in group IV and 88.8% for those in group V.

Table 4. Discriminative accuracy of the PORT/PSI and SOFA scores in predicting in-hospital mortality
A comparison of AUCs for the prediction of inhospital mortality showed that the PORT/PSI model predicted mortality significantly better than the SOFA model (difference AUC 0.1, 95% CI 0.02-0.18; p: 0.01) (Figure 1).

Discussion
To our knowledge, this is the first study in Mexico that evaluates two specific predictive scores for CAP to predict mortality in patients hospitalized with COVID-19. Hospital mortality in our study was high (67%), which was higher than that reported in other studies (10)(11)(12)(13)(14). This could be because during the study period, the criterion for hospital admission was supplemental oxygen requirement, which is why our research concentrates on a higher number of seriously or critically ill patients than previous studies.
We observed that the PORT/PSI score underestimated the severity of disease in patients with COVID-19; therefore, we lowered the cutoff point to obtain better results. This phenomenon could be explained by the so-called "silent hypoxemia" observed in patients with SARS-CoV-2 pneumonia, where severe hypoxemia with low respiratory rates could mask the severity of pneumonia (15).
In our study, the performance of PORT/PSI was slightly superior to that shown by Artero et al. in a study with 10,238 patients, with an AUC of 0.83 for in-hospital mortality (2). Our results were similar to those found by Fan  by which to assess the risk of in-hospital mortality in patients with COVID-19.
Information is scarce regarding the application of the SOFA score in COVID-19 patients. Raschke et al., in a retrospective study of 675 patients with COVID-19 requiring mechanical ventilation, reported an AUC of 0.59 for hospital mortality (19). Our AUC was higher than that of Raschke et al.; nevertheless, we showed that the SOFA score possesses inadequate discriminant accuracy to be used in COVID-19 patients.
The study has some limitations: 1) The sample size is small compared to those of previous studies; thus, our study has a greater probability of sampling error. 2) The study is a single-center study that is not externally validated and is retrospective and observational in its design. 3) Due to problems of limited logistics and resources, patient follow-up was limited only to hospitalization, so the association of prognostic scores with mortality after patient discharge due to clinical improvement was not evaluated.

Conclusions
The PORT/PSI score performed better than the SOFA score in predicting in-hospital mortality in patients with COVID-19. The PORT/PSI score could underestimate the severity of disease in patients with COVID-19, but it is a good tool with which to predict in-hospital mortality upon admission to the hospital.
Funding: This study did not benefit from any funding organization in the public or commercial.