Agreement of the 2020 Brazilian Society of Cardiology cardiac risk stratification protocol in cardiac rehabilitation programs: a cross-sectional studySUMMARY
BACKGROUND: Cardiac risk stratification protocols are widely used in cardiac rehabilitation programs, but divergent classifications limit their applicability. In 2020, the Brazilian Society of Cardiology updated its protocol; however, its agreement with established protocols and inter-rater reliability in cardiac rehabilitation programs remain unclear.
OBJECTIVE: The aim of this study was to evaluate the agreement between the 2020 Brazilian Society of Cardiology cardiac risk stratification protocol and protocols used in clinical practice and to assess inter-rater agreement in its application.
DESIGN AND SETTING: Cross-sectional observational study conducted in a cardiac rehabilitation program at a public university in Brazil.
METHODS: Medical records of 51 cardiac rehabilitation programs participants were independently analyzed by two cardiovascular physical therapists using the 2020 Brazilian Society of Cardiology, American Association of Cardiovascular and Pulmonary Rehabilitation, American College of Sports Medicine, Pashkow, and 2013 Brazilian Society of Cardiology protocols. Agreement between protocols and inter-rater reliability was assessed using the Kappa coefficient.
RESULTS: The 2020 Brazilian Society of Cardiology protocol showed moderate agreement with Pashkow (k=0.544; 95%CI 0.340–0.748; p<0.001) and the 2013 Brazilian Society of Cardiology protocol (k=0.708; 95%CI 0.498–0.918; p<0.001), good-to-excellent agreement with American Association of Cardiovascular and Pulmonary Rehabilitation (k=0.780; 95%CI 0.604–0.956; p<0.001), and weak, nonsignificant agreement with American College of Sports Medicine (k=0.003; 95%CI -0.015 to 0.021 p=0.866). Inter-rater analysis demonstrated excellent agreement between evaluators (k=0.843; 95%CI 0.672–1.014; p<0.001).
CONCLUSION: The findings provide preliminary evidence regarding the agreement and reproducibility of the 2020 Brazilian Society of Cardiology protocol compared with established cardiac rehabilitation programs protocols; however, further studies are needed to evaluate its predictive validity for exercise-related events.
KEYWORDS:
Exercise; Rehabilitation; Cardiac rehabilitation; Risk assessment
INTRODUCTION
Exercise-based cardiac rehabilitation programs (CRPs) are class I, level A interventions for preventing and treating cardiovascular diseases, the leading cause of death worldwide1. Adverse events during supervised exercise, from minor clinical signs to severe complications such as acute myocardial infarction and cardiorespiratory arrest, have been reported2,3,4,5,6.
Risk stratification protocols classify patients as high, moderate, or low risk7,8,9 and are mostly based on complementary test variables associated with morbidity and mortality10. Despite limited exercise-specific validation, these protocols guide CRP professionals in monitoring and exercise prescription8.
Silva et al.9 identified seven protocols used in CRPs: American College of Sports Medicine (ACSM)11, Brazilian Society of Cardiology (BSC)12, Association of Cardiovascular and Pulmonary Rehabilitation (AACVPR)13, American Heart Association (AHA)14, Société Française de Cardiologie15, Spanish Society of Cardiology16, and Pashkow17. Classification criteria vary, complicating protocol selection and reliability8,9,18. Santos et al.8 reported low-to-moderate agreement among protocols, suggesting divergent classifications for the same patient and limiting clinical applicability8,9,10.
Inter-rater agreement studies indicate moderate-to-good consistency18, with disagreements related to hemodynamic standards, rhythm terminology, left ventricular dysfunction criteria, variability in clinical interpretation, and limited familiarity with some protocols.
METHODS
Study design and ethical aspects
This cross-sectional observational study was conducted in 2024, following the Strengthening the reporting of observational studies in epidemiology (STROBE) guidelines19. Data were obtained from the medical records of 51 CRP participants at São Paulo State University (UNESP), Presidente Prudente, Brazil. All participants provided informed consent. The study was approved by the institutional Research Ethics Board (CAAE: 78227324.8.0000.5402).
Participants
Sample size was calculated based on a minimum expected Kappa coefficient of 0.40, according to recommendations for agreement studies20,21, 5% error, and 80% power, yielding a minimum sample size of 42 participants, which was increased to 47 to account for potential dropouts20,21. Data extracted included sex, age, body mass index (BMI), and primary clinical diagnosis. Participants without cardiovascular disease were assigned to the prevention group.
Risk stratification
Stratification used clinical history, risk factors, signs/symptoms, and complementary/laboratory tests (most recent available), including blood glucose, lipids, and cardiovascular assessments. The 2020 BSC protocol1 was compared with AACVPR13, ACSM11, Pashkow17, and 2013 BSC12, selected for clinical relevance8,9,10,22. Two cardiovascular physical therapists independently performed stratification; participant identifiers were removed to ensure blinding. Patients were classified as low, moderate, high risk, or unstratified.
Protocol summaries
2020 BSC1
Stratifies risk based on time since cardiovascular event, intervention, or decompensation; functional capacity via exercise stress testing and/or ergospirometry; ischemic signs or symptoms at low workloads; heart failure or angina symptoms; additional clinical characteristics (e.g., chronic renal failure, oxygen desaturation, complex ventricular arrhythmia); and clinician judgment in pre-participation evaluation.
AACVPR13
Uses complementary tests, especially maximal exercise testing, to identify ischemic or other symptoms during exercise or recovery. Recommended for patients with a history of Acute Myocardial Infarction (AMI); absence of testing can complicate classification. Previously associated with prediction of complications during CRP.
ACSM11
Stratifies based on cardiovascular, metabolic, or respiratory risk factors and signs or symptoms; complementary test results are not required. Positive risk factors include family history of AMI, coronary revascularization, or sudden death; smoking; hypertension; dyslipidemia; fasting glucose >100 mg/dL; obesity; and inactivity. High-Density Lipoprotein Cholesterol >60 mg/dL is a negative risk factor. Signs assessed include angina-equivalent pain, dyspnea, syncope, edema, palpitations, claudication, heart murmur, and unusual fatigue.
Pashkow17
Developed for patients with a history of AMI. Uses complementary tests, including progressive exercise stress testing, electrocardiogram, and echocardiogram, to stratify risk. Known for its highest concordance with other protocols and strong inter-protocol agreement.
2013 BSC12
Based primarily on maximal exercise testing to identify ischemia, ventricular dysfunction, arrhythmias, and conduction disorders. Classifies patients as low, moderate, or high risk, recommending reassessment every 6–12 months. Any criterion classified as moderate or high risk determines the overall risk category.
Statistical analysis
Descriptive statistics characterized the sample (mean±standard deviation or frequencies). Normality was tested with Shapiro-Wilk. Agreement between protocols and inter-rater reliability used Kappa and their respective 95% confidence intervals (k<0.40 weak; 0.40–0.75 moderate; >0.75 excellent). Analyses were conducted in Statistical Package for the Social Sciences v22.0; significance was set at p<0.05.
RESULTS
This study analyzed data from 51 participants enrolled in a CRP, with no losses during the risk stratification process. As shown in Table 1, the sample comprised older adults of both sexes, with a mean age of 69.37±10.82 years and a mean BMI of 27.85±4.08 kg/m2, indicating an overweight status. Coronary insufficiency and AMI were the most prevalent diagnoses, observed in 19 and 11 participants, respectively.
Medical records included several complementary and laboratory examinations (Table 1), with exercise stress tests and echocardiograms being the most frequent, reported in 37 and 32 records, respectively. Table 2 presents the risk classification distribution for each protocol and the number of unstratified participants.
Patient risk classification by analyzed protocols and agreement with the 2020 Brazilian Society of Cardiology protocol.
Agreement between the 2020 BSC protocol and the other protocols is presented in Table 2. Significant moderate agreement was observed with the Pashkow and 2013 BSC protocols and significant excellent agreement with the AACVPR protocol. No significant agreement was found with the ACSM protocol, which showed weak concordance.
Inter-evaluator agreement for the updated BSC protocol is shown in Table 3. Three discrepancies were identified, mainly involving the interpretation of moderate-risk criteria or classification as unstratified in cases with incomplete complementary examinations. Four participants (7.84%) could not be stratified due to missing complementary examinations. Overall, the protocol demonstrated excellent and significant inter-rater agreement (κ=0.843; 95%CI 0.672–1.014; p<0.001).
Inter-rater agreement in risk stratification using the 2020 Brazilian Society of Cardiology protocol.
DISCUSSION
The main findings indicate that the 2020 BSC protocol showed moderate agreement with the Pashkow and previous BSC protocols, excellent agreement with the AACVPR protocol, and weak agreement with the ACSM protocol. Additionally, the updated BSC protocol demonstrated excellent inter-rater agreement (κ=0.84), likely reflecting greater objectivity and clearer criteria than the 2013 version (κ=0.73)18. Despite the overall clarity of the protocol, the item “other clinical characteristics” remains partially subjective.
To our knowledge, this is the first study to assess inter-rater agreement and concordance between the 2020 BSC protocol and other cardiac risk stratification protocols in CRP participants. The 2020 BSC protocol failed to stratify 7.84% of participants, compared with 9.80% for the AACVPR, Pashkow, and 2013 BSC protocols, mainly due to missing complementary examinations23.
The lack of agreement between the ACSM and 2020 BSC protocols should be interpreted considering their different conceptual approaches. Unlike protocols developed for supervised cardiac rehabilitation, the ACSM protocol was designed for broader exercise screening and adopts a more conservative classification, categorizing most individuals with cardiovascular disease as high risk. Since most participants had cardiovascular disease, the predominance of high-risk classifications under the ACSM protocol was expected. Santos et al.8 also reported low agreement between the ACSM and other protocols.
Santos et al.8 reported moderate agreement between the AACVPR and the 2013 BSC protocol (k=0.52; p<0.001), whereas the present study found excellent agreement between the AACVPR and the 2020 BSC protocol (k=0.78; 95%CI 0.604–0.956; p<0.001). This finding may reflect similarities in complementary examinations and exercise-related criteria.
Inter-rater analysis demonstrated excellent agreement between evaluators. The three disagreements mainly involved the interpretation of moderate-risk criteria or classification as unstratified cases with incomplete complementary examinations. However, inter-rater reliability was assessed using only two evaluators from the same institution, which may limit external generalizability. Intra-rater reliability was not evaluated.
Despite the significant agreement observed between the 2020 BSC protocol and most evaluated protocols, except ACSM, the findings should be interpreted cautiously. This study evaluated agreement rather than predictive validity for adverse events during exercise. Previous studies10,22 indicate that current cardiac risk stratification protocols have limited ability to predict events during supervised exercise, and the predictive ability of the 2020 BSC protocol remains untested.
Additionally, the sample size calculation did not account for multiple pairwise comparisons or hierarchical data structure related to inter-rater assessments and unstratified cases. The single-center design and limited access to complementary examinations may restrict the external validity of the findings.
CONCLUSION
The 2020 BSC protocol demonstrated moderate-to-excellent agreement with most evaluated cardiac risk stratification protocols and excellent inter-rater reliability. These findings contribute to the methodological assessment of the updated protocol and support its reproducibility. However, because this study did not evaluate the prediction of adverse events during exercise, no conclusions can be drawn regarding predictive validity or exercise safety. Further prospective studies are needed to evaluate the ability of the 2020 BSC protocol to predict clinical outcomes and exercise-related complications in cardiac rehabilitation programs.
DATA AVAILABILITY STATEMENT
The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.
REFERENCES
-
1. Carvalho T, Milani M, Ferraz AS, Silveira AD, Herdy AH, Hossri CAC, et al. Diretriz Brasileira de reabilitação cardiovascular – 2020. Arq Bras Cardiol. 2020;114(5):943-87. https://doi.org/10.36660/abc.20200407
» https://doi.org/10.36660/abc.20200407 - 2. Vanderlei LCM, Lopes PP, Tarumoto MH, Ramos EMC, Ramos D, Camargo Filho JCS. Analysis of signs and symptoms in ambulatorial programs of physical exercises for cardiac patients. Arq Ciênc Saúde. 2006;13(2):69-74.
-
3. Pavy B, Iliou MC, Meurin P, Tabet JY, Corone S. Safety of exercise training for cardiac patients: results of the French registry of complications during cardiac rehabilitation. Arch Intern Med. 2006;166(21):2329-34. https://doi.org/10.1001/archinte.166.21.2329
» https://doi.org/10.1001/archinte.166.21.2329 -
4. Scheinowitz M, Harpaz D. Safety of cardiac rehabilitation in a medically supervised, community-based program. Cardiology. 2005;103(3):113-7. https://doi.org/10.1159/000083433
» https://doi.org/10.1159/000083433 -
5. Thompson PD, Franklin BA, Balady GJ, Blair SN, Corrado D, Estes NAM, et al. Exercise and acute cardiovascular events: placing the risks into perspective. Circulation. 2007;115(17):2358-68. https://doi.org/10.1161/CIRCULATIONAHA.107.181485
» https://doi.org/10.1161/CIRCULATIONAHA.107.181485 -
6. Rabinstein AA. Sudden cardiac death. Handb Clin Neurol. 2014;119:19-24. https://doi.org/10.1016/B978-0-7020-4086-3.00002-3
» https://doi.org/10.1016/B978-0-7020-4086-3.00002-3 -
7. Rodrigues CF, Bothrel GB, Turci MA. Análise comparativa das diferentes ferramentas de estratificação de risco cardiovascular: revisão narrativa. Reas. 2021;13(9):e8733. https://doi.org/10.25248/reas.e8733.2021
» https://doi.org/10.25248/reas.e8733.2021 -
8. Santos AAS, Silva AKF, Vanderlei FM, Christofaro DGD, Gonçalves AFL, Vanderlei LCM. Analysis of agreement between cardiac risk stratification protocols applied to participants of a center for cardiac rehabilitation. Braz J Phys Ther. 2016;20(4):298-305. https://doi.org/10.1590/bjpt-rbf.2014.0159
» https://doi.org/10.1590/bjpt-rbf.2014.0159 -
9. Silva AK, Barbosa MP, Bernardo AF, Vanderlei FM, Pacagnelli FL, Vanderlei LC. Cardiac risk stratification in cardiac rehabilitation programs: a review of protocols. Rev Bras Cir Cardiovasc. 2014;29(2):255-65. https://doi.org/10.5935/1678-9741.20140067
» https://doi.org/10.5935/1678-9741.20140067 -
10. Ribeiro F, Takahashi C, Vanzella LM, Laurino MJL, Lima IM, Silva VES, et al. An investigation into whether cardiac risk stratification protocols actually predict complications in cardiac rehabilitation programs. Clin Rehabil. 2021;35(5):775-84. https://doi.org/10.1177/0269215520978499
» https://doi.org/10.1177/0269215520978499 - 11. American College of Sports Medicine. Diretrizes do ACSM para o teste de esforço e sua prescrição. 7th ed. Rio de Janeiro: Guanabara Koogan; 2007.
-
12. Moraes RS, Nóbrega ACL, Castro RRT, Negrão CE, Stein R, Serra SM, et al. Diretriz de reabilitação cardíaca. Arq Bras Cardiol. 2005;84(5):431-40. https://doi.org/10.1590/S0066-782X2005000500015
» https://doi.org/10.1590/S0066-782X2005000500015 - 13. American Association of Cardiovascular and Pulmonary Rehabilitation. Diretrizes para reabilitação cardíaca e programas de prevenção secundária. 4th ed. São Paulo (SP): Roca; 2007.
-
14. Fletcher GF, Balady GJ, Amsterdam EA, Chaitman B, Eckel R, Fleg J, et al. Exercise standards for testing and training: a statement for healthcare professionals from the american heart association. Circulation. 2001;104(14):1694-740. https://doi.org/10.1161/hc3901.095960
» https://doi.org/10.1161/hc3901.095960 - 15. Monpère C, Sellier P, Meurin P, Aeberhard PB, D’Agrosa Boiteux M, Iliou M, et al. Recommandations de la Société française de cardiologie concernant la pratique de la réadaptation cardiovasculaire chez l’adulte. Arch Mal Coeur. 2002;95(10):962-97.
- 16. Velasco JA, Cosín J, Maroto JM, Muñiz J, Casasnovas JA, Plaza I, et al. [Guidelines of the Spanish Society of Cardiology for cardiovascular disease prevention and cardiac rehabilitation]. Rev Esp Cardiol. 2000;53(8):1095-20. PMID: 10956605
-
17. Pashkow FJ. Issues in contemporary cardiac rehabilitation: a historical perspective. J Am Coll Cardiol. 1993;21(3):822-34. https://doi.org/10.1016/0735-1097(93)90116-I
» https://doi.org/10.1016/0735-1097(93)90116-I -
18. Ramos Haddad Garcia MI, Ricci-Vitor AL, Vanzella LM, França Silva AK, Takahashi C, Ribeiro F, et al. Interrater reliability across 7 established risk stratification protocols in cardiac rehabilitation. Arch Phys Med Rehabil. 2021;102(3):470-9. https://doi.org/10.1016/j.apmr.2020.08.020
» https://doi.org/10.1016/j.apmr.2020.08.020 -
19. Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vanderbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344-9. https://doi.org/10.1016/j.jclinepi.2007.11.008
» https://doi.org/10.1016/j.jclinepi.2007.11.008 -
20. Kraemer HC, Bloch DA. A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation. Stat Med. 1994;13(8):876-80. https://doi.org/10.1002/sim.4780130809
» https://doi.org/10.1002/sim.4780130809 -
21. Arifin WN. Sample size calculator. 2023. [cited on 2023 Sep 19]. Available from: http://wnarifin.github.io
» http://wnarifin.github.io -
22. Vanzella LM, Takahashi C, Ribeiro F, Lima IM, Silva AKF, Christófaro DGD, et al. Efficacy of risk stratification protocols and clinical, physical, and biochemical parameters to predict signals and symptoms during cardiovascular rehabilitation programs: protocol for an observational trial. Medicine (Baltimore). 2019;98(24):e15700. https://doi.org/10.1097/MD.0000000000015700
» https://doi.org/10.1097/MD.0000000000015700 -
23. Sousa FO, Medeiros KR, Gurgel Júnior GD, Albuquerque PC. From normative aspects to the reality of the Unified Health System: revealing barriers that curtail access to the health care network. Cien Saude Colet. 2014;19(4):1283-93. https://doi.org/10.1590/1413-81232014194.01702013
» https://doi.org/10.1590/1413-81232014194.01702013
Edited by
-
Scientifıc Editor:
Roseli Nomura https://orcid.org/0000-0002-6471-2125
