Training preceptors for feedback in pediatrics residence through digital educational material

Lima, Alessandra Miramontes; Cunha, Mariana Lucas Rocha

doi:10.1590/1981-5271v49.1-2024-0248

ABSTRACT

Introduction: The lack of preceptors’ adequate preparation to carry out assessments and provide effective feedback compromises the potential to motivate and guide the learning of resident physicians. In most contexts of health education, there is a perceived failure in the medical preceptors’ capacitation.

Objective: to describe the development of a digital educational material based on the principles of andragogy, competency-based education, work-based assessment and feedback to improve the skills of pediatric residency preceptors in providing quality feedback.

Method: This is a methodological study focused on developing an educational material in the form of a digital booklet, created using the CanvaPro® platform and validated by experts. The process involved documentary and bibliographic research to identify vulnerabilities in the assessment and feedback processes in medical residency programs, emphasizing the role of preceptors as clinical educators. Validation was conducted with experienced pediatric preceptors, using the Delphi method to achieve consensus of at least 80% in each round, with a review of dissenting items. The study included 13 judges, 84% of whom have been preceptors for over three years. Data collection took place in the first semester of 2024, and the study was approved by the Ethics Committee.

Results: The educational material consisted of textual elements with the conceptualization of competency-based education, andragogy, work-based assessment and feedback; complementary online interactive links with topics (such as preceptor; tips for bedside activities and case study; definition of competencies in medical education, entrustable professional activities; conceptualization of formative evaluation, feedback and coaching). Two rounds of the Delphi method were carried out to validate the material; in the first round, nine of the ten questions addressing the quality of the educational material obtained 80% or more agreement. Most of the judges’ observations were accepted.

Conclusion: The material produced was considered relevant and suitable to assist pediatric preceptors in applying the MiniCex instrument, which includes providing immediate feedback on performance assessment, by the experts who participated in the validation. Furthermore, the material is visually appealing and designed to offer a pleasant learning experience, with clear and practical content to contribute to the training of pediatric preceptors.

Keywords:
Educational Measurement; Clinical Competency; Feedback; Formative Feedback; Mentorship

RESUMO

Introdução: A falta de preparo adequado dos preceptores para realizar avaliação e fornecer feedback eficaz compromete o potencial de motivar e orientar o aprendizado de médicos residentes. Na maioria dos contextos da educação em saúde, existe uma falha percebida na capacitação dos preceptores.

Objetivo: Este estudo teve como objetivo descrever o desenvolvimento de um material educacional digital pautado nos princípios da andragogia, no ensino por competências, na aplicação de instrumentos de avaliação na prática clínica e feedback para aprimorar as habilidades dos preceptores de residência em pediatria.

Método: Trata-se de um estudo metodológico para o desenvolvimento de um material educacional em formato de cartilha digital, criado na plataforma CanvaPro® e validado por especialistas. O processo incluiu pesquisa documental e bibliográfica para identificar vulnerabilidades na avaliação e feedback na residência médica, com ênfase no papel do preceptor como educador clínico. A validação foi realizada com pediatras preceptores experientes, seguindo o método Delphi para a obtenção de consenso de pelo menos 80% em cada rodada, com revisão dos itens discordantes. Participaram do estudo 13 juízes, 84% deles atuam como preceptores há mais de três anos. A coleta de dados ocorreu no 1º semestre de 2024. A pesquisa foi aprovada pelo Comitê de Ética.

Resultado: O manual educativo foi constituído por elementos textuais com conceituação de ensino por competências, andragogia, avaliação em ambiente de prática e feedback; links interativos complementares para o aprofundamento em temas específicos (preceptor; dicas para atividades à beira leito e estudo de caso; caracterização de competências e atividades profissionais confiáveis; conceituação de avaliação formativa, feedback e coaching). Foram realizadas duas rodadas do Delphi para validação do material, e a primeira obteve 80% ou mais de concordância em nove das dez questões apresentadas. A maioria das observações dos juízes foram acatadas.

Conclusão: O material produzido foi considerado relevante e propício para auxiliar preceptores de pediatria a aplicar o instrumento Miniex, que inclui o fornecimento de feedback imediato à avaliação de desempenho, pelos especialistas que participaram da validação. Ademais, o material é visualmente atraente e projetado para oferecer uma experiência de aprendizado agradável, de conteúdo claro e prático para contribuir para a capacitação de preceptores de pediatria.

Palavras-chave:
Avaliação da Educação; Educação Baseada em Competências; Retroalimentação; Feedback Formativo; Preceptoria

INTRODUCTION

The lack of guidelines to guide teaching in Medical Residency Programs (MRPs) represents a significant challenge for preceptors, who are responsible for the direct supervision of resident physicians¹^)-(³. Learning in a practical environment requires diversified teaching strategies and assessments that allow qualifying learning in multiple domains, aiming to promote the acquisition of complex skills and essential competencies²^),(⁴.

Competency-based medical education values results, through the continuous development of skills necessary for clinical practice and is supported by formative assessment and feedback to guide the individualized progress of residents, promoting reflection and directing professional training⁴^)-(⁸. However, the applicability of feedback in medical education is not always consistent, failing to motivate and effectively guide learning⁸^)-(¹⁰.

Studies have addressed difficulties faced by preceptors during the supervision of residents, due to work overload, lack of didactic knowledge and lack of tools that contribute to the role of clinical educator¹⁰^)-(¹²^). Despite determinations by the residency committees to delegate resident evaluations to these professionals, little is invested in training and providing means to allow a quality formative evaluation¹³^{), (}¹⁴.

This article aims to describe the stages of construction of an educational material based on the principles of andragogy, competency-based teaching and the application of assessment and feedback instruments to improve the skills of pediatric residency preceptors.

METHOD

This is a methodological study for the development of educational material in the format of a digital booklet, available on an online graphic design platform. The methodology includes the following stages: documentary research, preparation and development of educational material and evaluation by experts in the field for validation of the prepared material.

The documentary and bibliographic research supported the project in the identification of vulnerabilities involving evaluation and feedback in medical residency, as well as on the role of the preceptor as a clinical educator. Its purpose was the creation of educational material in the form of an interactive booklet, addressing aspects of the teaching-learning process, highlighting topics of competency-based education, assessment instruments, feedback and the guidance of resident physicians by preceptors.

The creation and development phase involved the preparation of educational material in the form of a digital booklet and educational videos, included in the CanvaPro® platform. The didactic content was created from the literature review with the purpose of providing a solid theoretical basis founded on previous research and in alignment with current and relevant knowledge. Moreover, it has allowed identifying gaps in current knowledge and practices, to ensure that the strategies proposed in the produced material are effective and evidence-based. The material emphasized assessment instruments in a practice environment, particularly the Mini-Clinical Evaluation Exercise (mini-CEX) as it is a validated instrument translated into Portuguese, applicable in several clinical scenarios and includes feedback at the end of the activity. The educational videos brought simulated scenarios of interaction between preceptor and resident, allowing the identification of communication patterns and desirable behaviors, aiming at a reflection on what a formative quality assessment should be like.

The validation phase of the educational material included the appreciation of the booklet by a group of preceptors from the general pediatrics program, invited to analyze the content, provide comments and propose suggestions for adaptations and validation of the material, using the Delphi method to obtain expert consensus¹⁵^)-(¹⁷. According to the Delphi method, communication rounds should be repeated until consensus or the best agreement is reached after a predetermined number of rounds¹⁶. Ideally, in committees with six or more judges, an agreement of 80% is suggested as indicative of consensus¹⁵^),(¹⁷. The Delphi method rounds were carried out in a virtual environment, through an exchange of e-mails, with individual interaction, between the author and each participant, thus maintaining confidentiality between the participants, and only the author had access to the answers.

The experts were invited to participate based on the selection of a preceptor called a ‘seed’, which then, using the Snowball Method, allowed the indication of the other research participants¹⁹. The participants were included according to the criteria previously established by Jasper²⁰: (1) professional specialization in the field of study: Master’s degree or Ph.D. in health education; (2) extensive experience acquired by practice in the area: preceptor pediatricians, with more than 3 years of experience in the position of preceptorship; (3) have passed a specific test or have a degree that proves specialization in the area: have completed and have a certificate of the training course in medical or multiprofessional preceptorship in accordance with the requirements established by the Ministry of Education and Culture (MEC); (4) have recognition and a high classification attributed by an authority: have a teaching position at the teaching hospital where they work as preceptors. None of the individuals invited to participate refused to do so. The evaluation of the material took place in the first half of 2024.

The validation of the material occurred after approval by the Research Ethics Committee under opinion number 6,524,469. The experts were instructed about the objective of the study and consented to participate by signing the informed consent form.

For approval of the educational material content, the values of agreement between the judges suggested by the literature of at least 80% were adopted for each round ¹⁸. Items that did not reach 100% agreement were reviewed until a consensus of 80% was reached among the answers to each item.

In the implementation stage, the final version of the educational material will be made available through a public sharing link, which can be consulted by professionals who work in the pediatric medical preceptorship area.

RESULTS

The educational manual consisted of textual elements with teaching conceptualization, andragogy, evaluation instruments in a practice environment and feedback; interactive links that direct to complementary materials on the Canva® platform itself for the expansion of specific topics, namely, approach to: preceptor; instructions for bedside activities and case study; characterization of competencies, entrustable professional activities (EPA) and specific specialty achievements (SSA); conceptualization of formative assessment, feedback and coaching.

In addition to the textual content, complementary materials addressing prominent topics for consultation through QR code, content suggestions for complementary reading with citations of articles and videos with public access, with Youtube® links, are included in the booklet. The educational videos produced for this material are available at the end of the booklet.

The visual presentation selected by the experts, among four created design possibilities and the booklet outline, with the pages that contain the learning objectives, the arrangement of chapters, the orientation icons and the closing of the booklet, are summarized in Figure 1.

Figure 1
Visual presentation (design), learning objectives and arrangement of chapters of the educational material.

The validation of the content of the educational manual was carried out by 13 preceptors, most of whom (11; 84%) had been working in preceptorship for more than three years and in this group, four professionals had training in preceptorship (4; 3%) and two had a Master’s degree in Health Education. Two participants had as a criterion having a teaching position in the institution where they work as preceptors. The demographic profile of the participants consists of 11 female participants and 2 male participants; with an age range of 25 to 34 years (2; 15,4%); 35 to 49 years (7; 53.8%) and 50 to 64 years (4; 30,7%). In relation to the Institutions where they work, 7 preceptors work in Public Institutions, 5 of which are located in the city of São Paulo or greater São Paulo (Taboão da Serra and Guarulhos) and 2 in the city of Recife/Pernambuco; 5 in Private Institutions, with 2 in the city of São Paulo and 1 in the city of Umuarama-Paraná; and 1 preceptor works in a Public Institution with Private Management in the city of São Paulo, state of São Paulo.

The stages of the Delphi method were conducted by the researcher and during the rounds (2 in total), each professional answered the online questionnaire in the format of a Google® form and presented their suggestions about the material. The proposals were analyzed and the contributions considered pertinent and aligned with the purposes of the educational material. Therefore, they were included in the final version of the booklet.

The first round of the study obtained 80% or more of agreement in nine of the ten questions, among the 13 judges who answered the questionnaire (Table 1). The only item that did not reach consensus (with 76% agreement) was the one that referred to the validity of the material to train preceptors for evaluation using the MiniCex method. The justification of the participants who disagreed was the fact that the MiniCex application was not addressed in more detail. After this contribution, additional materials were included addressing the instrument, its application, and the booklet was sent for further consideration.

Thumbnail

Table 1
Distribution of the experts’ agreement index. São Paulo, 2024.

In the 2^nd round, all items were considered appropriate and reached 100% agreement.

The booklet in digital format is available on the CanvaPro® Platform through the link:

https://www.canva.com/design/DAGMPQe0LNI/Gkfs_jcIAdUuGGjYq3HcfA/edit?utm_content=DAGMPQe0LNI&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton

To access spreadsheets of complementary material to the booklet, if you do not get direct access through Canva®, you can use the following links:

DISCUSSION

In medical education, feedback has been linked to evaluation tools, as a form of performance communication⁸^),(¹⁰.

Effective feedback is essential for the acquisition of new knowledge and skills, and is especially applicable to competency-based education, guiding through constructive stimulus, the reinforcement of technical skills and desirable behaviors, culminating in the acquisition of competencies and nuances of knowledge⁵^),(⁶^),(⁸^),(¹⁰.

Medical residency, considered the standard of excellence of medical specialization and the archetype of competency-based education, must ensure that recent graduates reach the level of competency required by the specialty. In this context of training, in the practice of care, the evaluation goes beyond the cognitive aspect, representing a challenge for preceptors³^),(⁴.

An effective evaluation process requires not only faculty training, but also a system that integrates several evaluation methods, contemplating all aspects of learning (knowledge, skills and attitudes) and also ensures the validity and reliability of the applied instruments³^),(⁷.

Methods for assessing clinical skills are based on the direct observation of performance, in real or simulated situations, with the application of immediate feedback, to articulate efficiency in the performed activity. A well-structured and periodically carried out evaluation system, accompanied by continuous feedback, is crucial to improve conduct and ensure the resident’s qualification³.

Given the preceptor’s role in the guidance of resident physicians, the continuous development of training programs aimed at improving these competencies should be considered paramount, since most of these professionals do not have specific training to evaluate performance in an objective and impartial way, with feedback communication¹¹^),(¹⁵^),(²⁴.

The literature highlights several barriers that compromise the effectiveness of feedback in the educational context and, among the mentioned obstacles, cultural aspects are cited (resistance to the integration of continuous formative feedback into the evaluation process); structural aspects (lack of curricular integration and between formative feedback and summative assessment; deficiency in the training of those who provide feedback) and personal aspects (personal vulnerability in providing and receiving feedback and lack of specific training)²¹^{), (}²²^{), (}²⁴^),(²⁵.

Many of the raised topics require a considerable investment of resources and a restructuring of the education system. In this context, the training of preceptors, responsible for the residents’ evaluation and feedback, emerges as a valuable opportunity to improve the quality of education²². Thus, the produced material demonstrates potential to meet the need for a more qualified training of preceptors.

Considering the judges’ evaluation, the only item that did not attain consensus in the first round was the one related to the effectiveness of the material to train preceptors in the evaluation through the MiniCex method.

The MiniCex was chosen in this study because it is an evaluation method applicable in a practice environment, which has been translated into Portuguese and validated to be applied in several clinical scenarios and practice environments, from outpatient clinics to emergency rooms. Additionally, the MiniCex is linked to immediate feedback after the evaluation, therefore being an opportune instrument in formative evaluations²⁶.

The experts’ opinion led to the review and emphasis on this specific topic, corroborated by findings of a study that emphasizes that the application of MiniCex in clinical scenarios increases reflection on clinical practice itself and positively impacts the preceptor’s professional development²⁷. It is expected that, with the training for the application of MiniCex, the preceptors will become better prepared in the context of medical education of competencies.

The author’s personal experience as a pediatric preceptor, combined with peer observation, to reports from professionals involved in resources aimed at this purpose. This situation motivated the development of educational material in the form of a digital booklet, with the objective of promoting the adoption of practices that use performance-based evaluation and feedback instruments, integrated with the monitoring of residents²⁸^),(²⁹. The General Pediatrics Medical Residency Program was chosen as a pilot, due to the preceptorship of pediatric residents and the published research, highlighting the importance of training directed to preceptors and the lack of dealing with one of the basic areas of medical specialization and the author’s own experience, who works in this field.

The publication in digital format with the integration of technology in health education aims to overcome challenges related to access to education, being made available to pediatric preceptors throughout the national territory, motivating the appropriation of instruments and good practices in the evaluation and guidance of residents under their supervision⁴.

Evidence indicates that providing this type of training not only improves the feedback skills of preceptors but also has a positive impact on the learners’ perceptions; with the dissemination of initiatives aimed at this purpose, it is possible to foster a culture of effective application of feedback, contributing to the improvement of learning and broad professional development in the health area²⁹^)-(³³.

Implications

The material produced was considered adequate to help pediatric preceptors to apply the MiniCex instrument, which includes the provision of immediate feedback to the performance evaluation, by the specialists who participated in the validation, with 84% or more of agreement, in most of the evaluated items.

Study limitations and suggestions for future studies

The low adherence of the group of judges invited to validate the material limited the generalization of the results in a more robust and representative way. Of thirty-five messages sent by email, WhatsApp® or direct message on the LinkedIn profile, only twenty-three participants filled out the electronic consent form and of these, only 13 completed the evaluation of the material.

Regarding the booklet content, some suggestions were not incorporated into the final version, such as the inclusion of attached material with practical tips. The booklet is designed to provide key information to preceptors, who often lack basic knowledge of education and assessment principles but are responsible for verifying the residents’ aptitude. For this reason, a descriptive approach to the most relevant topics in the initial project was chosen, emphasizing the importance of evaluation and feedback.

However, the suggestion of guidance for practical situations is extremely relevant and should be considered in future projects and preferably with collaborative versions between different institutions, addressing common situations and challenges in the practice of preceptorship, with guidance and suggestions for approach in more challenging cases.

The inclusion of videos of realistic situations was disregarded at this time, as it was not aligned with the purpose of including videos as educational material, seeking to illustrate positive behavior and communication during evaluation and feedback activities, according to the theory of “Role Modeling”³⁴^),-(³⁶.

Another rejected suggestion was to include group feedback modalities, since studies corroborate the concept that constructive feedback focused on learning guidance is better received individually and, after a clinical meeting, while positive feedback addressing procedural skills or decision-making may be more effective if discussed in groups²⁹. As the main objective of the material was to highlight formative feedback, it is important to highlight the importance of being individualized, to meet the specific needs of each learner²⁹.

The digital booklet, which will be made available online, also has as a limitation the need for periodic updates, essential to keep up with the evolution of new practices and feedback tools, so that complementary materials or editions must be incorporated with regular updates.

CONCLUSIONS OR FINAL CONSIDERATIONS

The educational material will be published on a digital platform, with free access and public sharing.

Future research may expand and increase some issues indicated by the specialists, such as the construction of materials with real scenarios and difficulties faced in the practice of monitoring residents, with focused guidelines.

In addition, it is suggested that the content derived from this manual can be adapted to other specialties in medical residency, such as the contents of topics and videos adapted to other clinical situations and that can, in the future, also be validated by specialists in the area.

It is expected that this research can foster interest in the development of new educational technologies and practical implementation in different contexts of medical education, especially to train those involved in medical residency training.

References

¹ Gross CJ, Chiel LE, Gomez AR, Marcus CH, Michelson CD, Winn AS Defining the essential components of a teaching service Pediatrics. 2020;146(1). doi: https://doi.org/10.1542/peds.2020-0651.
» https://doi.org/https://doi.org/10.1542/peds.2020-0651
² Carvalho Filho ADM, Santos AAD, Wyszomirska RMDAF, Gauw JHD, Gaia IMSRS, Houly RM. Formação na residência médica: visão dos preceptores. Rev Bras Educ. 2022;46(2):e20210312. doi: 10.1590/1981-5271v46.2-20210312.
» https://doi.org/10.1590/1981-5271v46.2-20210312
³ Witheridge A, Scott-Smith W. Revisiting Miller’s pyramid in medical education: the gap between traditional assessment and diagnostic reasoning Int J Med Educ. 2019;10:191-2.
⁴ Carvalho Filho AdM, Santos AAd, Wyszomirska RMdAF, Gauw JHd, Gaia IMSRS, Houly RM. Training in Medical Residency: the preceptors’ view Revista Brasileira de Educação Médica. 2022;46(2).
⁵ Botti SH, Rego S. Processo ensino-aprendizagem na residência médica. Rev Bras Educ Med. 2010;34(1):132-40.
⁶ Ten Cate O. Competency-based postgraduate medical education: past, present and future GMS J Med Educ. 2017 Nov 15;34(5):Doc69.
⁷ van Woezik TET, Oosterman JP, Reuzel RPB, van der Wilt GJ, Koksma JJ. Practice-based learning: an appropriate means to acquire the attitude and skills for evidence-based medicine Int J Med Educ . 2020;11:140-5.
⁸ Schuwirth LWT, van der Vleuten CPM. A history of assessment in medical education Adv Health Sci Educ Theory Pract. 2020;25(5):1045-56. doi: https://doi.org/10.1007/s10459-020-10003-0.
» https://doi.org/https://doi.org/10.1007/s10459-020-10003-0
⁹ Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning Med Educ. 2019 Jan;53(1):76-85. doi: https://doi.org/10.1111/medu.13645.
» https://doi.org/https://doi.org/10.1111/medu.13645
¹⁰ Teunissen PW, Stapel DA, van der Vleuten C, Scherpbier A, Boor K, Scheele F. Who wants feedback? An investigation of the variables influencing residents’ feedback-seeking behavior in relation to night shifts Acad Med. 2009;84(7):910-7.
¹¹ Santos C, Kroeff R. A contribuição do feedback no processo de avaliação formativa. EDUCA - Rev Multidiscip Educ. 2018;5(11):20-39. [acesso em junho de 2024]. Disponível em: Disponível em: https://periodicos.unir.br/index.php/EDUCA/article/view/2776/2359
» https://periodicos.unir.br/index.php/EDUCA/article/view/2776/2359
¹² Oliveira LM, Oliveira SR dos SM, Fonseca M da CV. Da assistência à docência: narrativas de médicos sobre os múltiplos caminhos que os tornaram preceptores. Rev Bras Educ Med. 2021;45(1):e004. doi: https://doi.org/10.1590/1981-5271v45.1-20200057.
» https://doi.org/https://doi.org/10.1590/1981-5271v45.1-20200057
¹³ Jordan A. Ser preceptor no SUS: valorização para qualificação do ensino em serviço. Cad Pesq Educ. 2022;2:146-53.
¹⁴ Ferreira IG, Cazella SC, Costa MR da. Preceptoria médica: concepções e vivências de participantes de curso de formação em preceptoria. Rev Bras Educ Med. 2022;46(4):e162. doi: https://doi.org/10.1590/1981-5271v46.4-20220168.
» https://doi.org/https://doi.org/10.1590/1981-5271v46.4-20220168
¹⁵ Nordi ABdA, Kishi RGB, Carvalho BB, Evangelista DN, Gaion JPdBF, Saggin J, et al. World experiences in preceptorship in medical undergraduate education: an integrative review Rev Bras Educ Med. 2022;46(1):e013. doi: 10.1590/1981-5271v46.1-20210312.
» https://doi.org/10.1590/1981-5271v46.1-20210312
¹⁶ Ryan MS, Darden A, Paik S, D’Alessandro D, Mogilner L, Turner TL, et al. Key studies in medical education from 2017: a narrative review Acad Pediatr. 2019;19(4):357-67. doi: https://doi.org/10.1016/j.acap.2018.12.007.
» https://doi.org/https://doi.org/10.1016/j.acap.2018.12.007
¹⁷ Romero-Collado A. Essential elements to elaborate a study with the (e)Delphi method Enferm Intensiva. 2020;2(2):100-4. doi: https://doi.org/10.1016/j.enfie.2020.09.003.
» https://doi.org/https://doi.org/10.1016/j.enfie.2020.09.003
¹⁸ Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications Inf Manage. 2004;42(1):15-29.
¹⁹ Humphrey-Murto S, Varpio L, Wood T, Gonsalves C, Ufholz L, Mascioli K, et al. The use of the Delphi and other consensus group methods in medical education research: a review Acad Med . 2017;92(10):1491-1498.
²⁰ Vinuto J. A amostragem em bola de neve na pesquisa qualitativa. Temáticas. 2014;22: 203-20.
²¹ Jasper MA. Expert: a discussion of the implications of the concept as used in nursing J Adv Nurs. 1994 Oct;20(4):769-76.
²² Branfield Day L, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program Acad Med . 2020;95(11):1712-1717.
²³ Bing-You R, Hayes V, Varaklis K, Trowbridge R, Kemp H, McKelvy D. Feedback for learners in medical education: what is known? A scoping review Acad Med . 2017 Sept;92(9):1346-54.
²⁴ Auto B de SD, Vasconcelos MVL de, Peixoto ALV de A. Clinical skills assessment and feedback in pediatric residency Rev Bras Educ Med. 2021;45(2):227-235.
²⁵ Miranda ML de, Cunha S da, Barbosa L, Balarini M de M, Pinto SA dos S, Afonso DH. Qualidade da residência médica na perspectiva das partes interessadas: revisão de escopo e painel Delphi. Rev Bras Educ Med. 2024;48(3):e088. doi: https://doi.org/10.1590/1981-5271v48.3-2024-0305.
» https://doi.org/https://doi.org/10.1590/1981-5271v48.3-2024-0305
²⁶ Stagini S, Peres LVC. Teachers and students’ perceptions about feedback in clinical internships in medical school Rev Bras Educ Med. 2021;45(3):e149.
²⁷ Bing-You R, Ramani S, Ramesh S, Hayes V, Varaklis K, Ward D, et al. The interplay between residency program culture and feedback culture: a cross-sectional study exploring perceptions of residents at three institutions. Med Educ Online. 2019 [cited 2024 Aug 12];24(1):1611296.
²⁸ Megale L, Gontijo ED, Motta JAC. Avaliação de competência clínica em estudantes de medicina pelo Miniexercício Clínico Avaliativo (Miniex). Rev Bras Educ Med . 2009;33(2):166-75. doi: https://doi.org/10.1590/S0100-55022009000200002.
» https://doi.org/https://doi.org/10.1590/S0100-55022009000200002
²⁹ Chen W, Lai MM, Li TC, Chen PJ, Chan CY, Lin CC. Professional development is enhanced by serving as a mini-CEX preceptor J Contin Educ Health Prof. 2011;31(4):225-30. doi: https://doi.org/10.1002/chp.20134.
» https://doi.org/https://doi.org/10.1002/chp.20134.
³⁰ Pricinote SCMN, Pereira ERS. Percepção de discentes de medicina sobre o feedback no ambiente de aprendizagem. Rev Bras Educ Med .2016;40(3):470-480.
³¹ Natesan S, Jordan J, Sheng A, Carmelli G, Barbas B, King A, et al. Feedback in medical education: an evidence-based guide to best practices from the council of residency directors in emergency medicine West J Emerg Med. 2023 May 5;24(3):479-94.
³² Dohms MC, Collares CF, Tibério IC. Video-based feedback using real consultations for a formative assessment in communication skills BMC Med Educ . 2020;20:57. doi: 10.1186/s12909-020-1955-6.
» https://doi.org/10.1186/s12909-020-1955-6
³³ Ramani S, Könings KD, Mann KV, Pisarski EE, Van Der Vleuten CPM. About politeness, face, and feedback Acad Med . 2018;93(9):1348-58.
³⁴ Thampy H, Willert E, Ramani S. Assessing clinical reasoning: targeting the higher levels of the pyramid J Gen Intern Med. 2019;34(8):1631-6. doi: https://doi.org/10.1007/s11606-019-04953-4.
» https://doi.org/https://doi.org/10.1007/s11606-019-04953-4
³⁵ Cardoso MD, Dias PLM, Rocha Cunha ML, Mohallem A, Dutra LA. How is feedback perceived by Brazilian students and faculty from a nursing school? Nurse Educ Pract. 2024;79:104057. doi: 10.1016/j.nepr.2024.104057.
» https://doi.org/10.1016/j.nepr.2024.104057
³⁶ Hao Q, Smith DH, Ding L, Ko A, Ottaway C, Wilson J, et al. Towards understanding the effective design of automated formative feedback for programming assignments Comput Sci Educ. 2022;32(1):105-27.
³⁷ Mohammadi E, Mirzazadeh A, Shahsavari H, Sohrabpour AA. Clinical teachers’ perceptions of role modeling: a qualitative study BMC Med Educ . 2021;21(1):261. doi: https://doi.org/10.1186/s12909-021-02648-1.
» https://doi.org/https://doi.org/10.1186/s12909-021-02648-1
³⁸ Horsburgh J, Ippolito K. A skill to be worked at: using social learning theory to explore the process of learning from role models in clinical settings. BMC Med Educ . 2018;18(1):156. doi: https://doi.org/10.1186/s12909-018-1251-x.
» https://doi.org/https://doi.org/10.1186/s12909-018-1251-x
³⁹ Moroz A, King A, Kim B, Fusco H, Carmody K. Constructing a Shared Mental Model for Feedback Conversations: Faculty Workshop Using Video Vignettes Developed by Residents. MedEdPORTAL. 2019;15:1082.

8
Evaluated by double blind review process.
SOURCES OF FUNDING
The authors declare no sources of funding.

Chief Editor:
Rosiane Viana Zuza Diniz.

Associate Editor:
Aristides Palhares Neto.

Publication Dates

Publication in this collection
31 Mar 2025
Date of issue
2025

History

Received
09 Sept 2024
Accepted
20 Dec 2024

Este é um artigo publicado em acesso aberto sob uma licença Creative Commons

[1] ¹ Gross CJ, Chiel LE, Gomez AR, Marcus CH, Michelson CD, Winn AS Defining the essential components of a teaching service Pediatrics. 2020;146(1). doi: https://doi.org/10.1542/peds.2020-0651.
» https://doi.org/https://doi.org/10.1542/peds.2020-0651

[2] ² Carvalho Filho ADM, Santos AAD, Wyszomirska RMDAF, Gauw JHD, Gaia IMSRS, Houly RM. Formação na residência médica: visão dos preceptores. Rev Bras Educ. 2022;46(2):e20210312. doi: 10.1590/1981-5271v46.2-20210312.
» https://doi.org/10.1590/1981-5271v46.2-20210312

[3] ³ Witheridge A, Scott-Smith W. Revisiting Miller’s pyramid in medical education: the gap between traditional assessment and diagnostic reasoning Int J Med Educ. 2019;10:191-2.

[4] ⁴ Carvalho Filho AdM, Santos AAd, Wyszomirska RMdAF, Gauw JHd, Gaia IMSRS, Houly RM. Training in Medical Residency: the preceptors’ view Revista Brasileira de Educação Médica. 2022;46(2).

[5] ⁵ Botti SH, Rego S. Processo ensino-aprendizagem na residência médica. Rev Bras Educ Med. 2010;34(1):132-40.

[6] ⁶ Ten Cate O. Competency-based postgraduate medical education: past, present and future GMS J Med Educ. 2017 Nov 15;34(5):Doc69.

[7] ⁷ van Woezik TET, Oosterman JP, Reuzel RPB, van der Wilt GJ, Koksma JJ. Practice-based learning: an appropriate means to acquire the attitude and skills for evidence-based medicine Int J Med Educ . 2020;11:140-5.

[8] ⁸ Schuwirth LWT, van der Vleuten CPM. A history of assessment in medical education Adv Health Sci Educ Theory Pract. 2020;25(5):1045-56. doi: https://doi.org/10.1007/s10459-020-10003-0.
» https://doi.org/https://doi.org/10.1007/s10459-020-10003-0

[9] ⁹ Watling CJ, Ginsburg S. Assessment, feedback and the alchemy of learning Med Educ. 2019 Jan;53(1):76-85. doi: https://doi.org/10.1111/medu.13645.
» https://doi.org/https://doi.org/10.1111/medu.13645

[10] ¹⁰ Teunissen PW, Stapel DA, van der Vleuten C, Scherpbier A, Boor K, Scheele F. Who wants feedback? An investigation of the variables influencing residents’ feedback-seeking behavior in relation to night shifts Acad Med. 2009;84(7):910-7.

[11] ¹¹ Santos C, Kroeff R. A contribuição do feedback no processo de avaliação formativa. EDUCA - Rev Multidiscip Educ. 2018;5(11):20-39. [acesso em junho de 2024]. Disponível em: Disponível em: https://periodicos.unir.br/index.php/EDUCA/article/view/2776/2359
» https://periodicos.unir.br/index.php/EDUCA/article/view/2776/2359

[12] ¹² Oliveira LM, Oliveira SR dos SM, Fonseca M da CV. Da assistência à docência: narrativas de médicos sobre os múltiplos caminhos que os tornaram preceptores. Rev Bras Educ Med. 2021;45(1):e004. doi: https://doi.org/10.1590/1981-5271v45.1-20200057.
» https://doi.org/https://doi.org/10.1590/1981-5271v45.1-20200057

[13] ¹³ Jordan A. Ser preceptor no SUS: valorização para qualificação do ensino em serviço. Cad Pesq Educ. 2022;2:146-53.

[14] ¹⁴ Ferreira IG, Cazella SC, Costa MR da. Preceptoria médica: concepções e vivências de participantes de curso de formação em preceptoria. Rev Bras Educ Med. 2022;46(4):e162. doi: https://doi.org/10.1590/1981-5271v46.4-20220168.
» https://doi.org/https://doi.org/10.1590/1981-5271v46.4-20220168

[15] ¹⁵ Nordi ABdA, Kishi RGB, Carvalho BB, Evangelista DN, Gaion JPdBF, Saggin J, et al. World experiences in preceptorship in medical undergraduate education: an integrative review Rev Bras Educ Med. 2022;46(1):e013. doi: 10.1590/1981-5271v46.1-20210312.
» https://doi.org/10.1590/1981-5271v46.1-20210312

[16] ¹⁶ Ryan MS, Darden A, Paik S, D’Alessandro D, Mogilner L, Turner TL, et al. Key studies in medical education from 2017: a narrative review Acad Pediatr. 2019;19(4):357-67. doi: https://doi.org/10.1016/j.acap.2018.12.007.
» https://doi.org/https://doi.org/10.1016/j.acap.2018.12.007

[17] ¹⁷ Romero-Collado A. Essential elements to elaborate a study with the (e)Delphi method Enferm Intensiva. 2020;2(2):100-4. doi: https://doi.org/10.1016/j.enfie.2020.09.003.
» https://doi.org/https://doi.org/10.1016/j.enfie.2020.09.003

[18] ¹⁸ Okoli C, Pawlowski SD. The Delphi method as a research tool: an example, design considerations and applications Inf Manage. 2004;42(1):15-29.

[19] ¹⁹ Humphrey-Murto S, Varpio L, Wood T, Gonsalves C, Ufholz L, Mascioli K, et al. The use of the Delphi and other consensus group methods in medical education research: a review Acad Med . 2017;92(10):1491-1498.

[20] ²⁰ Vinuto J. A amostragem em bola de neve na pesquisa qualitativa. Temáticas. 2014;22: 203-20.

[21] ²¹ Jasper MA. Expert: a discussion of the implications of the concept as used in nursing J Adv Nurs. 1994 Oct;20(4):769-76.

[22] ²² Branfield Day L, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program Acad Med . 2020;95(11):1712-1717.

[23] ²³ Bing-You R, Hayes V, Varaklis K, Trowbridge R, Kemp H, McKelvy D. Feedback for learners in medical education: what is known? A scoping review Acad Med . 2017 Sept;92(9):1346-54.

[24] ²⁴ Auto B de SD, Vasconcelos MVL de, Peixoto ALV de A. Clinical skills assessment and feedback in pediatric residency Rev Bras Educ Med. 2021;45(2):227-235.

[25] ²⁵ Miranda ML de, Cunha S da, Barbosa L, Balarini M de M, Pinto SA dos S, Afonso DH. Qualidade da residência médica na perspectiva das partes interessadas: revisão de escopo e painel Delphi. Rev Bras Educ Med. 2024;48(3):e088. doi: https://doi.org/10.1590/1981-5271v48.3-2024-0305.
» https://doi.org/https://doi.org/10.1590/1981-5271v48.3-2024-0305

[26] ²⁶ Stagini S, Peres LVC. Teachers and students’ perceptions about feedback in clinical internships in medical school Rev Bras Educ Med. 2021;45(3):e149.

[27] ²⁷ Bing-You R, Ramani S, Ramesh S, Hayes V, Varaklis K, Ward D, et al. The interplay between residency program culture and feedback culture: a cross-sectional study exploring perceptions of residents at three institutions. Med Educ Online. 2019 [cited 2024 Aug 12];24(1):1611296.

[28] ²⁸ Megale L, Gontijo ED, Motta JAC. Avaliação de competência clínica em estudantes de medicina pelo Miniexercício Clínico Avaliativo (Miniex). Rev Bras Educ Med . 2009;33(2):166-75. doi: https://doi.org/10.1590/S0100-55022009000200002.
» https://doi.org/https://doi.org/10.1590/S0100-55022009000200002

[29] ²⁹ Chen W, Lai MM, Li TC, Chen PJ, Chan CY, Lin CC. Professional development is enhanced by serving as a mini-CEX preceptor J Contin Educ Health Prof. 2011;31(4):225-30. doi: https://doi.org/10.1002/chp.20134.
» https://doi.org/https://doi.org/10.1002/chp.20134.

[30] ³⁰ Pricinote SCMN, Pereira ERS. Percepção de discentes de medicina sobre o feedback no ambiente de aprendizagem. Rev Bras Educ Med .2016;40(3):470-480.

[31] ³¹ Natesan S, Jordan J, Sheng A, Carmelli G, Barbas B, King A, et al. Feedback in medical education: an evidence-based guide to best practices from the council of residency directors in emergency medicine West J Emerg Med. 2023 May 5;24(3):479-94.

[32] ³² Dohms MC, Collares CF, Tibério IC. Video-based feedback using real consultations for a formative assessment in communication skills BMC Med Educ . 2020;20:57. doi: 10.1186/s12909-020-1955-6.
» https://doi.org/10.1186/s12909-020-1955-6

[33] ³³ Ramani S, Könings KD, Mann KV, Pisarski EE, Van Der Vleuten CPM. About politeness, face, and feedback Acad Med . 2018;93(9):1348-58.

[34] ³⁴ Thampy H, Willert E, Ramani S. Assessing clinical reasoning: targeting the higher levels of the pyramid J Gen Intern Med. 2019;34(8):1631-6. doi: https://doi.org/10.1007/s11606-019-04953-4.
» https://doi.org/https://doi.org/10.1007/s11606-019-04953-4

[35] ³⁵ Cardoso MD, Dias PLM, Rocha Cunha ML, Mohallem A, Dutra LA. How is feedback perceived by Brazilian students and faculty from a nursing school? Nurse Educ Pract. 2024;79:104057. doi: 10.1016/j.nepr.2024.104057.
» https://doi.org/10.1016/j.nepr.2024.104057

[36] ³⁶ Hao Q, Smith DH, Ding L, Ko A, Ottaway C, Wilson J, et al. Towards understanding the effective design of automated formative feedback for programming assignments Comput Sci Educ. 2022;32(1):105-27.

[37] ³⁷ Mohammadi E, Mirzazadeh A, Shahsavari H, Sohrabpour AA. Clinical teachers’ perceptions of role modeling: a qualitative study BMC Med Educ . 2021;21(1):261. doi: https://doi.org/10.1186/s12909-021-02648-1.
» https://doi.org/https://doi.org/10.1186/s12909-021-02648-1

[38] ³⁸ Horsburgh J, Ippolito K. A skill to be worked at: using social learning theory to explore the process of learning from role models in clinical settings. BMC Med Educ . 2018;18(1):156. doi: https://doi.org/10.1186/s12909-018-1251-x.
» https://doi.org/https://doi.org/10.1186/s12909-018-1251-x

[39] ³⁹ Moroz A, King A, Kim B, Fusco H, Carmody K. Constructing a Shared Mental Model for Feedback Conversations: Faculty Workshop Using Video Vignettes Developed by Residents. MedEdPORTAL. 2019;15:1082.

	5	4	3	2	1
1. Relevance and Clarity of Content
The material addresses the principles of the MiniCex method and its application	10 (77%)	3 (23%)	0	0	0
The information is presented in a clear and understandable manner	8 (61%)	4 (31%)	1(8%)	0	0
The content is relevant for the training of preceptors in the evaluation of pediatric residents.	11 (85%)	2 (15%)	0	0	0
2. Applicability and practicality
The educational material provides examples that facilitate the understanding and application of the MiniCex method.	9 (69%)	3 (23%)	1(8%)	0	0
The material includes guidance on how to provide immediate feedback in a constructive manner.	7 (54%)	6 (46%)	0	0	0
The material addresses challenges that preceptors may face when evaluating residents and offers strategies to overcome them	5 (38%)	6 (46%)	1(8%)	1(8%)	0
3. Visual presentation
Graphic design and layout make for an enjoyable learning experience	6 (46%)	5 (38%)	1(8%)	1(8%)
The technical quality of the videos is adequate and facilitates the understanding of the content	7 (54%)	6 (46%)	0	0	0
I consider this material effective for the training of pediatric preceptors in the application of the MiniCex method and providing immediate feedback.	7 (54%)	3 (23%)	3 (23%)	0	0