INSTITUTIONAL SELF-EVALUATION: PROPOSITIONS AND EVALUATIVE EVIDENCE FOR THE GRADUATE PROGRAMS AT FEDERAL UNIVERSITY OF GOIÁS

Autoevaluación institucional: propuestas y evidencias evaluativas para los programas de posgrado de la UFG

Abstract

In the last decades, the rapid growth of Higher Education Institutions (HEIs) has led to the development of several methods for their evaluation, culminating in the creation of the SINAES Law in 2004. However, postgraduate education is still inadequately covered in this evaluation process. To minimize this gap, this study proposed the creation of an exclusive evaluative instrument for postgraduate education at the Federal University of Goiás (UFG). The aim is to identify the decisive factors for the institution's development and its graduate programs from the students' perspective. This study is important because it contributes to the development of evaluation methods for postgraduate education in higher education institutions and for future researchers. The methodology consists of bibliographic and documentary analysis based on scientific texts and relevant legislation, as well as semi-structured interview techniques, case studies, and statistical inference. Based on the results, points that require the most improvement were identified, such as teaching and evaluation methodologies, the pedagogical project, program planning, as well as general infrastructure. In addition, the lack of actions to encourage scientific, technical, artistic, and extension production was also found.

Keywords:
institutional self-assessment; postgraduate studies; CPA.

Resumo

Nas últimas décadas, o rápido crescimento das Instituições de Ensino Superior (IES) resultou na elaboração de diversos métodos para avaliá-las, culminando na concepção da Lei do SINAES em 2004. No entanto, a pós-graduação ainda é pouco contemplada nesse processo avaliativo. Para minimizar essa lacuna, este estudo propôs a criação de um instrumento avaliativo exclusivo para a pós-graduação na

Universidade Federal de Goiás (UFG). O objetivo é identificar os fatores decisivos para o desenvolvimento da instituição e de seus programas de pós-graduação a partir da perspectiva dos estudantes. Esse estudo é importante porque contribui para o desenvolvimento de métodos avaliativos para a pós-graduação em IES e para futuros pesquisadores. A metodologia é composta pela análise bibliográfica e documental, a partir de textos científicos e legislações pertinentes, além de técnicas de entrevista semiestruturada, estudo de caso e inferência estatística. Com base nos resultados, foram identificados pontos que mais precisam de melhorias, como as metodologias de ensino e avaliação, o projeto pedagógico, o planejamento dos programas, bem como a infraestrutura em geral. Além disso, foi constatada a falta de ações que incentivem a produção científica, técnica, artística e de extensão.

Palavras-chave:
autoavaliação institucional; pós-graduação; CPA.

Resumen:

En las últimas décadas, el rápido crecimiento de las Instituciones de Educación Superior (IES) ha resultado en la implementación de diversos métodos para evaluarlas, culminando en la creación de la Ley del SINAES en 2004. Sin embargo, la educación de posgrado todavía recibe poca atención en este proceso de evaluación. Con el fin de minimizar esta brecha, este estudio propuso la creación de un instrumento de evaluación exclusivo para la educación de posgrado en la Universidad Federal de Goiás (UFG). El objetivo es identificar los factores decisivos para el desarrollo de la institución y sus programas de posgrado desde la perspectiva de los estudiantes. Este estudio es importante porque contribuye al desarrollo de métodos de evaluación para la educación de posgrado en IES y para futuros investigadores. La metodología se compone de análisis bibliográfico y documental a partir de textos científicos y legislación pertinente, además de técnicas de entrevistas semiestructuradas, estudios de caso e inferencia estadística. Según los resultados, se identificaron áreas que requieren mejoras, como las metodologías de enseñanza y evaluación, el proyecto pedagógico, la planificación de los programas de posgrado, así como la infraestructura en general. Además, se observó la falta de acciones que fomenten la producción científica, técnica, artística y de extensión.

Palabras clave:
autoevaluación institucional; estudios de posgrado; CPA

1 INTRODUCTION

The expansion of the higher education system in Brazil occurred in a disordered manner. As a result, the 1960s saw a substantial increase in the number of courses and student enrollments at this educational level. This expansion led to the need for procedures to measure the quality of services provided by Higher Education Institutions (HEIs), as well as the quality of teaching and learning (Peixoto, 2011). The National System for the Evaluation of Higher Education (SINAES) is the current regulatory and assessment mechanism for Brazilian HEIs, established in 2004. SINAES represents a milestone in the evaluation process of higher education, introducing important aspects such as institutional evaluation, the combination of self-evaluation and external evaluation, the recognition of formative and qualitative assessment processes, and the inclusion of meta-evaluation (Weber, 2010).

SINAES aims to ensure a national evaluation process. One of its key components is institutional self-evaluation, carried out by Internal Evaluation Committees (CPA) (Brasil, 2004). These committees are responsible for conducting internal institutional assessments, systematizing information, and providing data to the National Institute for Educational Studies and Research Anísio Teixeira (INEP) (Teixeira Júnior; Rios, 2017). In graduate education, the implementation of a systematic and continuous evaluation process for programs and courses began in 1976 under the coordination of the Brazilian Federal Agency for Support and Evaluation of Graduate Education (CAPES) (Pinto; Mello; Melo, 2016).

In this context, Higher Education Institutions (HEIs) conduct internal evaluations periodically, adapting procedures to their specific realities. Most of the results and information generated by these committees over recent decades have focused on undergraduate education. Although institutional accreditation and reaccreditation instruments include a dimension for graduate education policy, there are few selfevaluation activities targeting this level (Brito et al., 2021; Lehfeld et al., 2010).

One explanation is that the evaluation of graduate programs has always been managed by CAPES (2019; 2024). Another factor is the predominance of private colleges and university centers, which mainly provide undergraduate education. Nevertheless, graduate students and faculty are also important sources of evidence for institutional development, due to their close connection to scientific progress within the institution. Graduate students are often alumni of these HEIs and their undergraduate courses, and they benefit from institutional policies related to teaching, management, student retention, and infrastructure. They also participate in teaching internships and supervise undergraduate research activities.

To address the gap in self-evaluation processes, this study proposed the development of a specific assessment instrument for graduate students (stricto sensu) at the Federal University of Goiás (UFG). Based on empirical evidence, the findings were incorporated into the university's improvement plan.

The main objective was to identify, from graduate students’ perspectives, which factors influenced institutional and program development in 2022. Specifically, the study examined whether internal program evaluation was heterogeneous across the eleven dimensions of the applied instrument. It also assessed students’ future perspectives and the relationship between internal and external evaluation scores.

This research provides a direct contribution to the scientific field, given the lack of assessment tools and results for graduate education. Empirical results inform the Institutional Development Plan (PDI) of UFG and other institutional or program improvement plans. The study also enables the participation of graduate students in the evaluation process, facilitating the identification of potential improvements in management and institutional policies.

2 EVALUATION PROCESS CONDUCTED BY CPAs IN BRAZIL

To analyze the evaluation process carried out by Internal Evaluation Committees (CPAs) at Brazilian HEIs, a content analysis was conducted on self-evaluation reports produced and published by these committees. Reports from 122 HEIs, including federal, state, and private institutions, were examined. The selection focused on institutions similar to UFG, the central case of this study. Criteria included student enrollment, graduate program offerings, physical infrastructure, and CPA composition.

Of the institutions examined, 32 (26%) did not have accessible self-evaluation reports. Of the 90 reports reviewed, 27% were published before 2019. This indicates that many CPAs are not reporting evaluation results at the intervals required by SINAES. In eight institutions, a single evaluation instrument is used for both undergraduate and graduate students, with no separation of responses. This absence of differentiation limits the utility of the data for planning and analysis.

Of the institutional reports analyzed, only 19 institutions employ assessment instruments designed specifically for graduate education. These institutions include: Universidade Federal do Rio Grande (FURG), Pontifícia Universidade Católica do Rio de Janeiro and Pontifícia Universidade Católica de São Paulo (PUC-Rio and PUC-SP), Universidade Estadual de Londrina (UEL), Universidade Estadual de Maringá (UEM), Universidade Estadual do Maranhão (UEMA), Universidade Estadual de Ponta Grossa (UEPG), Universidade Estadual do Sudoeste da Bahia (UESB), Universidade Federal do ABC (UFABC), Universidade Federal do Cariri (UFCA), Universidade Federal de Ciências da Saúde de Porto Alegre (UFCSPA), Universidade Federal de Lavras (UFLA), Universidade Federal da Paraíba (UFPB), Universidade Federal do Paraná (UFPR), Universidade Federal de Santa Catarina (UFSC), Universidade Federal de São João delRei (UFSJ), Universidade Federal de Santa Maria (UFSM), Universidade do Estado da Bahia (UNEB), and Universidade Federal de Alfenas (UNIFAL).

The most of these institutions use social media, official websites, and institutional email to invite students to participate in the evaluation process. The questionnaires are primarily composed of closed-ended questions, offering little opportunity for student engagement. Most items measure student satisfaction with institutional aspects but do not address the underlying reasons for their responses, which reduces the effectiveness of evaluation and institutional planning. UEMA shows the highest participation rate (39%), while UFLA shows the lowest (5%).

Responses from graduate students at these 19 institutions indicate the need to increase awareness of the institutional evaluation process and to promote greater student participation in self-evaluation. There are also requests to improve, expand, and revise assessment instruments, as well as to enhance the dissemination of results. At the program level, there is a demand to expand the number of master's and doctoral programs, adapt curricula to current labor market requirements, and ensure the inclusion of students with disabilities.

Analysis of self-evaluation reports from 90 HEIs indicates that approximately 79% of CPAs do not employ specific methods for assessing graduate education, even after two decades since the creation of SINAES. Therefore, this research is relevant to the current landscape of institutional evaluation in Brazil, informing not only UFG but also other institutions that have not yet integrated graduate education into their assessment processes. The questionnaire developed in this study may be applied or adapted by these institutions according to their specific context and needs.

3 THEORETICAL FRAMEWORKS FOR EVALUATIVE DIMENSIONS IN INSTITUTIONAL SELF-EVALUATION WITHIN GRADUATE EDUCATION

The evaluation of graduate programs, especially in public universities, must align with CAPES external assessment and SINAES institutional guidelines to ensure compliance with quality, educational, social, and scientific standards. However, there is limited guidance for structuring institutional self-evaluation to address academic development, infrastructure, and management. Regulatory frameworks emphasize that university and graduate program development depends on the interaction of programs with institutional structures and other levels of higher education (Brasil, 2021b; CAPES, 2024).

This need for alignment arises from the absence of specific guidelines for selfevaluation in graduate programs, which may limit assessment effectiveness. In public universities facing management and resource challenges, establishing appropriate internal and external evaluation standards is essential. Current regulations require HEIs to demonstrate excellence in teaching, research, outreach, and infrastructure. Clearer guidelines for self-evaluation are necessary to meet legal and quality requirements set by evaluation systems. Table 1 presents self-evaluation dimensions for graduate programs, aligned with SINAES legislation and CAPES guidance. These dimensions allow institutions to adapt indicators and assessment tools to their specific contexts.

Table 1
Evaluative Dimensions for Graduate Programa Self-Evaluation in Relation to CAPES and SINAES

Evaluative Dimension	Description	Relationship with CAPES and SINAES
Research Area Development	Evaluation of research areas, their relevance, and alignment with academic and social needs.	CAPES assesses research areas for relevance, impact, and alignment with academic and social trends. SINAES verifies alignment with the institution's PDI.
Pedagogical Project and Its Update	Monitoring and updating the pedagogical project, ensuring coherence with the program and needs.	CAPES emphasizes pedagogical coherence and educational innovation. SINAES requires periodic updates to meet labor market and societal needs.
Planning	Analysis of the program’s strategic planning, with clear objectives and targets.	CAPES reviews strategic planning for goal effectiveness. SINAES views institutional planning as essential for developing teaching and research strategies.
Teaching, Learning, and Assessment Methods	Evaluation of teachinglearning and assessment methods employed.	CAPES reviews effectiveness and alignment with best pedagogical practices. SINAES assesses innovative use and societal relevance.
Faculty	Assessment of faculty qualifications, academic production, and experience.	CAPES considers qualifications, scientific output, and experience. SINAES examines management and professional development actions.
Scientific Production	Analysis of the program’s academic output: articles, books, chapters, and other results.	CAPES treats scientific production as a key indicator of knowledge generation. SINAES considers social impact and contribution.
Thesis and Dissertation Production	Evaluation of theses and dissertations, focusing on quality and scientific relevance.	CAPES assesses quantity, quality, originality, and contribution. SINAES evaluates social responsibility and professional impact.
Social Engagement, National and International Impact	Analysis of outreach activities and their national and international impact.	CAPES values integration and social contribution. SINAES assesses local and national development impact.
Management and Coordination	Assessment of program management and coordination efficiency.	CAPES reviews efficiency and goal achievement. SINAES examines institutional management, leadership, and administrative organization.
Evaluative Dimension	Description	Relationship with CAPES and SINAES
Partnerships with Companies and Institutions	Analysis of partnerships with companies and institutions to expand research and impact.	CAPES values partnerships for research expansion. SINAES values strategic partnerships for knowledge exchange and social/economic impact.
Infrastructure	Assessment of infrastructure: laboratories, equipment, and facilities for teaching and research.	CAPES evaluates adequacy for program activities. SINAES assesses physical and technological conditions for teaching and research.

4 METHOD

This research relied primarily on qualitative bibliographic and documentary analysis. Bibliographic analysis draws on secondary sources previously analyzed by other authors, while documentary analysis uses primary sources not yet examined (Lima Junior et al., 2021). This approach allows the researcher to gain deeper familiarity with the subject and facilitates critical reflection. Reviewing various materials also enables the synthesis of major debates, contributing to an understanding of higher education assessment in Brazil.

Semi-structured interviews were also conducted, using both closed and openended questions. This format allows respondents to express their views more freely and in greater depth, supporting the collection of more reliable data (Guazi, 2021). For the analysis of CPA self-evaluation reports, a case study methodology was used, offering a more detailed understanding of the object of study (Meirinhos; Osório, 2016). This step made it possible to assess which parameters have been used by HEIs and which have been overlooked, ensuring the questionnaire adequately represented students’ actual needs.

Several steps guided this research. First, the scientific literature and relevant legislation on graduate evaluation procedures and standards were reviewed. Next, selfevaluation reports from other HEIs were analyzed to determine if, and how, graduate program evaluation is being conducted nationally. A specific questionnaire for UFG’s stricto sensu graduate programs was then developed, in line with regulatory requirements and gaps identified in the reports. Finally, statistical procedures were conducted, including data collection, statistical processing, inference, and results analysis.

4.1 Data and Sources

To design the questionnaire, ten dimensions from Article 3 of the SINAES Law (BRASIL, 2004) and five dimensions proposed by CAPES for multidimensional assessment were considered: teaching and learning; internationalization; scientific output; innovation and knowledge transfer; and economic and social impact (CAPES, 2019). The instrument was also informed by five levels from INEP’s external institutional evaluation framework: institutional planning and assessment; institutional development; academic policies; management policies; and infrastructure (INEP, 2017).

Survey items were organized into eleven categories: research areas; pedagogical project; planning; teaching and assessment methods; faculty; scientific production; thesis and dissertation development; outreach, social engagement, and national/international impact; coordination and management; partnerships and collaborations; and infrastructure. Five additional questions addressed students’ future perspectives, which institutions can adjust as needed. The questionnaire was designed to meet respondents’ needs and encourage participation. It also covered the same criteria used by CAPES and INEP in external reviews, enabling regular internal assessment since external evaluation occurs every four years. Most questions were constructed to measure adequacy according to student-established standards.

The questionnaire was administered through the university’s online academic system (SIGAA). To participate, students had to be enrolled in at least one course in a master’s or doctoral program during the evaluated semester. At the time, 4,727 students were enrolled in stricto sensu graduate programs at the university. The form remained open for 19 days, ending before the close of the 2022-1 semester, to minimize bias from students with academic failures or pending requirements.

4.2 Statistical Procedures

The sample analyzed using mathematical statistics to identify variations and similarities in the data. This method was selected for its effectiveness in presenting clear and objective information to both the academic community and the public. The analysis incorporated instrumental samples, considering different perspectives such as faculty and future academic or professional plans. Significant differences between academic units and between external and internal evaluation scores were also identified. Results were presented using graphs, figures, and tables to highlight heterogeneity.

4.3 Statistical Inference

This study also employed statistical inference to draw conclusions about population characteristics based on sample data (Sampaio et al., 2018; Farias, 2008). As noted by Reis et al. (2015), this approach is appropriate for small samples and enables decision-making without analyzing the entire population. Using this method increases the reliability and applicability of results by allowing for population parameter estimation with a minimal margin of error. Classical statistical inference techniques include confidence intervals (CI) and hypothesis testing.

In this research, confidence intervals were used to estimate the proportion of the population that considers each questionnaire item adequate. According to Farias (2008), the confidence interval provides the margin of error for a parameter estimate, indicating the range in which the true value is expected to fall with a specified level of confidence. Confidence intervals at the 90% and 95% levels were calculated to produce more precise estimates of questionnaire item means and to better understand variation in the results. A 95% confidence interval means that, for 95% of samples drawn from the population, the true parameter value will fall within the calculated range (Farias, 2008).

The basic formula for the confidence interval for a proportion is:

$$I{C}_{(1-\alpha )}(p)\approx (\hat{p}-z_{1-\frac{\alpha }{2}}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}},\hat{p}+z_{1-\frac{\alpha }{2}}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}})$$

where:

IC is the confidence interval.

p is the population proportion of students who consider the evaluation indicators fully adequate.

$\hat{p}$ is the sample proportion of students who consider the evaluation indicators fully adequate.

n is the sample size.

z is the standard normal statistic, determined by the confidence level and sample size.

1 - α is the chosen confidence level.

5 RESULTS AND DISCUSSION

The assessment instrument was completed by 614 students out of a total population of 4,727. Of these respondents, 372 were master’s students and 242 were doctoral students, representing 60.6% and 39.4% of the sample, respectively. The sample included students from 28 academic units. For some courses, however, the academic unit could not be identified, leading to the creation of the “Other” category.³. This absence of affiliation is due to the multidisciplinary nature of these programs, which are not linked to a specific academic unit. Figure 1 shows the academic units and the overall mean from the internal evaluation.

Figure 1
Mean Internal Evaluation Scores by Academic Unit

Among UFG’s master’s programs, 58 were identified in the institutional evaluation. Twenty-three respondents did not specify their graduate program. Currently, 2,586 students are enrolled in academic and professional master’s programs. The survey participation rate for master’s students was 14.38%. For doctoral students, participation was 11.3%, considering a total of 2,141 active students and 37 identified doctoral programs.

The instrument contained 48 items evaluating program adequacy and 5 items on students’ future academic and professional perspectives. Table 2 presents the mean adequacy percentages by evaluation dimension and item (see appendix). The lowest scores were observed in classroom infrastructure and student support services. Auditoriums, computer laboratories, libraries, and IT resources also received low ratings. Indicators for artistic production, outreach activities, and program visibility at national and international levels presented similarly low values. The pedagogical project and program planning dimensions received low scores, particularly regarding the incorporation of student input.

Table 2
Mean Adequacy Percentage by Item in the Internal Evaluation and Mean Percentage by Dimension

Indicators				Items				Mean Adequacy percentage (%)
	1	2	3	4	5	6	7
1	78%	76%	92%	88%				83%
2	69%	70%	87%	74%	82%			76%
3	80%	70%	86%	65%	74%			75%
4	71%	74%	71%					72%
5	94%	91%	85%	77%	89%			87%
6	72%	77%	71%	66%	65%			70%
7	88%	86%	66%					80%
8	60%	63%	70%	63%	76%			67%
9	85%	83%	78%	83%	75%			81%
10	86%							86%
11	63%	65%	67%	67%	59%	68%	66%	65%

In contrast, most programs were rated positively regarding alignment with the institution’s mission, goals, and objectives, as well as in the planning of teaching, research, and outreach activities. The highest adequacy percentages were associated with program coordination, particularly in terms of regular communication and support provided to students and faculty. Additional strengths included the implementation of relevant program development actions and the dissemination of academic mobility opportunities.

Faculty received favorable evaluations for academic qualifications, professional experience, availability for supervision, research output, and the number of faculty available for advising. Respondents also gave positive assessments to the integration of research, teaching, and outreach activities, as well as to course content, theses and dissertations, and faculty expertise in the program’s research areas.

Table 3 presents the evaluation results for the master's programs included in the sample. Three programs are not rated, as they do not appear in the dataset provided by Plataforma Sucupira (2023). The Chemical Engineering program reported 100% adequacy, but this result was based on a single response. The master’s program in Health Care and Assessment had the lowest score, with 33.3%, also based on one response. The professional master’s programs in Public Administration (PROFIAP) and in Mathematics (PROFMAT) had the highest participation, with 18 and 14 respondents, respectively.

Table 3
Main Results of the Internal Evaluation of Master’s Programs

Master’s programs	Students Responses		Participation (%)	UFG Evaluation (%)	CAPES Rating
Ecology and Evolution	30	4	13,3	80.7	7
Environmental Sciences	28	5	17,9	85,0	6
Physics	30	8	26,7	52,9	6
Geography	54	7	13,0	72,3	6
History	82	9	11.0	86,1	6
Letters and Linguistics	78	11	14,1	89,4	6
Tropical Medicine	20	3	15,0	79,9	6
Chemistry	40	4	10.0	50.9	6
Agornomy	39	5	12,8	62,5	5
Social Anthropology	51	6	11.8	80.9	5
Biology of Parasite-Host	21	4	19,0	82,8	5
Animal Science	42	6	14,3	69,1	5
Biological Science	32	6	18,8	89,6	5
Education	71	11	15,5	80.7	5
Nursing	34	6	17,6	88,5	5
Mechanical Engineering	26	4	15,4	91.1	5
Philosophy	31	2	6,5	83,3	5
Plant Genetics	18	2	11.1	52,1	5
Mathematics	20	5	25,0	88,8	5
Dentistry	34	4	11.8	91.7	5
Business Administration	47	9	19,1	89,6	4
Agribusiness	21	5	23,8	71.3	4
Art and Visual Culture	29	3	10.3	45,8	4
Animal Biodiversity	23	1	4,3	47,9	4
Computer Science	62	8	12,9	89,3	4
Food Science	23	2	8,7	70.8	4
Political Science	40	6	15,0	85,1	4
Accounting Sciences	33	10	30.3	47,9	4
Health Sciences	236	11	4,7	79,0	4
Pharmaceutical Science	55	2	3,6	78,1	4
Communication	28	5	17,9	62,9	4
Agrarian Law	59	2	3,4	94,8	4
Human Rights	66	10	15,2	83,5	4
Mathematics Education	43	4	9,3	86,5	4
Electrical Engineering	15	1	6,7	95,8	4
Genetics and Molecular Biology	29	4	13,8	74,5	4
Geotechnics and Civil Construction	44	7	15,9	76,2	4
Nutrition and Health	24	4	16,7	89,6	4
Master’s programs	Students Responses		Participation (%)	UFG Evaluation (%)	CAPES Rating
Culutral Performance	25	4	16,0	45,3	4
Professional Master’s in Law and Public Policy	38	5	13,2	56,0	4
Professional Master’s in Basic Education	109	10	9,2	65,2	4
Professional Master’s in Health Teaching	41	9	22,0	42,9	4
Professional Master’s in Public Health	92	11	12,0	80.4	4
Project and City	34	5	14,7	79,2	4
Sociology	37	10	27,0	66,4	4
Animal Science (Zootecnia)	23	4	17,4	58,3	4
Performing Arts	43	8	18,6	76,0	3
Health Care and Assessment	26	1	3,8	33,3	3
Economics	15	4	26,7	66,7	3
Physical Education	43	3	7,0	88,2	3
Environmental and Sanitary Engineering	25	4	16,0	70.3	3
Chemical Engineering	26	1	3,8	100.0	3
Professional Master’s in Public Administration	40	18	45,0	67,9	3
Professional Master’s in Production Engineering	53	5	9,4	67,5	3
Psychology	96	8	8,3	82,9	3
Professional Master’s in Physical Education	41	11	26,8	75,0	-
Professional Master’s in History Teaching	45	8	17,8	84,4	-
Professional Master’s in National Network Mathematics	72	14	19,4	81.2	-
Not specified	-	23	-	-	-

Table 4 presents data on the evaluation of doctoral programs. The highest adequacy percentages were observed in Pharmaceutical Sciences (97.9%) and Ecology and Evolution (95.3%). The lowest scores were found in Communication (45.8%) and Biotechnology and Biodiversity (47.9%). The doctoral programs with the highest participation rates were Health Sciences and Social Anthropology, with 18 and 17 responses, respectively.

Table 4
Main Results of the UFG Evaluation of Doctoral Programs

Doctoral Programs	Student s	Responses	Participation (%)	UFG Evaluation (%)	CAPES Rating
Ecology and Evolution	53	4	7,5	95,3	7
Environmental Sciences	61	7	11.5	73,8	6
Physics	37	5	13,5	82,5	6
Geography	103	11	10.7	74,1	6
History	103	9	8,7	87,7	6
Letters and Linguistics	112	15	13,4	84,6	6
Tropical Medicine	61	7	11.5	63,1	6
Chemistry	64	2	3,1	65,6	6
Agronomy	79	10	12,7	71.0	5
Social Anthropology	56	17	30.4	71.8	5
Biology of Parasite-Host	16	5	31.3	73,8	5
Animal Science	71	7	9,9	78,3	5
Biological Science	39	2	5,1	67,7	5
Education	91	12	13,2	72,4	5
Nursing	57	11	19,3	86,6	5
Philosophy	30	5	16,7	81.3	5
Plant Genetics	27	3	11.1	78,5	5
Mathematics	37	3	8,1	77,8	5
Dentistry	38	2	5,3	94,8	5
Business Administration	36	8	22,2	72,1	4
Agribusiness	43	4	9,3	90.1	4
Animal Biodiversity	32	9	28,1	74,3	4
Biotechnology and Biodiversity	20	1	5,0	47,9	4
Computer Science	67	6	9,0	82,6	4
Food Science	30	2	6,7	82,3	4
Health Sciences	239	18	7,5	56,0	4
Pharmaceutical Sciences	44	1	2,3	97,9	4
Comunication	31	4	12,9	45,8	4
Agrarian Law	18	2	11.1	69,8	4
Human Rights	71	6	8,5	82,3	4
Science and Mathematics Education	66	14	21.2	72,9	4
Electrical Engineering	29	1	3,4	60.4	4
Genetics and Molecular Biology	39	9	23,1	59,7	4
Geotechnics and Civil Construction	35	4	11.4	52,6	4
Nutrition and Health	27	2	7,4	84,4	4
Cultural Performance	50	7	14,0	64,6	4
Animal Science (Zootecnia)	46	7	15,2	77,7	4

The questionnaire also included items on students’ personal and professional perspectives. Based on these responses, Figure 2 summarizes the percentage of respondents for each statement.

Figure 2
Overall Mean Percentage for the Five Perspctive Questions

Analysis showed that, in more than 96% of the 29 academic units, at least one student reported considering dropping out at some point during their program. In the School of History (FH), 70% of respondents indicated they had considered leaving the program, while in the School of Medicine (FM), no students reported such intentions.

When asked about job and professional development opportunities for graduates with similar training, all respondents from the School of Law (FD) and the Institute of Mathematics and Statistics (IME) answered positively; only 20% of students from FH agreed. Regarding alumni outcomes, all students from FD and the School of Electrical, Mechanical, and Computer Engineering (EMC) stated that graduates are finding opportunities in both the national and international job market. The School of Education (FE) reported the lowest agreement, at 20%.

For motivation to pursue graduate studies based on academic engagement, all respondents from the Institute of Chemistry (IQ), School of Philosophy (FAFIL), and School of Social Sciences (FCS) agreed. In contrast, only half of respondents from FM and FH cited this as their main motivation. Regarding foreign language proficiency, 75% of students from the School of Physical Education and Dance (FEFD) identified it as a barrier. Conversely, all respondents from the School of Philosophy (FAFIL) indicated that foreign language skills were not an obstacle to their professional training.

This study also included inferential statistical analysis of the programs, with results shown in Table 5. The confidence interval formula for proportions was used, as the evaluation metric was based on students’ assessment of each criterion. Ninety and ninety-five percent confidence intervals were applied to the eleven indicators for each item and the five perception questions, as shown in Table 5. These intervals were calculated to estimate the percentage of total adequacy for each program, based on the sample data.

Table 5
Statistical Inference for the Eleven Evaluation Dimensions and the Five Perception Questions

Program Indicators	P	n	z (a=90%)	z (a=95%)	IC (90%)		IC (95%)		Margin of Error (90%)	Margin of Error (95%)
Inidcator 1	0.83	614	1.645	1.960	0.8051	0.8549	0.8003	0.8597	0.0249	0.0297
Inidcator 2	0.76	614	1.645	1.960	0.7316	0.7884	0.7262	0.7938	0.0284	0.0338
Inidcator 3	0.75	614	1.645	1.960	0.7213	0.7787	0.7157	0.7843	0.0287	0.0343
Inidcator 4	0.72	614	1.645	1.960	0.6902	0.7498	0.6845	0.7555	0.0298	0.0355
Inidcator 5	0.87	614	1.645	1.960	0.8477	0.8923	0.8434	0.8966	0.0223	0.0266
Inidcator 6	0.70	614	1.645	1.960	0.6696	0.7304	0.6638	0.7362	0.0304	0.0362
Inidcator 7	0.80	614	1.645	1.960	0.7734	0.8266	0.7684	0.8316	0.0266	0.0316
Inidcator 8	0.67	614	1.645	1.960	0.6388	0.7012	0.6328	0.7072	0.0312	0.0372
Inidcator 9	0.81	614	1.645	1.960	0.7840	0.8360	0.7790	0.8410	0.0260	0.0310
Inidcator 10	0.86	614	1.645	1.960	0.8370	0.8830	0.8326	0.8874	0.0230	0.0274
Inidcator 11	0.65	614	1.645	1.960	0.6183	0.6817	0.6123	0.6877	0.0317	0.0377
Perception Indicators	p	n	z (a=90%)	z (a=95%)	IC (90%)		IC (95%)		Margin of Error (90%)	Margin of Error (95%)
Inidcator 1	0.35	614	1.645	1.960	0.3183		0.3817		0.3123	0.3877
Inidcator 2	0.67	614	1.645	1.960	0.6388		0.7012		0.6328	0.7072
Inidcator 3	0.70	614	1.645	1.960	0.6696		0.7304		0.6638	0.7362
Inidcator 4	0.78	614	1.645	1.960	0.7525		0.8075		0.7472	0.8128
Inidcator 5	0.41	614	1.645	1.960	0.3773		0.4427		0.3711	0.4489

Analysis of the eleven dimensions shows that Indicator eleven had the lowest mean value (0.65), whereas Indicator five had the highest mean value (0.87). These indicators correspond to infrastructure and faculty, respectively, in the graduate programs. The results indicate a high level of confidence, with a maximum margin of error of ±0.0317 and a minimum of ±0.0223 for the 90% confidence interval. For the 95% confidence interval, these margins of error are ±0.0377 and ±0.0266, respectively.

In the analysis of the perception questions, Indicator one presented the lowest mean (0.35), while Indicator four showed the highest mean (0.78). These refer, respectively, to the average proportion of students who reported an intention to drop out and the average proportion who stated that graduates are entering the job market. Results demonstrate a high level of confidence, with a maximum margin of error of ±0.0327 and a minimum of ±0.0275 for the 90% confidence interval. For the 95% interval, the maximum margin of error is ±0.0389 and the minimum is ±0.0328.

Finally, a comparative analysis was conducted between the scores assigned to master's and doctoral programs by CAPES (Sucupira, 2023) and the internal evaluation scores. To illustrate the relationship between internal and external program assessments, Figure 3 shows the mean internal evaluation score corresponding to each CAPES rating.

Figure 3
Mean Internal Evaluation Scores Compared to External Evaluation Scores in UFG Master’s and Doctoral Programs

The analysis suggests that internal and external evaluation scores for master’s programs are not closely correlated. However, this pattern does not apply to all programs, as some show positive or negative correlations between the two evaluations. For instance, the master’s program in Chemical Engineering received 100% adequacy in the internal evaluation but was assigned the lowest external rating (score 3). In contrast, the Ecology and Evolution program achieved high scores in both evaluations, with a CAPES rating of 7 and an internal evaluation score of 80.7%.

For doctoral programs, a more evident correlation exists between internal and external evaluation scores. The doctoral program in Ecology and Evolution, rated 7 by CAPES, also had the second highest internal score (95.3%). Conversely, the doctoral programs in Biotechnology and Biodiversity, and in Communication, had the lowest internal adequacy scores (47.9% and 45.8%, respectively) and received a CAPES rating of 4. Some exceptions remain; for example, Pharmaceutical Sciences had a high internal adequacy score (97.9%) but was assigned a CAPES rating of 4.

6 FINAL CONSIDERATIONS

The limited scientific literature on institutional self-evaluation in graduate education motivated the present study. Most higher education institutions still do not employ assessment tools specifically designed for graduate programs. This is concerning, given the central role of graduate education in academic institutions and its frequent omission from evaluation and policy planning processes. To address this gap, a dedicated questionnaire was developed and administered to students in UFG’s stricto sensu graduate programs.

The main objective was to assess, from the students’ perspective, which factors are decisive for the institutional development of the university and its graduate programs in 2022. More specifically, the study analyzed whether these factors vary by faculty, university, academic unit, graduate program, and by internal and external evaluation scores, as well as academic characteristics such as dropout intentions and future academic and professional plans.

The findings identified academic units with both high and low evaluation scores, providing a basis for targeted improvements. The analysis of student perspectives on future academic and professional trajectories may also inform policies and strategies for graduate program development. Notable discrepancies were observed between internal and external evaluation scores, particularly for items related to program planning, pedagogical design, outreach activities, social engagement, infrastructure, and scientific output. Regarding future perspectives, areas for improvement include addressing the high reported intention to withdraw and strengthening foreign language proficiency.

This research is among the first to address self-evaluation in graduate education and may serve as a reference for further studies, supporting other institutions in integrating graduate programs into their evaluation processes. The results can also guide UFG in developing policies more closely aligned with graduate student needs, fostering greater engagement in institutional activities. It is expected that graduate program assessment at UFG will be further refined and extended to include both lato sensu students and graduate faculty.

• 3
  This category includes academic master’s programs in Human Rights, Environmental Sciences, and Science and Mathematics Education, as well as the professional master’s program in Teaching in Basic Education.

Data Availability

The data supporting this study are not publicly available. They can be requested directly from the authors, with justification.

REFERENCES

• BRASIL. Lei n° 10.861. de 14 de abril de 2004 Institui o Sistema Nacional de Avaliação da Educação Superior (SINAES) e dá outras providências. Brasília: Presidência da República, 2004. Disponível em: https://www.planalto.gov.br/ccivil_03/_ato2004-2006/2004/lei/l10.861.htm Acesso em: 10 fev. 2022.
  » https://www.planalto.gov.br/ccivil_03/_ato2004-2006/2004/lei/l10.861.htm
• BRASIL. Ministério da Educação. Fundação Coordenação de Aperfeiçoamento de Pessoal de Nível Superior. Portaria n° 68, de 3 de maio de 2021 Dispõe sobre a equivalência entre conceitos e notas utilizados nos processos avaliativos realizados pela CAPES, para avaliação dos programas de pós-graduação stricto sensu CAPES, Brasília, 2021b. Disponível em: https://www2.unesp.br/Home/propg/legislacao/portaria-capes-n-68-de-3-de-maiode-2021---dou--.pdf Acesso em: 15 fev. 2022.
  » https://www2.unesp.br/Home/propg/legislacao/portaria-capes-n-68-de-3-de-maiode-2021---dou--.pdf
• BRASIL. Portaria que institui a equivalência entre conceitos e notas utilizados nos processos avaliativos realizados pela CAPES (Portaria nº 68, de 3 de maio de 2021). Brasil, 2021a. Disponível em: https://www2.unesp.br/Home/propg/legislacao/portaria-capes-n-68-de-3-de-maiode-2021---dou--.pdf Acesso em 15 fev. 2022
  » https://www2.unesp.br/Home/propg/legislacao/portaria-capes-n-68-de-3-de-maiode-2021---dou--.pdf
• BRITO, Renato de Oliveira et al Comissão Própria de Avaliação (CPA): sua atuação na construção do diálogo entre comunidade acadêmica e direção da IES. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 26, n. 01. p. 6888, mar. 2021. Disponível em: https://periodicos.uniso.br/avaliacao/article/view/4651 Acesso em 14 maio 2022.
  » https://periodicos.uniso.br/avaliacao/article/view/4651
• CAPES. Coordenação de Aperfeiçoamento do Pessoal de Nível Superior. Ficha de avaliação, Brasília, 2019. Disponível em: https://www.gov.br/capes/pt-br/centrais-deconteudo/10062019-fichaavaliacao-pdf Acesso em 10 out. 2022.
  » https://www.gov.br/capes/pt-br/centrais-deconteudo/10062019-fichaavaliacao-pdf
• CAPES. Coordenação de Aperfeiçoamento do Pessoal de Nível Superior. Sobre a avaliação. Brasília, 2024. Disponível em: https://www.gov.br/capes/pt-br/acesso-ainformacao/acoes-e-programas/avaliacao/sobre-a-avaliacao/avaliacao-o-quee/sobre-a-avaliacao-conceitos-processos-e-normas/conceito-avaliacao Acesso em 10 nov. 2024.
  » https://www.gov.br/capes/pt-br/acesso-ainformacao/acoes-e-programas/avaliacao/sobre-a-avaliacao/avaliacao-o-quee/sobre-a-avaliacao-conceitos-processos-e-normas/conceito-avaliacao
• FARIAS, Ana Maria Lima. Inferência estatística Niterói: Universidade Federal Fluminense. Departamento de Estatística, 2008. Disponível em: https://www.professores.uff.br/anafarias/wpcontent/uploads/sites/210/2021/03/inferencia-1pop-light-00.pdf Acesso em 25 de jan. 2023.
  » https://www.professores.uff.br/anafarias/wpcontent/uploads/sites/210/2021/03/inferencia-1pop-light-00.pdf
• GUAZI, Taísa Scarpin. Diretrizes para o uso de entrevistas semiestruturadas em investigações científicas. Revista Educação, Pesquisa e Inclusão, Boa Vista, RR, v. 2, p. 1-20. 2021. Disponível em: https://revista.ufrr.br/repi/article/view/e202114 Acesso em: 11 nov. 2024.
  » https://revista.ufrr.br/repi/article/view/e202114
• INEP. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Instrumento de avaliação institucional externa presencial e a distância - recredenciamento Brasília, 2017. Disponível em: https://download.inep.gov.br/educacao_superior/avaliacao_institucional/instrumentos/2017/IES_recredenciamento.pdf Acesso em 12 out. 2022.
  » https://download.inep.gov.br/educacao_superior/avaliacao_institucional/instrumentos/2017/IES_recredenciamento.pdf
• LEHFELD, Neide Aparecida de Souza et al Reflexões sobre o processo de autoavaliação institucional: o olhar de uma comissão própria de avaliação. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 15, n. 1. p. 177194, mar. 2010. Disponível em: https://periodicos.uniso.br/avaliacao/article/view/340 Acesso em 17 mai. 2022.
  » https://periodicos.uniso.br/avaliacao/article/view/340
• LIMA JUNIOR, Eduardo Brandão et al Análise documental como percurso metodológico na pesquisa qualitativa. Cadernos da FUCAMP, Campinas, v. 20. n. 47, p. 36-51. 2021. Disponível em: https://www.fucamp.edu.br/editora/index.php/cadernos/article/view/2356/1451 Acesso em 10 fev. 2022.
  » https://www.fucamp.edu.br/editora/index.php/cadernos/article/view/2356/1451
• MEIRINHOS, Manuel; OSÓRIO, António. O estudo de caso como estratégia de investigação em educação. EduSer, Revista de Educação, Bragança, Portugal, v 2, n. 2, p. 49-65, 2016. Disponível em: https://doi.org/10.34620/eduser.v2i2.24 Acesso em 8 nov. 2024.
  » https://doi.org/10.34620/eduser.v2i2.24
• PEIXOTO, Maria do Carmo de Lacerda. Avaliação Institucional Externa no SINAES: considerações sobre a prática recente. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 16, n. 1. p. 11-36, mar. 2011. Disponível em: https://periodicos.uniso.br/avaliacao/article/view/880 Acesso em 14 mai. 2022.
  » https://periodicos.uniso.br/avaliacao/article/view/880
• PINTO, Rodrigo; MELLO, Simone; MELO, Pedro. Meta-avaliação: uma década do Processo de Avaliação Institucional do SINAES. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 21. n. 1. p. 89-107, mar. 2016. Disponível em: https://periodicos.uniso.br/avaliacao/article/view/2509 Acesso em 14 mai. 2022.
  » https://periodicos.uniso.br/avaliacao/article/view/2509
• REIS, Elisabeth et al Estatística aplicada 6. ed. Lisboa - Portugal: Edições Sílabo, 2015. v. 1.
• SAMPAIO, Nilo. Antônio de Souza; ASSUMPÇÃO, Alzira, Ramalho Pinheiro de; FONSECA, Bernardo, Bastos da. Estatística inferencial Belo Horizonte, MG, 2018.
• SUCUPIRA. Plataforma Sucupira. Cursos avaliados e reconhecidos. 2023. Disponível em: https://sucupira.capes.gov.br/sucupira/public/consultas/coleta/programa/quantitativos/quantitativoBuscaAvancada.jsf Acesso em 6 jan. 2023.
  » https://sucupira.capes.gov.br/sucupira/public/consultas/coleta/programa/quantitativos/quantitativoBuscaAvancada.jsf
• TEIXEIRA JUNIOR, Paulo Roberto; RIOS, Mônica Piccione Gomes. Dez anos de SINAES: um mapeamento de teses e dissertações defendidas no período 2004 - 2014. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 22, n. 03, p. 793-816, nov. 2017. Disponível em: https://periodicos.uniso.br/avaliacao/article/view/3150 Acesso em 14 maio 2022.
  » https://periodicos.uniso.br/avaliacao/article/view/3150
• WEBER, Silke. Avaliação e regulação da Educação Superior: conquistas e impasses. Revista Educação & Sociedade, Campinas, v. 31. n. 113, p. 1247-1269, out./dez. 2010. Disponível em: https://www.scielo.br/j/es/a/PC6HmxQWbHFQLgKPrrYF5XD/?lang=pt Acesso em: 18 maio. 2022.
  » https://www.scielo.br/j/es/a/PC6HmxQWbHFQLgKPrrYF5XD/?lang=pt

• Editor de seção:
  André Pires

• Editora de Layout:
  Silmara Pereira da Silva Martins

Publication Dates

• Publication in this collection
  10 Oct 2025
• Date of issue
  2025

History

• Received
  20 June 2023
• Accepted
  06 Mar 2025
• Reviewed
  12 June 2025