Abstract
This study analyzed the results of the National Student Performance Examination (ENADE) in history programs across six editions held between 2004 and 2021 in Brazil. The aim was to understand and critically discuss the changes that occurred in the examination and the underlying conceptions of assessment present in higher education evaluation policies. To this end, the following qualitative methodological procedures were employed: literature review, bibliographical research, and documentary analysis. The sociological and socio-historical approach to the assessment, along with the critical studies adopted as the theoretical-methodological framework, made it possible to understand the connections between assessment paradigms and the theoretical and political conceptions that shaped the ENADE.
Keywords:
assessment policies; ENADE; history undergraduate Courses.
Resumo
O presente estudo analisou os resultados do Exame Nacional de Desempenho de Estudantes no Curso de História, em seis edições realizadas no período compreendido entre 2004 e 2021, com o intuito de apreender e problematizar as mudanças ocorridas no exame e as concepções de avaliação que estiveram presentes nas políticas de avaliação da Educação Superior. Para tanto, elegeram-se os procedimentos metodológicos de abordagem qualitativa, a saber, revisão de literatura, pesquisa bibliográfica e pesquisa documental. A abordagem sociológica e sóciohistórica da avaliação e os estudos críticos que foram adotados como referencial teórico-metodológico possibilitaram compreender os nexos existentes entre os paradigmas de avaliação e as concepções teóricas e políticas que fundamentaram o Exame Nacional de Desempenho de Estudantes.
Palavras-chave:
políticas de avaliação; Enade; cursos de História.
Resumen
El presente estudio analizó los resultados del Examen Nacional del Desempeño de los Estudiantes en el contexto brasileño, en el curso de Historia, en seis ediciones realizadas en el periodo de 2004 a 2021, con el intuito de aprehender y problematizar los cambios ocurridos en el examen y las concepciones de evaluación presentes en las políticas de evaluación de la Enseñanza Superior. Por lo tanto, elegimos os procedimientos metodológicos de abordaje cualitativa, a saber, revisión de literatura, investigación bibliográfica e investigación documental. La abordaje sociológica y socio-histórica de la evaluación y los estudios críticos fueran adoptados como referencial teórico-metodológico, que posibilitaran comprender los nexos existentes entre los paradigmas de evaluación y las concepciones teóricas y políticas que fundamentaran el Examen Nacional del Desempeño de los Estudiantes.
Palabras clave:
políticas de evaluación; Enade; carreras de Historia.
1 INTRODUCTION
In Brazil, the National Student Performance Examination (ENADE) is one of the assessment tools that compose the National System for the Evaluation of Higher Education (SINAES), created by Law No. 10,861 of April 14, 2004 (Brasil, 2004). The exam is officially presented as an instrument designed to assess students’ performance in relation to the content established in undergraduate curricular guidelines.
As ENADE has gained importance in recent years, this article analyzes the results of the examination for History programs across six editions held between 2004 and 2021. It aims at identifying and discussing the underlying conceptions of assessment and examine the possibilities and limitations of this exam in measuring the quality of undergraduate programs within the framework of higher education policies.
The research underpinning this article focused on ENADE documentation related to the history programs, including the examination reports from the 2005, 2008, 2011, 2014, 2017, and 2021 editions. The edition originally scheduled for 2020 was conducted at the end of 2021 due to the COVID-19 pandemic. The choice of history programs reflected both the researchers’ academic background and the feasibility of conducting a detailed and comparative documentary analysis across all ENADE editions by focusing on a single program.
The investigation adopted a qualitative methodological approach. It combined literature review, bibliographical research, and documentary analysis (Alves-MazzottiGewandsznajder, 1999; Bardin, 2009).
The bibliographical research drew on key studies in higher education assessment, particularly those addressing the regulatory and evaluative policies developed by the Brazilian State since the 1990s. Notable contributions include Afonso (2009), Barreyro (2008), Barroso (2005), Costa, Oliveira, and Gomes (2020), Dias Sobrinho (2003, 2010, 2013), Fernandes (2006), Luckesi (2011), Polidori (2009), Rothen and Santana (2018), and Verhine (2015).
The documentary research prioritized legislation related to SINAES - laws, ministerial ordinances, and decrees issued by the Ministry of Education (MEC) - as well as technical reports from the National Institute for Educational Studies and Research Anísio Teixeira (INEP) on ENADE. These included the examinations administered in 2005, 2008, 2011, 2014, 2017, and 2021 (Brasil, 2005; 2008; 2011; 2014; 2017; 2021), the examination manuals for each edition, and the summary reports for history programs.
It is important to highlight the extensive supporting materials produced by INEP, particularly the summary reports for history programs, which contain hundreds of pages in each edition3. These reports include ENADE guidelines, INEP’s analyses of both objective and essay items in the general and history-specific components, correlation studies, student profiles and perceptions, and the levels of difficulty observed in the test items.
Based on these reports, tables were prepared to organize the ENADE results presented and analyzed in this article. The summary reports from the 2005, 2008, 2011, and 2014 editions disclose the performance of history programs and ENADE scores for both teaching degree and bachelor’s degree programs4. In this article, the data are presented in tables following the organization used in the reports, maintaining the distinction between teaching and bachelor’s degrees only for the 2017 and 2021 editions.
This article is organized into three sections. The first discusses ENADE’s role in assessing undergraduate programs, emphasizing institutional procedures for recognizing the quality of higher education in Brazil and the agencies that jointly oversee regulation, assessment, and supervision processes.
The second presents data concerning ENADE - its initial conception, subsequent adjustments, and the results published in the summary reports across six editions of the exam.
The third addresses the limitations of ENADE as a reliable instrument for measuring the quality of history programs and examines the relationship between its assessment paradigms and the political-pedagogical conceptions that guided the examination.
2 ENADE IN THE ASSESSMENT OF UNDERGRADUATE PROGRAM QUALITY
The responsibility for providing and regulating higher education in Brazil is shared among the federal government, federative units, and municipalities. According to the Census of Higher Education (Brasil, 2024), in 2022 the country had 2,595 higher education institutions (HEIs), of which about 88% were private and only 12% public. Among public HEIs, 42.6% were state institutions (133), 38.5% federal (120), and 18.9% municipal (59) (Brasil, 2024).
The MEC, through its secretariats and related agencies, regulates the federal higher education system, which includes federal public institutions and all private HEIs. Consequently, in 2022 approximately 93% of HEIs in Brazil were under MEC’s authority, while state and municipal institutions remained under their respective governments.
Within the organizational framework of INEP, the SINAES includes three types of assessment: institutional, program-based, and student performance. Together they ensure the continued operation of HEIs and undergraduate programs.
Institutional assessment defines the profile and meaning of institutional performance through activities, programs, projects, and sectors. To this end, SINAES established ten institutional dimensions encompassing processes of self-assessment and external assessment (Brasil, 2004).
Program assessment, in turn, focuses on the teaching conditions provided to students, prioritizing three dimensions: (a) didactic-pedagogical organization, (b) faculty profile, and (c) physical facilities. These results support the processes of program authorization, recognition, and renewal (Brasil, 2004).
Student performance is assessed through the ENADE. Official documents define the exam’s objectives as verifying students’ mastery of curricular content established by national guidelines, assessing their ability to adapt to the evolution of knowledge, and assessing their competence to understand issues beyond their specific professional field, such as those related to the Brazilian and global contexts and to other areas of knowledge (Brasil, 2005).
Over time, SINAES underwent changes that moved it away from its original conception. Between 2004 and 2007, it placed greater emphasis on institutions rather than on programs or student learning (Dias Sobrinho, 2010).
However, from 2007 onward, further adjustments reshaped its design, notably with the creation of three quality indicators: the Indicator of Difference between observed and expected performance (indicador de diferença entre os desempenhos observado e esperado - IDD); the preliminary program score (conceito prelimiar de curso - CPC); and the overall program index (índice geral de cursos - IGC).
As a result, ENADE gained prominence in the composition of program and institutional quality indicators. Thus, student performance on standardized exams became increasingly associated with quality itself.
Current legislation establishes that ENADE results contribute to other quality indicators that form the CPC, which measures undergraduate program quality based on four dimensions: ENADE score, value added (IDD), faculty composition (percentage of professors with master’s and doctoral degrees and employment status), and the ENADE student questionnaire. In the CPC weighting, 70% corresponds to students’ ENADE performance and 30% to the institution’s faculty profile (Brasil, 2017).
Thus, SINAES occupies a central role in institutional processes of quality assurance in higher education. In addition to determining program re-accreditation through the CPC, institutions receive a score from the IGC, another quality indicator within SINAES. The IGC is calculated annually using data from undergraduate programs (such as the CPC) and from graduate programs evaluated by CAPES. The IGC score - ranging from 1 to 5 - is also used by external agencies and the media to convey information about higher education quality in Brazil.
The MEC, through its secretariats - particularly the Secretariat of Regulation and Supervision of Higher Education (SERES) - makes regulatory decisions regarding institutional accreditation and program authorization and recognition based on the results of SINAES assessments coordinated by INEP. This regulatory process reflects a conception of education that links the quality of HEIs and their programs to their performance in SINAES indicators - ENADE, CPC, and IGC - with special emphasis on student outcomes in ENADE.
3 ENADE: CONCEPTION, CHANGES, AND TRENDS
ENADE has gained prominence in the assessment of higher education quality. According to the OECD report (2018), Brazil is the only one among the 38 member countries of this organization with a higher education system based on the external assessment of students’ learning outcomes within its regulatory and quality assurance framework.
Based on students’ exam results, an ENADE score is generated for each participating program. This score results from formulas and statistical calculations that have changed throughout the exam’s editions. In the first cycle, between 2005 and 2008, the exam was conducted on a sample basis, and the calculation of each HEI score in each participating area included the results of both incoming and graduating students who took the test.
From 2011 onward, ENADE began assessing only graduating students, and the score obtained became a weighted average of the standardized scores for the respective undergraduate program in general knowledge (accounting for 25%) and specific knowledge (accounting for 75% of the final score). Final scores are distributed across five categories, ranging from 1 to 5, with 1 representing the lowest and 5 the highest performance. Thus, the ENADE score is a conversion of the students’ results in general and specific components, each ranging from 0 to 100. The score attributed to a given program is defined based on the average results of all programs nationwide. This means that a 50% average achieved by a program may correspond to an ENADE score of 5, whereas a 60% average in another program may result in a score of 4 or 3, depending on the national average.
Given the importance of the program averages in calculating each institution’s ENADE score, this article presents the results of the history program’s general and specific tests over the six editions of the exam, as stated in the ENADE summary reports.
3.1 ENADE summary reports: quantitative and qualitative analysis
The ENADE summary reports for history programs are documents prepared by the technical team of INEP that aim to appraise the exam results from both quantitative and qualitative perspectives. These documents highlight the guidelines for History programs, the statistical formulas used to calculate the ENADE score, and general analyses of student performance in the exam, expressed through statistical data presented in graphs, tables, maps, and histograms.
The basic statistics prioritized in the reports include: population size, sample size, number of participants, mean, standard error of the mean, standard deviation, median, minimum score, and maximum score, in addition to the ENADE score (Brasil, 2005). These statistics are used to present the results of the tests, both for the general and specific history components, encompassing both open-ended and multiple-choice questions.
In this study, six ENADE summary reports for history programs were analyzed, corresponding to the 2005, 2008, 2011, 2014, 2017, and 2021 editions. It included reports for programs and institutions as well as other documents related to the exam5.
By analyzing these reports, the study sought to demonstrate how the technical and statistical tools were applied to assess the performance of history students in the general and specific knowledge tests. To this end, tables were created to synthesize the results of all history programs participating in the six editions, with the aim of demonstrating the significance of this exam as a key indicator of the quality of undergraduate programs.
The tables presented in the following sections were developed from data that, in the ENADE summary reports, are available separately for each edition. These data were synthesized and grouped to allow for a broader and comparative analysis across editions, focusing on information regarding the number of participating history programs, the average scores obtained by the programs in the exams, and the ENADE scores distributed by region of the country, administrative category, and academic organization 6.
This study sought to show that ENADE produces reports oriented more toward rankings, comparison, and competitive logic than toward the assessment of students’ learning processes. These reports aim to position undergraduate programs in relation to the results obtained at the national level.
3.2 History programs: ENADE participants (2005-2021)
Table 1 presents the number of history programs that participated in the six editions of ENADE, including both teaching and bachelor’s degrees, between 2005 and 2021, distributed by region and administrative category. It shows the total number of programs nationwide and by region, as well as the percentage of this total divided between public and private institutions. The analysis revealed variations in the number of programs evaluated across the six editions.
History programs participating in ENADE by region and administrative category, encompassing teaching and bachelor’s degree (2005-2021)
The year with the highest participation was 2008 (430 programs), while the lowest was 2017 (341 programs). In 2005, 408 programs participated; in 2011, 385 programs; in 2014, 406 programs; and in 2021, 346 programs7.
In all editions the regional distribution of programs showed the predominance of the Southeast region in the number of programs evaluated, followed by the Northeast, South, and Central-West respectively. The North region consistently presented the lowest number of history programs assessed in all ENADE editions.
Regarding administrative category, Table 1 shows that in the 2005, 2008, and 2011 editions, most history programs evaluated by ENADE belonged to private institutions, with percentages of 56.6% (2005), 56.7% (2008), and 53.8% (2011). In the 2014, 2017, and 2021 editions, public institutions accounted for the majority: 57.6% (2014), 52.8% (2017), and 53.8% (2021). An important difference was observed in the distribution of History programs in Brazil: in the Northeast, North, and Central-West regions, public institutions predominated, while in the Southeast and South regions, private institutions were the majority. This pattern remained across all editions of ENADE, although with varying percentages.
3.3 Average performance of history programs in ENADE exams (2005-2021)
Table 2 presents the average scores of students in the History exams across six ENADE editions, covering the country’s regions. The 2005 and 2008 editions considered the scores of both incoming and graduating students, but from 2011 onward, ENADE began evaluating only graduating students. The average ENADE score includes results from the general knowledge test (10 questions) and the specific knowledge test (30 questions).
The data in Table 2 show that in the first two ENADE editions, with the exception of the North region in 2005, the average scores were higher for graduating students than incoming students across all regions and in the national average. The better performance of graduating students confirms the expected trend in ENADE, since it was expected that program training would foster the competencies assessed in the exam.
In 2005, the highest average was recorded in the South (41.8), followed by the Southeast (41.3), Northeast (40.3), North (39.0), and Central-West (38.8). In 2008, the highest averages were observed in the South (42.2), Southeast (42.1), North (42.0), Central-West (38.7), and Northeast (34.4).
The 2011 edition included only graduating students, with the lowest average in the Southeast (31.6), below the national average (32.8). Once again, the South achieved the highest score (35.3), followed by the Central-West and North (33.9 each) and the Northeast (32.0).
In 2014, only graduating students participated. The highest averages were recorded in the Southeast (44.3), followed by the Central-West (40.0), South (39.8), Northeast (38.7), and North (36.9), with the last three below the national average.
A new feature of the 2017 ENADE summary report (Brasil, 2017) was the presentation of averages and other statistics for both teaching and bachelor's degree programs. In that edition, the regions with the highest averages for teaching programs were the South (49.0), Southeast (46.8), Central-West (42.1), Northeast (41.7), and North (40.9). For bachelor’s programs, the highest average was in the Southeast (48.7), followed by the Northeast (48.1), South (46.6), Central-West (41.5), and North (35.8).
In 2021, the highest averages for teaching programs were in the South (52.4), Southeast (51.7), and Northeast (44.4), while lower averages were observed in the Central-West (43.0) and North (40.7). For bachelor’s programs, the highest average was in the Central-West (57.1), followed by the Southeast (56.4), South (54.7), Northeast (51.1), and North (35.6).
From 2005 to 2021, the South region consistently achieved the highest averages in five of the six ENADE editions for history programs. The lowest averages varied by region: North (2014, 2017, 2021), Southeast (2011), Northeast (2008), and Central-West (2005). Considering the national average, relative to 2005 (40.6), scores decreased in 2008 (38.3) and 2011 (32.8) and increased in 2014, 2017, and 2021
Table 3 presents the national average score of the history exam across the six editions of ENADE, distributed by administrative category. Starting in 2011, the ENADE summary reports replaced the administrative categories federal, state, municipal, and private with the broader categories public and private.
Based on the data in Table 3, it was found that, comparatively, the averages of graduating students were higher than those of incoming students in the 2005 and 2008 editions, both in public and private institutions.
In both editions, the highest averages were observed in federal institutions for both incoming and graduating students. The results also showed that the averages in private institutions were higher than those in state and municipal institutions, the latter presenting the lowest averages - below the national mean in both ENADE editions. Considering that federal, state, and municipal institutions are all public8, the 2005 data indicate that the overall average for incoming students in public institutions (36.6) was higher than in private ones (35.4). Among graduating students, however, the average for private institutions (40.8) exceeded that of public ones (39.2).
In the 2008 edition, the averages of private institutions (36.3 for incoming and 38.2 for graduating students) were higher than the averages of public institutions (34.6 and 39.9, respectively). The lower averages of state and municipal institutions contributed to the overall lower average of public institutions.
In 2011, private institutions again scored higher than public ones. In the last three editions (2014, 2017, and 2021), public institutions achieved higher averages than private institutions, except for bachelor’s programs in private institutions in 2017 (48.3), which exceeded the public average (46.8).
3.4 ENADE scores in history programs (2005-2021)
Table 4 presents the ENADE scores for history programs, distributed by region, for the six editions held between 2005 (Brasil, 2005) and 2021 (Brasil, 2021).
The 2005 edition (Brasil, 2005) included 371 history programs evaluated by ENADE. Of this total, 143 programs (38.5%) obtained score 3; 65 programs achieved score 4 (17.5% of the national total); and 34 programs received score 2. Across Brazil, 11 programs, corresponding to 3% of the total evaluated, achieved the maximum score (5), while 6 programs obtained the minimum score (1). Approximately 30% of the programs (112) received no score. Moreover, with the exception of the North region, where score 2 was the most frequent, score 3 predominated in the other regions (Brasil, 2005).
In the 2008 edition (Brasil, 2008), among the 430 history programs evaluated, 2.6% received the minimum score; 3.7% the maximum score; 36.7% obtained score 3; and 17% received no score. Except for the Northeast region, where the modal value9 (Brasil, 2017) was 2, score 3 was the mode in the other regions.
The 2011 edition (Brasil, 2011) had 385 participating programs, of which 153 (39.8%) were classified with score 3, the modal value. About 34% of the programs obtained score 4; 9.6% received score 2; 8.3% obtained score 5; and 4.6% received score 1. Additionally, 3.6% of the programs received no score. Score 3 was also the modal value in all regions of the country, except in the South, where the mode was 4.
The data in Table 4 show that the 2014 edition (Brasil, 2014) had 364 participating History programs in ENADE. Of this total, 143 (39.3%) were classified with score 3, the modal value; about 32.7% obtained score 2; 13.5% score 4; 7.4% score 1; 3.8% score 5; and 3.3% received no score. Score 3 predominated in the North, Southeast, and South regions. In the Northeast and Central-West regions, the modal value was score 2.
The 2017 edition (Brasil, 2017) of ENADE included 341 History programs. Of this total, 156 (45.7%) were classified with score 3, the modal value. Score 2 was the second most frequent nationwide (22.3%), and score 4 the third (22.0%). Additionally, 12 programs (3.5%) received score 5; 12 programs (3.5%) received score 1; and 10 programs (2.9%) received no score. The distribution of scores by region showed that the North, Northeast, and Southeast regions had score 3 as the modal value. In the Central-West region, the mode was 2, assigned to 40.5% of programs, and in the South region, the mode was 4, assigned to 45.0% of programs.
In the 2021 edition (Brasil, 2021), among the 330 history programs evaluated by ENADE, 112 (33.9%) received score 3 as the modal value. Score 2 was assigned to 90 programs, followed by score 4 (85 programs). Only 19 programs received score 5, and 15 programs score 1. The modal value, represented by score 3, was observed in all regions except the South, where the mode was 4.
Thus, it was found that, nationwide, score 3 was the modal value across all six ENADE editions for history programs. In 2005, it represented 38.5% (Brasil, 2005); 36.7% in 2008 (Brasil, 2008); 39.8% in 2011 (Brasil, 2011); 39.3% in 2014 (Brasil, 2014); 45.7% in 2017 (Brasil, 2017); and 33.9% in 2021 (Brasil, 2021).
Another trend observed in the six ENADE editions, both at national and regional levels, was that most programs were classified as satisfactory. According to ENADE, programs that receive scores 3, 4, or 5 are considered satisfactory, while those that receive scores 1 or 2 are considered unsatisfactory. Thus, the proportion of history programs classified as satisfactory according to ENADE was 59% in 2005, 63.9% in 2008, 82% in 2011, 56.6% in 2014, 71.2% in 2017, and 65.4% in 2021.
The percentage of programs that obtained score 5 - considered excellent by ENADE - was low. In 2005 (Brasil, 2005), only 3% of programs received this score; in 2008, 3.7% (Brasil, 2008); in 2011, 8.3% (Brasil, 2011); in 2014, 3.8% (Brasil, 2014); in 2017, 3.5% (Brasil, 2017); and in 2021, 5.7% (Brasil, 2021).
Table 5 presents the distribution of history programs participating in ENADE by administrative category, according to the scores they obtained. The ENADE summary reports for the 2005 (Brasil, 2005) and 2008 (Brasil, 2008) editions used the administrative categories federal, state, municipal, and private. The 2011 (Brasil, 2011), 2014 (Brasil, 2014), 2017 (Brasil, 2017), and 2021 (Brasil, 2021) editions presented the categories public and private.
The data in Table 5 show that the number of programs evaluated across the six editions of ENADE varied. In 2008 (Brasil, 2008), more programs were assessed than in 2005 (Brasil, 2005), both in public institutions (federal, state, and municipal) and in private ones.
In 2005 (Brasil, 2005), all 11 programs that received the maximum score of 5 were offered by public institutions. A score of 3 was the modal value for programs at state, municipal, and private institutions, while federal institutions had 4 as the mode. That year, 70% of programs from the federal network, 56.2% from the state, 50% from the municipal, and 57.5% from the private were assessed as satisfactory.
In 2008 (Brasil, 2008), among 430 history programs assessed by ENADE, 16 achieved the top score of 5, half from the federal network, two from the state, and six from the private. Except for the state network, whose mode was 2, the modal value for federal, municipal, and private networks was 3. In that edition, 70.7% of federal, 51.0% of state, 54.5% of municipal, and 68.8% of private programs were considered satisfactory.
In 2011 (Brasil, 2011), the national mode (score 3) was the same for public and private institutions. Of the 32 programs rated 5, 20 were private and 12 public. About 75.0% of public and 87.9% of private programs were classified as satisfactory.
Table 5 shows that of the 14 HEIs rated 5, 11 were public and three private. The modal score was 3 for both public and private institutions.
In 2014 (Brasil, 2014), 50% of public and 63.8% of private programs were considered satisfactory. The modal score was 2 in public and 3 in private institutions.
In 2017 (Brasil, 2017), of the 341 participating programs, 180 (52.8%) were public and 161 (47.2%) private. Of the 12 programs rated 5, ten were public and two private. The modal score was 3 for both. About 89.4% of public and 90.7% of private history programs were considered satisfactory.
In 2021 (Brasil, 2021), of the 330 programs assessed, 179 were public and 151 private. The modal score was 3 for both public and private institutions. Of the 19 programs rated 5, 16 were public and three private.
Across the six editions of ENADE, most history programs were assessed as satisfactory. Considering that the categories federal, state, and municipal used in ENADE summary reports for 2005 and 2008 refer to public institutions, the percentages of their programs rated satisfactory were: 58.7% in 2005 (Brasil, 2005); 58.5% in 2008 (Brasil, 2008); 75% in 2011 (Brasil, 2011); 50.5% in 2014 (Brasil, 2014); 89.4% in 2017 (Brasil, 2017); and 68.1% in 2021 (Brasil, 2021). In private institutions, the percentage of satisfactory programs was lower than in public ones in 2005 and 2021, and higher in the other editions.
4 LIMITATIONS OF ENADE AS AN INDICATOR OF THE QUALITY OF HIGHER EDUCATION
The summary reports of the ENADE history examination reveal the potential use of its results for comparing institutional and program performance. It is possible to establish comparisons regarding the average performance of students in the general and specific knowledge tests, as well as in relation to the ENADE score.
Moreover, the ENADE results released by the MEC enable statistical studies correlating variables that seek to explain student performance in the exam, linked to internal and external factors of HEIs and programs.
However, despite these possibilities, many studies have shown the limitations of ENADE as an indicator of the quality of undergraduate programs. These are technicaloperational, methodological, and political-pedagogical hindrances casting doubts on the credibility of the higher education assessment system to evaluate the quality of programs offered by HEIs (Polidori, 2009; Rothen; Barreyro, 2011; Dias Sobrinho, 2013; Haas; Pimenta, 2018; Fernandes, 2022; Fernandes; Gomes, 2025).
One limitation identified in these studies concerns the methodology adopted for the development of the test items in the ENADE exam. The test uses the Classical Test Theory (CTT), a traditional method commonly employed in learning assessment tests. However, this methodology is inappropriate for providing comparative diagnostics of learning progression among students and classes assessed (Fletcher, 1994; Klein, 2009). This means that it is not possible to make comparisons across different editions of ENADE, as the items and tests do not share similar cognitive characteristics. Thus, ENADE results do not allow verification of whether the performance of history students improved from one edition to another, since both the students and the tests change each time.
The appropriate methodology for comparing results across different tests and editions, as in the case of ENADE, is Item Response Theory (IRT), rather than CTT (Suares; Nascimento, 2012). The authors point out that IRT revolutionized the design of cognitive and other types of tests, allowing comparisons between two tests and, consequently, between two or more points in time (Suares; Nascimento, 2012, p. 73).
Furthermore, comparing student and program performance within the same edition of ENADE also presents limitations due to the adopted methodology. The ENADE score assigned to programs results from statistical equations based on the standard average performance. According to a technical note from INEP (Brasil, 2008), the ENADE score is calculated for each analyzed unit, consisting of the set of programs within a specific assessment area (coverage/classification) of the same HEI, in a given municipality.
This score is a conversion of the students’ points obtained in the general and specific components of the test, each ranging from 0 to 100. The score assigned to a given program is defined based on the averages achieved by all programs nationwide. This means that even with low average test scores, a program may achieve the maximum ENADE score if the performance of others is worse. As illustrated in examples discussed in this article, a program’s average score may correspond to an ENADE score of 5 in one edition, while a higher average in another edition may result in ratings of 4 or 3, depending on the national average. Moreover, in the same field - such as History - a student’s 50% score may represent good performance if the test has a high level of difficulty, but in an easier test, that same score cannot be considered good. Since the items are not calibrated and do not follow a standardized difficulty level across editions, it is impossible to determine whether students achieving 50% or 60% in ENADE present good or poor performance (OECD, 2018).
Barreyro (2008) and Dias Sobrinho (2010) highlighted the lack of transparency regarding the final score of each program. In a hypothetical scenario, even if all students obtained low scores in the ENADE test - below 30%, for example - the distribution of averages would be concentrated at the lower end of the performance scale, but programs could still achieve scores of 3 or higher and be considered satisfactory.
Another issue identified is the proportion of graduating students who did not take the test. According to the OECD report (2018), throughout the ENADE editions, the percentage of students required to take the exam but who did not attend ranged from 10% to 15%. Added to this, there is a high percentage of students submitting blank exams or leaving parts of the test unanswered, as well as others marking responses randomly, without taking the exam seriously (OECD, 2018).
The limitations of ENADE generate distrust regarding its ability of objectively assessing student learning in undergraduate programs and, consequently, the quality of those programs. As ENADE has become central to defining program quality, the limitations of its adopted methodology have become more evident. These limitations compromise the validity of its results and of the quality indicators that rely on it as a primary instrument, such as the CPC and the IGC.
The limitations of ENADE are not only technical, operational, or methodological but also paradigmatic in terms of assessment. Dias Sobrinho (2010, 2013) highlights important differences between two assessment paradigms that help explain the changes observed in ENADE. The first, described by the author as a static and summative assessment paradigm, prioritizes final outcomes of the educational process and tends to reflect a legal-bureaucratic order. The second, referred to as the formative and dynamic assessment paradigm, prioritizes processes and changes observed throughout the formative path, “the causalities that lead to understanding the results, and tends to value the formative dimension” (Dias Sobrinho, 2010, p. 216).
The formative conception of assessment presents the following characteristics: it is a continuous and progressive process; a flexible activity within the teaching and learning process; it involves monitoring student learning; it guides educational activity; it promotes the development of each individual. It is also contextualized and integrative rather than exclusionary; it is grounded in dialogue; it emphasizes providing feedback to students throughout the teaching-learning process; and it intervenes in the learning process both collectively and individually (Pinto; Rocha, 2011).
Formative assessment is related to the adoption of an integrated curriculum and requires coherent working proposals for both students and professors, so that assessment and curriculum advance together. Luckesi (2011) also supports this conception, viewing formative assessment as a dialectical moment in the process of advancing educational action, implying symmetry between assessment and curriculum: “As diagnostic, it represents a dialectical moment of sensing the stage one is in and its distance from the perspective that lies ahead as a goal to be reached” (Luckesi, 2011, p. 83).
As can be seen, the characteristics of formative assessment are absent from the evaluative conception identified in ENADE. Restricted to students in the final year of their programs, the exam does not allow for the observation of the development of skills, competencies, and other learning outcomes throughout the formative process. Therefore, the data show that ENADE aligns with the summative function of assessment and, consequently, its intrinsic limitations. Unlike the formative function, summative assessment is not used to modify the teaching-learning process but only to report the performance achieved on the tests.
Enade also presents limitations arising from the asymmetry between assessment and curriculum. Although the test items are developed based on the National Curriculum Guidelines (DCNs) for each program, this does not mean that they are aligned with what is taught and learned within HEIs. It can be argued that this lack of precision prevents causal inferences about the effects of the exam on students’ learning programs. Thus, if the tests are not aligned with what is taught in the programs, they are not useful for understanding what students have learned during their formative trajectory (Fernandes, 2022; Fernandes; Gomes, 2025).
This lack of alignment is even more evident in the ten general knowledge questions that make up the ENADE exams and are the same for all programs. As mentioned, ENADE aims to measure student performance through an assessment composed of 40 questions - ten focusing on general competencies (general education) and 30 focusing on specific disciplinary competencies. In each edition of the exam, the general knowledge questions are common to all programs. These general questions assess broad competencies that go beyond the intentional learning objectives of the programs. Since there is no guarantee that the programs are structured to foster the development of the general competencies assessed by ENADE, it becomes contradictory to evaluate quality based on what may or may not have been developed in the programs.
The contradictions of ENADE also appear in the specific-knowledge questions of each program. It is pertinent to question whether a short test such as ENADE (30 specific-knowledge questions) can effectively assess the types of knowledge and competencies that students are expected to acquire during their training. By having such a limited number of questions in the specific component some subjects within history programs are either not assessed or represented by only one or two questions - often with different approaches from those adopted in the programs - thus affecting the exam’s reliability.
It has been observed that ENADE aligns with the static and summative assessment paradigm (Dias Sobrinho, 2013), distant from the initial proposal rooted in the formative paradigm. As a result, the quality of a program and, by extension, of an institution depends on the performance of students in a single test and their opinions on a few selected items, “without considering the specificities of each field, the differences in prior intellectual capital, and individual commitments and interests” (Dias Sobrinho, 2010, p. 217).
As the static and summative paradigm prevailed, ENADE has become an instrument for classifying programs and institutions, fostering the creation of rankings “of market-oriented assessments and media simplifications, more akin to advertising visibility than to the truth of quality assessment” (Barreyro, 2008, p. 867).
Therefore, ENADE has encouraged a meritocratic logic in higher education by adopting a static assessment paradigm aimed at comparing the scores received by HEI programs. With advances in psychometrics, statistics, technology, and computational resources, ENADE provides data for comparing programs, colleges, university centers, and universities, thus promoting program ranking, competitiveness, and rivalry -typical elements of the “neoliberal rationality” and the market-based conception of quality absorbed into higher education (Dardot; Laval, 2016).
5 FINAL CONSIDERATIONS
This article analyzed the results of the ENADE in history programs across six editions held between 2005 and 2021. The analysis was based on documents issued by the MEC and its affiliated agencies, including INEP, as well as on publications addressing higher education assessment.
It was found that changes in SINAES over time have established ENADE as a key indicator of undergraduate program quality. In practice, this has meant valuing student performance in standardized exams as synonymous with program quality.
The results indicate that ENADE offers several possibilities, as it provides data on student performance in general and specific knowledge tests, as well as the ENADE score for participating programs. The data generated by ENADE reports were found to be significant, as they enabled the understanding of important aspects of History programs. For instance, it was observed that average scores in the specific component of History were lower than those in the general knowledge section. Therefore, the ENADE reports revealed evidence of greater difficulties faced by History students in the specific knowledge portion of the exam. This trend was consistent across the six editions of ENADE analyzed in this article.
Despite the observed possibilities, technical-operational, methodological, and political-pedagogical limitations were identified, casting doubts on the credibility of the higher education assessment system in judging the quality of programs offered by HEIs.
The limitations identified in ENADE, such as the reliance on CTT and the predominance of a summative dimension in the exam, prevent comparisons of student and program performance across different editions. In this sense, ENADE was found to have limitations in determining the exam’s contributions to improving student learning (Dias Sobrinho, 2013; Fernandes, 2022; Fernandes; Gomes, 2025).
The formative conception of assessment emphasizes that processes shape the forms of interaction among individuals and are responsible for student learning. In this regard, the added value of academic programs can only be identified through tools and modalities that assess the learning process, which is not the case of ENADE. A single test administered at the end of an undergraduate program, as in the case of ENADE, merely reveals student performance in that specific exam. Such an instrument is considered insufficient to attribute value to a program or institution, or to evaluate the knowledge acquired throughout the academic journey. Therefore, in the field of learning assessment, there is no justification to uphold the idea that ENADE can measure knowledge accumulated during the student’s academic trajectory or define a program’s quality.
By attributing the quality of history programs to performance in ENADE tests, the higher education assessment and regulation system reproduces a meritocratic logic that views academic success and failure as achievements resulting from individual effort, disregarding the political and social contexts to which institutions and students are linked, contexts marked by diversity and inequality.
Given the hegemony of student performance tests as privileged indicators of quality, despite their limitations, it is worth questioning whether there are alternative assessment policies beyond the neoliberal agenda absorbed by the Brazilian state since the 1990s. Would it be possible to implement a national assessment system that considers the multiple variables and factors influencing educational quality - factors that are often neglected and/or devalued in favor of individual student performance? Alternatively, are we bound to live with regulatory and assessment systems that, although technically sophisticated, reveal little about the teaching and learning processes that actually occur within HEIs?
-
3
A total of 2,300 pages from the ENADE summary reports for the history program were analyzed, including all editions of the exam conducted between 2004 and 2021.
-
4
The Brazilian education system distinguishes between the Bacharelado (Bachelor's Degree) and the Licenciatura (Teaching Degree), with the latter specifically qualifying graduates to teach at the K-12 level.
-
5
INEP, the agency responsible for the technical and operational aspects of ENADE, has managed to maintain a consistent standard of technical quality for the exam throughout its editions, which is recognized by several researchers and assessment specialists (Fernandes, 2022). However, the technical quality of assessment instruments is only one of the elements required for an evaluation process considered appropriate from the perspective of formative assessment. According to Dias Sobrinho (2003), assessment cannot be confused with or reduced to good technical instruments and statistical reports on results obtained in standardized tests.
-
6
The data and tables presented in this article are available in the PhD dissertation: Políticas de avaliação na educação superior: a racionalidade neoliberal na definição de qualidade no Enade, Fernandes, 2022.
-
7
In 2021, 346 history programs were registered to take ENADE, but 330 were actually assessed. A total of 16 programs were registered but had no graduating students, which made their participation in the exam unfeasible.
-
8
To compare the average scores in the 2005 and 2008 editions, the value of the simple arithmetic average was calculated by summing the averages of the federal, state, and municipal (public) institutions and dividing this sum by the number of elements - in this case, three.
-
9
In statistics, the mode is the category or class that appears most frequently in a data set.
ACKNOWLEDGEMENTS
The authors are grateful to the Coordination for the Improvement of Higher Education Personnel (CAPES) for the research funding.
DATA AVAILABILITY
The contents underlying the research text are available in the article.
References
- AFONSO, A. J. Avaliação educacional: regulação e emancipação: para uma sociologia das políticas avaliativas contemporâneas. São Paulo: Cortez, 2009.
- ALVES-MAZZOTTI, A. J.; GEWANDSZNAJDER, F. O método nas ciências naturais e sociais: pesquisa quantitativa e qualitativa. São Paulo: Thomson, 1999.
- BARDIN, L. Análise de conteúdo Lisboa: Edições 70, 2009.
- BARREYRO, G. B. De exames, rankings e mídia. Avaliação: Revista da Avaliação da Educação Superior, Campinas, Sorocaba, v. 13, n. 3, p. 863-868, 2008.
- BARROSO, J. O Estado, a educação e a regulação das políticas públicas. Educação & Sociedade, Campinas, v. 26, n. 92, p. 725-751, 2005.
- BRASIL. Lei nº 10.861, de 14 de abril de 2004. Institui o Sistema Nacional de Avaliação da Educação Superior - Sinaes e dá outras providências. Diário Oficial da União: seção 1, n. 72, p. 3, 2004.
-
BRASIL. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep). Censo da Educação Superior 2023: notas estatísticas. Brasília, DF: Inep, 2024. Disponível em: https://download.inep.gov.br/publicacoes/institucionais/estatisticas_e_indicadores/notas_estatisticas_censo_escolar_2023.pdf Acesso em: 22 mar. 2024.
» https://download.inep.gov.br/publicacoes/institucionais/estatisticas_e_indicadores/notas_estatisticas_censo_escolar_2023.pdf - BRASIL. Ministério da Educação e do Desporto. Portaria Normativa nº 12, de 5 de setembro de 2008 Institui o Índice Geral de Cursos da Instituição de Educação Superior (IGC). Brasília: MEC, 2008.
- BRASIL. Sistema Nacional de Avaliação da Educação Superior - Sinaes. Exame Nacional de Desempenho dos Estudantes - Enade. Relatório Síntese de Área (História) Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep), 2005.
- BRASIL. Sistema Nacional de Avaliação da Educação Superior - Sinaes. Exame Nacional de Desempenho dos Estudantes - Enade. Relatório Síntese de Área (História). Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep), 2011.
- BRASIL. Sistema Nacional de Avaliação da Educação Superior - Sinaes. Exame Nacional de Desempenho dos Estudantes - Enade. Relatório Síntese de Área (História). Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep), 2014.
- BRASIL. Sistema Nacional de Avaliação da Educação Superior - Sinaes. Exame Nacional de Desempenho dos Estudantes - Enade. Relatório Síntese de Área (História) Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep), 2017.
- BRASIL. Sistema Nacional de Avaliação da Educação Superior - Sinaes. Exame Nacional de Desempenho dos Estudantes - Enade. Relatório Síntese de Área (História) Brasília: Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (Inep), 2021.
- COSTA, A. F. de C.; OLIVEIRA, J. F.; GOMES, D. F. Mudanças na avaliação da educação superior no período 2016 a 2019: flexibilização, (des)regulamentação e autorregulação. REVELLI: Revista de Educação, Linguagem e Literatura, Inhumas, n. 12, p. 1-21, 2020.
- DARDOT, P.; LAVAL, C. A nova razão do mundo: ensaio sobre a sociedade neoliberal. Tradução: M. Echalar. São Paulo: Boitempo, 2016.
- DIAS SOBRINHO, J. Avaliação e transformações da educação superior brasileira (1995-2009): do provão ao Sinaes. Avaliação: Revista da Avaliação da Educação Superior, Campinas, Sorocaba, v. 15, n. 1, p. 195-224, 2010.
- DIAS SOBRINHO, J. Avaliação: políticas educacionais e reformas da educação superior. São Paulo: Cortez, 2003.
- DIAS SOBRINHO, J. Educação superior: bem público, equidade e democratização. Avaliação: Revista da Avaliação da Educação Superior, Campinas, Sorocaba, v. 18, n. 1, p. 107-126, 2013.
- FERNANDES, D. Para uma teoria da avaliação formativa. Revista Portuguesa de Educação, Braga, v. 19, n. 2, p. 21-50, 2006.
- FERNANDES, A. de O. Políticas de avaliação na educação superior: a racionalidade neoliberal na definição de qualidade no Enade. 2022. Tese (Doutorado em Educação) - Universidade Federal de Minas Gerais, Belo Horizonte, 2022.
-
FERNANDES, A. de O; GOMES, S. dos S. Das intenções formativas do Sinaes à racionalidade neoliberal da concorrência. Educ. Soc, Campinas, v. 46, e282340, 2025. Disponível em: https://www.scielo.br/j/es/a/cK3fg7GbNbNmsk4jZbXxNcq/ Acesso em: Acesso em: 19 out. 2025.
» https://www.scielo.br/j/es/a/cK3fg7GbNbNmsk4jZbXxNcq/ - FLETCHER, P. R. A Teoria da resposta ao item: medidas invariantes do desempenho escolar. Ensaio: Avaliação e Políticas Públicas em Educação, Rio de Janeiro, v. 1, n. 2, p. 21-28, 1994.
- HAAS, C. M.; PIMENTA, F. de C. R. A valorização da qualidade na avaliação da educação superior: dialogando com os marcos legais. In: TAUCHEN, G.; FÁVERO, A. A. (org.). Avaliação do ensino superior: perspectivas comparadas Curitiba: CRV, 2018. v. 3.
- KLEIN, R. Utilização da Teoria de Resposta ao Item no Sistema Nacional de Avaliação da Educação Básica (Saeb). Ensaio: Avaliação e Políticas Públicas em Educação, Rio de Janeiro, v. 40, n. 11, p. 283-296, 2003.
- LUCKESI, C. C. Avaliação da aprendizagem escolar: estudos e proposições. São Paulo: Cortez, 2011.
-
OCDE - Organização para a Cooperação e Desenvolvimento Econômico. Relatório Repensando a Garantia de Qualidade para o Ensino Superior no Brasil 2018. Disponível em: https://download.inep.gov.br/acoes_internacionais/ocde/Repensando_a_Garantia_de_Qualidade_para_o_Ensino_Superior_no_Brasil_PT.pdf . Acesso em: 13 jul. 2021.
» https://download.inep.gov.br/acoes_internacionais/ocde/Repensando_a_Garantia_de_Qualidade_para_o_Ensino_Superior_no_Brasil_PT.pdf - PINTO, R. O.; ROCHA, M. S. P. M. A avaliação formativa: reflexões sobre o conceito no período de 1999 a 2009. Estudos em Avaliação Educacional, São Paulo, v. 22, n. 50, p. 533-576, 2011.
- POLIDORI, M. M. Políticas de avaliação da educação superior brasileira: Provão, Sinaes, IDD, CPC, IGC e… outros índices. Avaliação: Revista da Avaliação da Educação Superior, Campinas, Sorocaba, v. 14, n. 2, p. 267-290, 2009.
- ROTHEN, J. C.; BARREYRO, G. B. Avaliação da Educação Superior no segundo governo Lula: “Provão II” ou a reedição de velhas práticas? Educ. Soc, Campinas, v. 32, n. 114, p. 21-38, jan./mar. 2011.
- ROTHEN, J. C.; SANTANA, A. da C. M. (org.). Avaliação da educação: referências para uma primeira conversa. Florianópolis: EdUFSCar, 2018.
- SUARES, S. D. S.; NASCIMENTO, P. M. M. Evolução do desempenho cognitivo dos jovens brasileiros no Pisa. Cadernos de Pesquisa, São Paulo, v. 42, n. 145, p. 68-87, jan./abr. 2012.
- VERHINE, R. E. Avaliação e regulação da educação superior: uma análise a partir dos primeiros 10 anos do Sinaes. Avaliação: Revista da Avaliação da Educação Superior, Campinas, Sorocaba, v. 20, n. 3, p. 603-619, 2015.
-
Section Editor:
Milena Pavan Serafim | Layout Editor: Silmara Pereira da Silva Martins
Edited by
-
Translated by:
Celso Rimoli E-mail: traducao@tikinet.com.br
