Abstract
The present study assessed the effect of university scholarships on student enrollment cancellation, as a proxy for dropout, in the University of Fortaleza's Welcome and Retention Incentive Program (PAIP). To do so, the propensity score matching method was employed, using students who applied for PAIP but were not awarded scholarships as the control group. The results showed that the program reduced the enrollment cancellation rate by approximately 3.8%. In a supplementary analysis, no significant differences were found between PAIP and other scholarship programs at the University of Fortaleza, suggesting that the obtained results may be applicable to other programs. Furthermore, an economic viability analysis of the program was conducted, revealing a cost-benefit ratio of R$ 3.01, meaning that for every real invested in the program, there was an average social return of R$ 3.01. The evidence suggests that PAIP is not only effective but also has a favorable cost-return relationship.
Key words:
scholarships; evasion; higher education
Resumo
O presente trabalho avaliou o efeito das bolsas universitárias sobre o cancelamento de matrícula dos alunos, proxy para evasão, no Programa de Acolhimento e Incentivo à Permanência (PAIP) da UFC. Para tanto, empregou-se o método de pareamento por escore de propensão, tomando como grupo de controle os alunos que se candidataram ao PAIP, mas não foram contemplados com bolsas. Os resultados mostraram que o programa reduziu em cerca de 3,8% a taxa de cancelamento de matrícula. Em uma análise complementar verificou-se que não existem diferenças significativas entre o PAIP e outros programas de bolsa da UFC, sugerindo que os resultados obtidos podem ser extensíveis aos demais programas. Ademais, realizou-se uma análise de viabilidade econômica do programa, constatando que ele apresenta uma relação custo-benefício de R$ 3,01, ou seja, para cada real investido no programa, houve um retorno social médio de R$ 3,01. As evidências encontradas indicam que o PAIP não apenas é efetivo, mas possui uma boa relação retorno custo.
Palavras-chave:
bolsas universitárias; evasão; ensino superior
Resumen
El presente estudio evaluó el efecto de las becas universitarias en la cancelación de matrículas de estudiantes, como un indicador de deserción, en el Programa de Acogida e Incentivo a la Permanencia (PAIP) de la Universidad de Fortaleza. Para ello, se empleó el método de emparejamiento por puntaje de propensión, utilizando a los estudiantes que solicitaron el PAIP pero no recibieron becas como grupo de control. Los resultados mostraron que el programa redujo la tasa de cancelación de matrículas en aproximadamente un 3.8%. En un análisis complementario, no se encontraron diferencias significativas entre el PAIP y otros programas de becas de la Universidad de Fortaleza, lo que sugiere que los resultados obtenidos pueden aplicarse a otros programas. Además, se realizó un análisis de viabilidad económica del programa, revelando una relación costo-beneficio de R$ 3.01, lo que significa que por cada real invertido en el programa, hubo un retorno social promedio de R$ 3.01. La evidencia sugiere que el PAIP no solo es efectivo, sino que también tiene una relación favorable entre costos y beneficios.
Palabras clave:
becas universitarias; deserción; educación superior
1 Introduction
Over the past twenty years, Brazil has experienced a significant expansion in the provision of higher education in the federal public network. This movement began in 1998 with the policy of increasing the number of places in federal university courses and gained new momentum after 2003, when the process of expanding these institutions into the interior of the country began-a trend that has been consolidating (Vinhais, 2013). Another important milestone was the Program to Support the Restructuring and Expansion Plans of Federal Universities (REUNI), established in 2007. As a result of this movement, between 1995 and 2020, the total number of enrollments in federal higher education institutions more than tripled, rising from 367,000 to over 1.25 million (Inep, 2022).
However, despite this facilitation of access to higher education, a significant number of students fail to complete their undergraduate courses each year. In 2019, for instance, the dropout rate at this level was 18.4% in public universities and reached 31.6% in distance learning programs (Semesp, 2021) 1. Among the various factors cited in the literature that may explain this dropout rate, the low socioeconomic status of students stands out (Singell Jr., 2004; Dowd; Coury, 2006; Allen et al., 2008). From this perspective, scholarship programs for students become central as policies aimed at ensuring students remain in universities.
In this context, programs that grant scholarships to students, whether based on academic performance or socioeconomic criteria, have increasingly represented a larger share of the budget of Brazilian public universities. Among federal institutions, spending on scholarships jumped from around 4.1% in 2015 to over 14.6% of the discretionary budget in 2019, according to data from the Integrated Planning and Budget System (SIOP) and the Treasury Department2. Despite the significant amount of resources allocated to this policy, relatively few studies in Brazil have sought to causally evaluate the effectiveness of such programs in improving the retention of undergraduate students.
Thus, aiming to contribute to filling this gap in the literature, this study evaluates the Reception and Permanence Incentive Program (PAIP) of the Federal University of Ceará (UFC). This program offers scholarships to undergraduate students to participate in projects aimed at combating dropout rates in their respective courses. Therefore, PAIP acts as an "umbrella" for various dropout prevention projects. However, this study does not intend to evaluate the actions supported by PAIP but rather to assess whether the receipt of scholarships by students involved in these projects affected their dropout rates.
To do so, administrative data from UFC were used, which allowed the observation of students' information at two points in time-during the selection process and one semester after the completion of the program. Unlike other studies, this evaluation was able to analyze students from various undergraduate courses, as well as control for participation in other scholarship programs offered by the institution.
In addition to the impact evaluation, an exercise assessing the program's economic viability was conducted, which indicated a positive social return on the investment in this policy. It is hoped that this case study provides general insights into scholarship policies employed by higher education institutions.
2 Theoretical Framework
School dropout is one of the major challenges facing Brazilian education, whether at the basic or higher levels, with negative effects on students, educational institutions, society, and the economy as a whole. This issue not only leads to a waste of resources but also hinders the development and well-being of those who drop out.
However, the very definition of dropout is controversial, as it is often distinguished from a related phenomenon: school abandonment. According to Inep (1998), "abandonment" refers to a situation where a student leaves school at a certain point but returns the following year, whereas "dropout" refers to a student leaving school and not returning to the educational system. Ideb (2012), in contrast, defines abandonment as a student's departure from the educational system and cessation of school activities without requesting a transfer.
Thus, given its characteristics, measuring dropout is not straightforward, as it requires tracking students even after they leave the educational system to verify whether they return to their studies later. Moreover, the exact moment when a student exits is often unclear, typically recorded as the failure to complete activities at one stage and the absence of enrollment in the next.
Unfortunately, few educational databases in Brazil allow for such individualized longitudinal tracking. As a result, it is not uncommon to use a student’s departure from school as a proxy for dropout. This approach, however, leads to an overestimation of dropout rates since some students who leave during the academic year return at a later time. Although this strategy is valid, given the limitations of available data, it should be used cautiously.
But what causes students to leave their courses before completion? The specialized literature identifies various factors that can be broadly grouped into internal and external factors to the institutions. External factors mainly refer to students' socioeconomic conditions and personal issues, while internal factors concern institutional management issues directly under the responsibility of Higher Education Institutions (HEIs).
Among the main internal factors are the quality of education offered, reflected by the faculty and academic curriculum (Cislaghi, 2008; Albuquerque, 2008), and deficiencies in physical infrastructure (Dias, Theóphilo, & Lopes, 2010). External factors include the lack of professional opportunities (Hotza, 2000; Noronha, Carvalho & Santos, 2001; Albuquerque, 2008), lack of identification with the course (Cardoso, 2008; Barbosa et al., 2016; Daitx, Loguercio, & Strack, 2016; Guimarães, Martins, & Lima, 2019; Souza, Sá, & Castro, 2019; Nascimento & Beggiato, 2020), low participation in academic activities, lack of family support, and poor academic performance (Singell Jr., 2004; Dowd & Coury, 2006; Allen et al., 2008).
Low participation in academic activities is often a result of the need to balance work and study due to students' and their families' limited financial conditions. In this regard, Polydoro (1995) argues that the lack of financial support and work-related issues are frequently cited as reasons for dropping out. Additionally, Wilhelm and Schlosser (2019) and Rocha et al. (2020) also argue that insufficient time to dedicate to studies due to work-related matters or courserelated activities significantly contributes to student attrition.
In this context, Murdock (1989) argues that student benefits play a role in reducing socioeconomic barriers within a university, enabling students from poorer families to enroll in and remain in undergraduate programs. International literature suggests a relationship between receiving financial aid and a reduction in the number of hours dedicated to work (Broton, GoldrickRab, Benson, 2016) and, consequently, an increase in graduation rates (Nora, 2003; Arendt, 2013; Li, 2016; Saccaro, França, Jacinto, 2016).
Despite the strong theoretical foundation supporting the adoption of scholarship policies for university students, empirical studies quantifying the causal impact of these programs on the likelihood of students leaving their courses remain scarce in Brazilian literature. In this regard, the present study aims to contribute to the literature in two aspects: (i) estimating the causal impact of a scholarship program from a major Brazilian educational institution, considering the potential biases inherent in this type of analysis, and (ii) evaluating the economic viability of this policy.
3 The UFC’s Program for Student Support and Retention
The PAIP (Programa de Acolhimento e Incentivo à Permanência) was established at the Federal University of Ceará (UFC) through Resolution No. 08 of the University’s Council for Teaching, Research, and Extension (CEPE) on April 26, 2013. The program aims to implement actions that promote the retention, development, and reintegration of undergraduate students in their respective courses (UFC, 2013).
At the end of each year, the Dean's Office for Undergraduate Studies (Prograd) issues a call for project proposals eligible to receive scholarship recipients under PAIP for the following year. This study focuses on scholarship recipients selected through Call No. 50/2018 (UFC, 2018). According to the call, projects could be proposed by academic program coordinators, undergraduate course coordinators, or full-time faculty members, and should pursue objectives related to: (i) monitoring and evaluating academic actions, (ii) helping students adapt in their first semesters, (iii) reducing student failure and dropout rates, and (iv) improving the quality of teaching and learning in undergraduate courses.
The selection of projects is based on several criteria: alignment of the project's objectives with those of PAIP, whether it is a renewal or new implementation, the plan of activities to be carried out by the scholarship holders, the number of students benefitted, the faculty member's teaching hours, and their participation in University Conferences (Encontros Universitários - EU).
Once proposals are approved, project coordinators must conduct the selection of scholarship recipients through specific calls for applications. The selection process can include one or more of three methods: (i) academic transcript analysis, (ii) theoretical-practical assessment, and/or (iii) interview. Each call's results include a list of ranked and potential candidates, who may be considered for future replacements.
Scholarships, valued at R$ 400.00, are granted for up to nine months, from March to November. Eligible students must be enrolled in at least the second and no later than the penultimate semester of an on-campus undergraduate program. They must also maintain at least twelve hours of weekly coursework during the scholarship period to avoid disqualification from the program. Additionally, scholarship recipients must dedicate twelve hours per week to project-related activities and present their work at the Encontros Universitários.
Scholarships may be renewed, contingent upon performance evaluations, for a maximum period equivalent to the standard duration of the student's degree program. Furthermore, PAIP scholarships cannot be combined with any other scholarships, except for other financial aid not classified as scholarships and granted by UFC itself.
The scholarship is automatically terminated if the recipient's relationship with UFC ends due to graduation, withdrawal, dropout, cancellation, or departure from the course (such as changing courses or transferring), as well as full enrollment suspension or institutional registration. In the latter two cases, students remain enrolled at UFC without attending any courses. Suspension is granted for specific reasons such as illness, relocation, employment, or military obligations. Institutional registration, on the other hand, may occur without justification as long as the student has completed all mandatory coursework from the first two semesters. However, if students remain suspended or institutionally registered for more than four semesters (consecutive or not), their connection to UFC is terminated.
Scholarships will also be canceled for students who fail more than two courses in which they are enrolled. Finally, students may also be disqualified at the request of the project coordinator for failure to meet the program's activities or for frequent unjustified absences.
4 Methodology
4.1 Database
To evaluate the effects of the PAIP, administrative data provided by UFC were used. This information pertains to a cohort of 1,118 students who applied to participate in the projects supported by the program in 2019, distinguishing between those who were actually awarded scholarships (251) and those who received scholarships from other programs (363) 3. The database also includes information about the students' undergraduate programs, date of university admission, number of course failures, and the General Academic Performance Index (IRA-Geral).
These students were monitored, and their academic status was observed 18 months later, in August 2020, eight months after the scholarship period ended. The data provided for this period allowed classification into one of five categories: Active, Canceled, Graduated, Institutional Registration, or Temporarily Withdrawn.
It is important to note that both Institutional Registration and Temporary Withdrawal were very rare, with only four cases (0.35% of the sample). Therefore, only students with active, completed, or canceled enrollment were considered in the analysis.
As a proxy for dropout, the cancellation of enrollment was used, which can occur under the following conditions: exceeding the failure limit due to absences (when the student fails a course more than once or fails four different courses), abandonment4, dual enrollment in undergraduate programs, failure to comply with a study plan5, withdrawal (requested by the student), institutional transfer, course change, administrative decision6, or death.
Given the definition of dropout, where students do not return after an interruption in studies, it is clear that enrollment cancellation includes situations that deviate from this concept. Unfortunately, the data do not allow for differentiation between the reasons for cancellation, so the results should be interpreted with caution. This issue is commonly referred to in the literature as a measurement error, which will not affect the estimators as long as the portion of enrollment cancellations unrelated to dropout (dual enrollment, institutional transfer, course change, administrative decision, or death) is not correlated with the explanatory variables (Greene, 2000).
Therefore, if this latter assumption does not hold, the interpretation should be made in terms of the observed variable (enrollment cancellation) rather than specifically as dropout. Despite this limitation, the fact that temporary absences (temporary withdrawal and institutional registration) can be distinguished from enrollment cancellations suggests that the latter variable strongly correlates with dropout, as the loss of academic ties tends to be more permanent. A detailed description of the variables can be seen in Table 1.
4.2 Empirical Strategy
The causal impact of a policy can be understood as the difference between the outcome achieved by its beneficiaries and the outcome they would have achieved had they not participated in the intervention. This latter scenario is called the counterfactual and cannot be observed, as it is impossible to view the same individuals simultaneously with and without the treatment (Holland, 1986). Thus, evaluation methods attempt to simulate the counterfactual by constructing comparison groups that are ideally identical, on average, to the treated group, except for not receiving the treatment (Foguel, 2017).
In the case of PAIP, since the program was not designed to be evaluated as a controlled experiment, a non-experimental approach was used. The main challenge with this method is constructing a control group that adequately represents the treated group (PAIP scholarship recipients) in the absence of treatment. This is achieved when there is no relationship between the potential outcome (if individuals had been treated) and the treatment assignment. In this case, since students applied to participate in the program, there is a selection bias, as unobservable characteristics of these students, such as motivation, could be related to a lower likelihood of dropping out.
To mitigate this issue, the control group included only students who applied but were not selected for the program. Those students who received scholarships from other UFC programs during the same period were also excluded from the control group. It's important to note that institutional rules allow students to receive financial aid cumulatively with scholarships.
These aids are of an assistance nature, meaning they are aimed at students with lower socioeconomic status. Unfortunately, the data do not allow for identification of whether a student received simultaneous financial aid with the scholarships, which could lead to an underestimation of the effect attributed to PAIP.
Finally, it should be noted that, although PAIP includes projects aimed at reducing dropout, this study only evaluates the effect on students who participated in the program as scholarship recipients and not on the target audience of the linked projects.
4.3 Propensity Score Matching (PSM)
As previously mentioned, the identification strategy consisted of comparing students selected to receive PAIP scholarships (treatment group) with those who applied but were not selected (control group). However, a direct comparison between these two groups may not be appropriate since it is likely that treated students tend to have better academic performance, as they were selected in the process, which could relate to dropout decisions. Therefore, controlling for differences in student characteristics is necessary.
Given the program's characteristics and the available data, the Propensity Score Matching (PSM) method was chosen. The main assumption of this method is that differences between the treatment and control groups can be mitigated by controlling for their observed characteristics (ignorability). More specifically, the counterfactual will consist of individuals whose conditional probability of receiving the treatment is almost identical to that of the treatment group.
The parameter to be estimated is the Average Treatment Effect on the Treated (ATT), which can be understood as the average difference in outcomes between treated and untreated individuals, assuming the latter group represents a good counterfactual for the former in the absence of treatment. The ATT can be expressed in terms of potential outcomes as:
Where Yi is the potential outcome for individual i whether they are treated, 1, or not treated, 0. In this context, the variable Ti indicates whether the individual actually received the treatment. Since the term E[Yi(0)|Ti = 1] cannot be observed, identifying the ATT requires additional assumptions. Specifically, conditions are sought under which the control group values can be used as a counterfactual, such that E[Yi(0)|Ti = 0] = E[Yi(0)|Ti = 1].
In this sense, the matching method assumes that when comparing two individuals, one from the control group and the other from the treatment group, with the same observable characteristics, the only factor differentiating them in terms of outcomes is participation in the program (Pinto, 2017). To address the dimensionality issue, Rosenbaum and Rubin (1983) proposed performing the matching based not directly on observable characteristics, but on the probability of receiving treatment (propensity score), conditional on these characteristics. Thus, the ATT can be obtained by:
Where P(Ti = 1|X) is the conditional probability, based on observed characteristics X, that the individual belongs to the treatment group Ti = 1. For this approach to be valid, two key assumptions are necessary:
Hypothesis H1, known as ignorability, implies that, after controlling for observable characteristics, potential outcomes are independent of treatment status. In other words, when taking these characteristics into account, the evaluator controls for all variables that are related to potential results and that also affect the individual's decision to participate or not. Hypothesis H2 requires that there be individuals in the control group with characteristics equivalent to those in the treatment group, and vice versa.
The propensity score is an estimate of the probability of participating in the program. It can be obtained through a discrete choice model, where the explanatory variables are characteristics that affect the likelihood of participation. Since the selection criteria can involve tests, curriculum analysis, and/or interviews, these criteria are reflected, to some extent, in the explanatory variables of IRA and number of failures, supporting the ignorability hypothesis (H1).
Matching could only be compromised if unconsidered characteristics also affected students' decision to participate. However, since only students who applied were selected for evaluation, it's reasonable to assume that unobserved factors are balanced between groups. This would only be invalid if project coordinators used unobserved characteristics as selection criteria, contrary to program guidelines.
Regarding hypothesis H2, it again relies on the fact that both treatment and control students applied for the treatment, and there is no reason to believe that such characteristics were used as selection criteria, making it plausible that the groups are balanced. Furthermore, it is possible to restrict the estimation to treatment group students for whom the matching was successful (common support), thereby enhancing the internal validity of the estimation.
In summary, the procedure for obtaining the Average Treatment Effect on the Treated (ATT) through Propensity Score Matching (PSM) can be distilled into three main steps: (i) estimation of the probability of receiving treatment (propensity score) using a logit model; (ii) matching the treatment group with the control group based on the propensity scores; and (iii) once the balance between the groups is verified, the program's effect is calculated by taking the difference in means between the matched groups.
There are several possible matching algorithms. Smith (2000) indicates that asymptotically, all matching strategies converge to the same result. However, according to Heckman, Ichimura, and Todd (1997), in small samples, there exists a trade-off between bias and variance of the estimator. In the present study, the kernel matching approach was chosen7, which, according to Caliendo and Kopeinig (2008), tends to produce more efficient estimators in smaller samples.
As noted by Garrido et al. (2014), the kernel matching method consists of calculating weights for each individual in the control group based on their distance from treatment group individuals in terms of the propensity score. Thus, untreated individuals closer to a treated individual tend to receive higher weights, while those further away in terms of the probability of receiving treatment receive lower weights. An additional advantage of kernel matching is the preservation of sample size, as there is less discarding of observations from the control group..
5 Results
5.1 Quality of Matching
The database comprises 1,118 students who applied to participate in the PAIP, all enrolled in 86 undergraduate programs at UFC. Among these, 614 participated in some scholarship program in 2019, with 251 under the PAIP framework. After kernel matching, 725 comparable students remained, consisting of 240 treated and 485 controls.
Table 2 presents the available characteristics of these students before and after matching. They were divided into the treatment group (PAIP scholarship recipients) and the control group (non-recipients). The results indicate that, prior to matching, students in the treatment group had a higher General IRA (+0.54), had been in university for a longer period (+0.4 semesters), and had a slightly lower number of failures. This aligns with the PAIP selection guidelines, which direct coordinators to select students based on evaluations and curricular analysis. Conversely, after matching, no statistically significant differences were observed between the characteristics of the two groups, suggesting that the method was effective in mitigating the differences between them.
In addition to the above variables, a dummy variable indicating the course in which the student was enrolled was considered as an observable characteristic for matching8. Graph 1 illustrates the percentage difference between treated and untreated students enrolled in each of the 68 undergraduate programs whose students participated in the PAIP selection process. It is noteworthy that, after matching, the percentage difference significantly decreases, predominantly centering around zero. In other words, on average, the groups have the same composition in terms of students from different courses. These indications reinforce that the matching was successful in aligning the observable characteristics of the two groups.
In the same vein, Table 3 presents the results of the Logit model used to construct the propensity score, which measures the effect of the observed characteristics on the likelihood of participation in the program. It can be observed that higher values of the General IRA and the number of semesters tend to increase the probability of a student being selected, while the number of failures appears to have no effect on this measure. This suggests that the selection of scholarship recipients by project coordinators adhered to the PAIP guidelines, which assert that selection should be based on examinations, curriculum analysis, and/or interviews.
Moreover, the model appears to be adequately fitted, as most coefficients were statistically significant and had the expected signs. Furthermore, the global Wald significance test rejects the null hypothesis that the coefficients are jointly equal to zero. As a robustness check, the estimation was repeated after matching, revealing a reduction, as expected, in the Pseudo-R2 from 0.11 to 0.01 (not significant), indicating that after matching, the observed characteristics no longer allow for distinguishing the two groups.
Another indication that the matching was successful can be visualized in Graph 2, which presents the probability distribution of the propensity score. It is apparent that there is an overlap of the two curves due to PSM, indicating that after this procedure, the treatment probabilities became equivalent between treated and untreated individuals, signaling compatibility between the two groups.
The greater balance between the groups, as well as the overlap of the propensity scores after matching, corroborate the satisfaction of the ignorability (H1) and common support (H2) hypotheses, such that the estimator obtained according to Equation 2 provides a good approximation of the average program effect on the benefited group of students (ATT).
5.2 Results
Table 4 presents the results of the estimation of the ATT. It is observed that the PAIP has a significant effect on the dropout rate, serving as a proxy for student attrition. The average observed impact was a reduction of 3.8 percentage points in the dependent variable, which is lower than the pre-matching results of -5.9 percentage points. This difference indicates that, in the absence of the procedure, a direct comparison between treated and control groups would produce significantly biased estimates, indicating an effect 55% greater than the actual one.
To assess the heterogeneity of results, the procedures were repeated for other subgroups of students. As illustrated in Table 5, students who did not receive scholarships under the PAIP, nor under any other UFC program, were compared with those who received another scholarship and with those who received any scholarship (PAIP or otherwise). Finally, PAIP scholarship recipients were compared with beneficiaries of other scholarship programs.
The results are consistent with the hypothesis that there are no significant differences between the programs in terms of reducing the dropout rate, with both exerting a negative effect of around 4% on student attrition. This evidence lends some external validity to the results; it can be inferred that providing scholarships to university students, regardless of the modality, is an effective policy for reducing dropout rates in higher education.
5.3 Economic Return Analysis
The mere evidence of the effectiveness of a program is not sufficient to recommend its adoption as a public policy; it is essential to ensure that the social return generated by the program exceeds the costs of its implementation (Peixoto, 2017). Therefore, an economic return analysis of the PAIP at UFC was conducted. To achieve this, it is necessary to translate the benefit of reduced attrition into monetary terms, enabling a comparison with the cost of the intervention.
In this regard, two primary angles were identified through which attrition in higher education impacts the economy: (i) the loss of value invested in students who drop out without completing their courses, and (ii) the loss of productivity of these students upon entering the labor market due to their lower qualifications compared to professionals with higher education.
Thus, to make the calculation feasible, several assumptions must be made. First, it was noted that PAIP students entered university approximately 6.7 semesters (or 3.36 years) ago, so it is assumed that the median student enrolled in early 2016 and would drop out around mid2019 (in May). It is also assumed that they would have graduated, had they not dropped out, by the end of 2020, thus starting to benefit from the diploma in 2021.
To identify the annual investment per student, estimates from the Ministry of Education (Brazil, 2018) were used, indicating that the average cost per student at UFC in 2016 was R$ 48,346.10. Considering the average inflation rate measured by the IPCA, this amount would correspond to R$ 53,780.53 in average values for 2019. Therefore, each student who dropped out would have cost, in nominal terms, R$ 180,732.59 over the time they spent at the institution (annual cost multiplied by average length of stay).
On the other hand, dropouts forfeit a future wage premium due to their education. According to data from RAIS, in 2019, the average monthly income of a Ceará worker with incomplete higher education was R$ 1,492.40, while those with completed higher education earned an average of R$ 2,620.40, resulting in a wage differential of R$ 1,128.54. Thus, considering the vacation bonus (one-third of the salary) and the Christmas bonus, the average annual productivity loss would be R$ 15,047.20 per dropout. This loss was assumed to extend for four years after graduation, given that the student could return to studies or find another source of income unrelated to their qualification level.
The choice of this time frame took into account the results of Marques (2020), who calculated the return on investment in education among dropouts using data from the 2019 Higher Education Census. The author concluded that 57.8% of students who dropped out resumed their studies within eight years, with 91% doing so within four years after dropping out. Furthermore, it is worth noting that this choice was conservative, as the uncertainty regarding the future salary of young people increases over time, thus limiting the analysis horizon was deemed appropriate.
When working with financial values, one must always consider the opportunity cost of the invested capital; that is, if the investment in the education of the dropouts had not been made, this resource could, for example, have been used to pay down the total federal government debt each year, thereby reducing interest expenses. In this exercise, a real interest rate of 3.6% per annum, offered by government bonds maturing in five years (IPCA+ 2026), was considered. Thus, a cash flow of losses resulting from attrition was constructed, and the Net Present Value for the year 2019 was calculated, following Peixoto (2019). The analysis presented can be seen in Table 6.
With these values, it is possible to calculate the program's benefit by multiplying the estimated impact by the cost arising from attrition, as illustrated in Table 7. The resulting average benefit was R$ 9,553.29 per student, which, when multiplied by 251 students served, led to an estimated total benefit of R$ 2,397,876.25 for the PAIP. This amount is significantly greater than the investment made in the program's implementation. According to data from the Integrated System of Property, Administration, and Contracts (SIPAC), provided by the UFC's Planning and Administration Pro-Rectory, R$ 795,600.00 was allocated for program scholarships in 2019, resulting in a cost-benefit ratio of 3.01; in other words, for every R$ 1.00 invested in the program, there was a social return of R$ 3.01.
It is important to note that changes in the parameters used may affect the return measure; however, considering the significant cost-benefit ratio (3.01), even if these changes reduced the benefit by up to two-thirds, the program would still make sense from an economic perspective. Thus, based on the arguments presented, it can be concluded that the PAIP is not only effective in reducing the dropout rate but also generates social benefits that justify its funding.
6 Conclusion
The present study aimed to evaluate the effect of receiving university scholarships on the probability of dropout among higher education students. Specifically, it assessed the PAIP at UFC, which provides scholarships to students linked to projects aimed at reducing attrition. In this regard, a cohort of students who benefited from the program in 2019 was tracked to observe whether they were still enrolled the following year.
To this end, the propensity score matching method was employed, using students who applied for scholarships but were not awarded one as candidates for the control group. The results showed that the program led to a reduction of approximately 3.8% in the dropout rate associated with enrollment cancellations. In a heterogeneity analysis, the PAIP beneficiaries were compared with those awarded scholarships from other programs, revealing no significant differences between the two groups. This suggests that the results obtained for the PAIP may be extendable to other scholarship programs at UFC. These findings are consistent with those found in the literature (Murdock, 1989; Nora, 2003; Gross; Hossler; Ziskin, 2007; Arendt, 2013).
Furthermore, a feasibility analysis of the program was conducted, determining that it generates an average benefit of approximately R$ 9,553.29 per student, resulting in a benefitcost ratio of 3.01; that is, for every real invested in the program, there was an average social return of R$ 3.01. Thus, even with significant changes to the underlying assumptions, the program would still prove viable.
Therefore, the evidence gathered leads to the conclusion that the PAIP is not only effective but also demonstrates a favorable cost-return ratio. The results also suggest that these properties may be, to some extent, generalized to other university scholarship programs, thereby justifying the resources invested in this policy.
-
1
There is no consensus on the most appropriate way to measure dropout rates. The Semesp Institute defines the dropout rate as the sum of students who have withdrawn (those with suspended enrollments, those who have discontinued a specific course, and deceased students) divided by the total number of students.
-
2
Expenses on student scholarships were considered those committed and classified under expenditure category 3.3.90.18.01 (Scholarships in the Country) in the discretionary budget actions 20RK (Operation of Federal Higher Education Institutions), 4002 (Higher Education Student Assistance), and 20GK (Support for Undergraduate, Graduate, Teaching, Research, and Extension Activities).
-
3
In addition to the PAIP, UFC has 18 other scholarship programs, also governed by Resolution CEPE No. 08/2013, Article 7. These programs encompass various modalities, such as student assistance, scientific initiation, initiation to teaching, as well as incentives for cultural and/or sports practices, among others.
-
4
Here, “abandonment” is understood as the absence of enrollment, leave of absence, or institutional enrollment for more than four academic semesters, whether consecutive or not.
-
5
A study plan is a set of mandatory curricular components and elective credit hours chosen to be completed within a specified period. It applies when a student exceeds the expected maximum time for completing their degree.
-
6
This decision may result from a penalty for an infraction as stipulated in UFC’s statute.
-
7
The radius and nearest neighbor algorithms were also tested, with consistent results observed.
-
8
The use of this variable in the matching process is justified by the plausible assumption that students from different courses may drop out for different reasons. For instance, daytime courses may be incompatible with the need to work. Furthermore, courses with lower admission scores may attract students with lesser aptitude for the profession, who chose that course due to an inability to access a more competitive one.
References
-
ALBUQUERQUE, T. Do abandono à permanência num curso de ensino superior. Sísifo, Revista de Ciências da Educação, Lisboa, n. 7, p. 19-28, set./dez. 2008. Disponível em: https://sisifo.ie.ulisboa.pt/index.php/sisifo/article/viewFile/115/189 Acesso em: 07 out. 2024.
» https://sisifo.ie.ulisboa.pt/index.php/sisifo/article/viewFile/115/189 -
ALLEN, J. et al Third-year college retention and transfer: Effects of academic performance, motivation, and social connectedness. Research in Higher Education, Georgia, v. 49, n. 7, p. 647664, 2008. Disponível em: https://doi.org/10.1007/s11162-008-9098-3 Acesso em: 07 out. 2024.
» https://doi.org/10.1007/s11162-008-9098-3 -
ARENDT, J. N. The effect of public financial aid on dropout from and completion of university education: evidence from a student grant reform. Empirical Economics, [S. l.], v. 44, n. 3, p. 1545-1562, 2013. Disponível em: https://doi.org/10.1007/s00181-012-0638-5 Acesso em: 07 out. 2024.
» https://doi.org/10.1007/s00181-012-0638-5 -
BARBOSA, R. M. S. et al Comparação da qualidade nutricional das refeições antes e depois do Plano Nacional de Assistência Estudantil no RJ, Brasil. Nutrición clínica y dietética hospitalaria, [S. l.], v. 36, n. 2, p. 74-82, 2016. Disponível em: https://doi.org/10.12873/362barbosaroseane Acesso em: 07 out. 2024.
» https://doi.org/10.12873/362barbosaroseane -
BRASIL. MINISTÉRIO DA EDUCAÇÃO. Nota Técnica MEC/SE Nº 4, de 22 de fevereiro de 2018. Disponível em: http://www.forplad.andifes.org.br/sites/default/files/forplad/comissaoplanejamento/NT_04-2018_e_anexos_-_apura%C3%A7%C3%A3o_do_custo_das_universidades.pdf Acesso em: 07 out. 2024.
» http://www.forplad.andifes.org.br/sites/default/files/forplad/comissaoplanejamento/NT_04-2018_e_anexos_-_apura%C3%A7%C3%A3o_do_custo_das_universidades.pdf -
BROTON, K. M.; GOLDRICK-RAB, S.; BENSON, J. Working for college: the causal Impacts of financial grants on undergraduate employment. Educational Evaluation and Policy Analysis, [S. l.], v. 38, n. 3, p. 477-494, 2016. Disponível em: https://doi.org/10.3102/0162373716638440 Acesso em: 07 out. 2024.
» https://doi.org/10.3102/0162373716638440 -
CALIENDO, M.; KOPEINIG, S. Some practical guidance for the implementation of propensity score matching. Journal of economic surveys, [S. l.], v. 22, n. 1, p. 31-72, 2008. Disponível em: https://doi.org/10.1111/j.1467-6419.2007.00527.x Acesso em: 07 out. 2024.
» https://doi.org/10.1111/j.1467-6419.2007.00527.x -
CARDOSO, C. B. Efeito da política de cotas na Universidade de Brasília: uma análise do rendimento e da evasão 2008. Dissertação (Mestrado em Educação) - Universidade de Brasília, Brasília, 2008. Disponível em: https://flacso.redelivre.org.br/files/2012/07/44.pdf Acesso em: 07 out. 2024.
» https://flacso.redelivre.org.br/files/2012/07/44.pdf -
CISLAGHI, R. Um modelo de sistema de gestão do conhecimento em um framework para a promoção da permanência discente no ensino de graduação 2008. Tese (Doutorado em Engenharia e Gestão do Conhecimento) - Universidade Federal de Santa Catarina, Florianópolis, SC., 2008. Disponível em: https://repositorio.ufsc.br/bitstream/handle/123456789/91544/250753.pdf Acesso em: 07 out. 2024.
» https://repositorio.ufsc.br/bitstream/handle/123456789/91544/250753.pdf -
DAITX, A. C.; LOGUERCIO, R. Q.; STRACK, R. Evasão e retenção escolar no curso de licenciatura em química do Instituto de Química da UFRGS. Investigações em Ensino de Ciências, Rio grande do Sul, v. 21, n. 2, p. 153-179, 2016. Disponível em: https://doi.org/10.22600/15188795.ienci2016v21n2p153 Acesso em: 07 out. 2024.
» https://doi.org/10.22600/15188795.ienci2016v21n2p153 -
DIAS, E. C. M.; THEÓPHILO, C. R.; LOPES, M. S. Evasão no ensino superior: estudo dos fatores causadores da evasão no curso de Ciências Contábeis da Universidade de Montes Claros - UNIMONTES - MG. In: CONGRESSO USP DE INICIAÇÃO CIENTÍFICA EM CONTABILIDADE, 7., 2010, São Paulo. Anais [...]. São Paulo: UNIOMONTES, 2010. p. 1-6. Disponível em: https://congressousp.fipecafi.org/anais/artigos102010/419.pdf Acesso em: 07 out. 2024.
» https://congressousp.fipecafi.org/anais/artigos102010/419.pdf -
DOWD, A. C.; COURY, T. The effect of loans on the persistence and attainment of community college students. Research in higher education, [S. l.], v. 47, n. 1, p. 33-62, 2006. Disponível em: https://doi.org/10.1007/s11162-005-8151-8 Acesso em: 07 out. 2024..
» https://doi.org/10.1007/s11162-005-8151-8 -
FOGEL, M. Modelos de Resultados Potenciais. In: MENEZES FILHO, N.; PINTO, C. (org.). Avaliação econômica de projetos sociais São Paulo: Fundação Itaú Social, 2017. p. 39-54. Disponível em: https://repositorio-api.insper.edu.br/server/api/core/bitstreams/4014ebc1-63864221-9b07-88c42a79569e/content Acesso em: 07 out. 2024.
» https://repositorio-api.insper.edu.br/server/api/core/bitstreams/4014ebc1-63864221-9b07-88c42a79569e/content -
GARRIDO, Melissa M. et al Methods for constructing and assessing propensity scores. Health services research, [S. l.], v. 49, n. 5, p. 1701-1720, 2014. Disponível em: https://doi.org/10.1111/1475-6773.12182 Acesso em: 07 out. 2024.
» https://doi.org/10.1111/1475-6773.12182 - GREENE, William H. Econometric analysis New Jersey: Prentice Hall, 2000.
-
GROSS, J. P.; HOSSLER, D.; ZISKIN, M. Institutional aid and student persistence: An analysis of the effects of institutional financial aid at public four-year institutions. Journal of Student Financial Aid, [S. l.], v. 37, n. 1, p. 6, 2007. Disponível em: https://doi.org/10.55504/08849153.1076 Acesso em: 07 out. 2024.
» https://doi.org/10.55504/08849153.1076 -
GUIMARÃES, O. M.; MARTINS, E. C. B.; LIMA, M. J. O. Evasão no Ensino Superior: A UNESP Campus de Franca - Período de 2013-2018. Revista Camine: Caminhos da Educação, São Paulo, v. 11, n. 1, 136-161, 2019. Disponível em: https://seer.franca.unesp.br/index.php/caminhos/article/view/3013 Acesso em: 07 out. 2024.
» https://seer.franca.unesp.br/index.php/caminhos/article/view/3013 -
HECKMAN, J. J.; ICHIMURA, H.; TODD, P. E. Matching as an econometric evaluation estimator: Evidence from evaluating a job training programme. The review of economic studies, [S. l.], v. 64, n. 4, p. 605-654, 1997. Disponível em: https://doi.org/10.2307/2971733 Acesso em: 07 out. 2024.
» https://doi.org/10.2307/2971733 -
HOLLAND, Paul W. Statistics and causal inference. Journal of the American statistical Association, [S. l.], v. 81, n. 396, p. 945-960, 1986. Disponível em: https://doi.org/10.1080/01621459.1986.10478354 Acesso em: 07 out. 2024.
» https://doi.org/10.1080/01621459.1986.10478354 -
HOTZA, M. A. S. O abandono nos cursos de graduação da UFSC em 1997: a percepção dos alunos-abandono. 2000. Dissertação (Mestrado em Psicologia) - Universidade Federal de Santa Catarina, Florianópolis, 2000. Disponível em: https://repositorio.ufsc.br/handle/123456789/79172 Acesso em: 07 out. 2024.
» https://repositorio.ufsc.br/handle/123456789/79172 -
IDEB - Índice de Desenvolvimento da Educação Básica. Formação em ação. 2012. Disponível em: http://www.nre.seed.pr.gov.br/cascavel/File/CIENCIAS_IndicedeDesenvolvimentodaEducaçãoBásica.pdf Acesso em: 30 set. 2023.
» http://www.nre.seed.pr.gov.br/cascavel/File/CIENCIAS_IndicedeDesenvolvimentodaEducaçãoBásica.pdf -
INEP - Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Sinopses estatísticas da educação superior 2022. Disponível em: https://www.gov.br/inep/pt-br/acesso-ainformacao/dados-abertos/sinopses-estatisticas/educacao-superior-graduacao Acesso em: 4 mar. 2022.
» https://www.gov.br/inep/pt-br/acesso-ainformacao/dados-abertos/sinopses-estatisticas/educacao-superior-graduacao -
INEP. Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira. Informe estatístico do MEC revela melhoria do rendimento escolar 1998. Disponível em: https://www.gov.br/inep/pt-br/assuntos/noticias/censo-escolar/informe-estatistico-do-mecrevela-melhoria-do-rendimento-escolar Acesso em: 30 set. 2023.
» https://www.gov.br/inep/pt-br/assuntos/noticias/censo-escolar/informe-estatistico-do-mecrevela-melhoria-do-rendimento-escolar -
LI, D. L. O novo ENEM e a plataforma SiSU: efeitos sobre a migração e a evasão estudantil. 2020. Tese (Doutorado) - Universidade de São Paulo, São Paulo, 2020. Disponível em: https://doi.org/10.11606/D.12.2016.tde-23112016-094256 Acesso em: 07 out. 2024.
» https://doi.org/10.11606/D.12.2016.tde-23112016-094256 -
MARQUES, F. T. A volta aos estudos dos alunos evadidos do ensino superior brasileiro. Cadernos de Pesquisa, São Paulo, v. 50, p. 1061-1077, 2020. Disponível em: https://doi.org/10.1590/198053147158 Acesso em: 07 out. 2024.
» https://doi.org/10.1590/198053147158 -
MURDOCK, T. A. Does financial aid really have an effect on student retention? Journal of Student Financial Aid, [S. l.], v. 19, n. 1, p. 1, 1989. Disponível em: https://doi.org/10.55504/08849153.1516 Acesso em: 07 out. 2024.
» https://doi.org/10.55504/08849153.1516 -
NASCIMENTO, L. C. S.; BEGGIATO, S. M. O. Evasão escolar na graduação em Musicoterapia da Universidade Estadual do Paraná. Educ. Form, Curitiba, v. 5, n. 3, p. e2080-e2080, 2020. Disponível em: https://doi.org/10.25053/redufor.v5i15set/dez.2080 Acesso em: 07 out. 2024.
» https://doi.org/10.25053/redufor.v5i15set/dez.2080 - NORA, A. The majority in the minority: Expanding the representation of Latina/o faculty, administrators, and students in higher education. Londres: Routledge, 2003.
-
NORONHA, A. B.; CARVALHO, B. M.; SANTOS, F. F. F. Estudo do perfil dos alunos evadidos da faculdade de economia, administração e contabilidade - Campus Ribeirão Preto. In: ENCONTRO NACIONAL DA ASSOCIAÇÃO NACIONAL DOS PROGRAMAS DE PÓS-GRADUAÇÃO EM ADMINISTRAÇÃO (ANPAD), 25., 2001, Campinas. Anais [...]. Campinas: ANPAD, 2001. p. 1-18. Disponível em: https://repositorio.usp.br/item/001218202 Acesso em: 07 out. 2024.
» https://repositorio.usp.br/item/001218202 -
PEIXOTO, B. O Cálculo do retorno econômico. In: MENEZES FILHO, N..; P. C. (org.) Avaliação econômica de projetos sociais São Paulo: Fundação Itaú Social, 2017. p. 111-144. Disponível em: https://repositorio-api.insper.edu.br/server/api/core/bitstreams/4014ebc1-6386-42219b07-88c42a79569e/content Acesso em: 07 out. 2024.
» https://repositorio-api.insper.edu.br/server/api/core/bitstreams/4014ebc1-6386-42219b07-88c42a79569e/content -
PINTO, C. Pareamento. In: MENEZES FILHO, N. P. C. (org.) Avaliação econômica de projetos sociais São Paulo: Fundação Itaú Social, 2017. p. 111-144. Disponível em: https://repositorioapi.insper.edu.br/server/api/core/bitstreams/4014ebc1-6386-4221-9b07-88c42a79569e/content Acesso em: 07 out. 2024.
» https://repositorioapi.insper.edu.br/server/api/core/bitstreams/4014ebc1-6386-4221-9b07-88c42a79569e/content -
POLYDORO, S. A. J. Evasão em uma instituição de ensino superior: desafios para a psicologia escolar. 1995. Dissertação (Mestrado em Psicologia) - Pontifícia Universidade Católica de Campinas, Campinas, 1995. Disponível em: https://www.scienceopen.com/document?vid=ebaba5e8-80ef-4810-8f16-3222d17fe3f5 Acesso em: 07 out. 2024.
» https://www.scienceopen.com/document?vid=ebaba5e8-80ef-4810-8f16-3222d17fe3f5 -
ROCHA, J. S. et al Evasão escolar no curso de licenciatura em química no IFPR Campus Paranavaí: um levantamento das possíveis causas. Brazilian Journal of Development, São José dos Pinhais, v. 6, n. 4, p. 20778-20797, 2020. Disponível em: https://doi.org/10.34117/bjdv6n4-305 Acesso em: 07 out. 2024.
» https://doi.org/10.34117/bjdv6n4-305 -
ROSENBAUM, P.; RUBIN, D. The central role of the propensity score in observational studies for causal effects. Biometrika, Grã-Bretanha, v. 70, p. 41-50, 1983. Disponível em: https://doi.org/10.1093/biomet/70.1.41 Acesso em: 07 out. 2024.
» https://doi.org/10.1093/biomet/70.1.41 -
SACCARO, A.; FRANÇA, M. T. A.; JACINTO, P. A. Retenção e evasão no ensino superior brasileiro: uma análise dos efeitos da bolsa permanência do PNAES. In: ENCONTRO NACIONAL DE ECONOMIA-ANPEC, 44., 2016, Foz do Iguaçu. Anais [...]. Foz do Iguaçu: ANPEC, 2016. Disponível em: https://repositorio.pucrs.br/dspace/bitstream/10923/10648/2/Retensao_e_evasao_no_ensino_superior_brasileiro_uma_analise_dos_efeitos_da_bolsa_permanencia_do_PNAES.pdf Acesso em: 07 out. 2024.
» https://repositorio.pucrs.br/dspace/bitstream/10923/10648/2/Retensao_e_evasao_no_ensino_superior_brasileiro_uma_analise_dos_efeitos_da_bolsa_permanencia_do_PNAES.pdf -
SEMESP. Mapa do ensino superior no Brasil 2021 São Paulo: Semesp, 2021. Disponível em: https://www.semesp.org.br/mapa-do-ensino-superior/edicao-11/download/ Acesso: 03 mar. 2022.
» https://www.semesp.org.br/mapa-do-ensino-superior/edicao-11/download/ -
SINGELL JR, L. D. Come and stay a while: does financial aid effect retention conditioned on enrollment at a large public university?. Economics of Education review, v. 23, n. 5, p. 459-471, 2004. Disponível em: https://doi.org/10.1016/j.econedurev.2003.10.006 Acesso em: Acesso em: 07 out. 2024.
» https://doi.org/10.1016/j.econedurev.2003.10.006 -
SMITH, J. A Critical survey of empirical methods for evaluating active labor market policies. Schweizerische Zeitschrift fr Volkswirtschaft und Statis-tik, [S. l.], v. 136, n. 3, p. 1-22, 2000. Disponível em: https://www.econstor.eu/handle/10419/70423 Acesso em: Acesso em: 07 out. 2024.
» https://www.econstor.eu/handle/10419/70423 -
SOUZA, T. S.; SÁ, S.; CASTRO, P. A. Evasão escolar no ensino superior: um estudo qualitativo via mapeamento de licenciaturas. Revista Lusófona de Educação, Lisboa, v. 44, p. 63-82, 2019. Disponível em: https://doi.org/10.24140/issn.1645-7250.rle44.04 Acesso em: 07 out. 2024.
» https://doi.org/10.24140/issn.1645-7250.rle44.04 -
UFC - UNIVERSIDADE FEDERAL DO CEARÁ. Edital nº 50/2018. Disponível em: https://prograd.ufc.br/wp-content/uploads/2018/11/edital-50-bolsa-de-apoio-a-projetos-de-graduaCAo-2019.pdf Acesso: 03 mar. 2022.
» https://prograd.ufc.br/wp-content/uploads/2018/11/edital-50-bolsa-de-apoio-a-projetos-de-graduaCAo-2019.pdf -
UFC - UNIVERSIDADE FEDERAL DO CEARÁ. Resolução CEPE nº 08, de 26 de abril de 2013. Disponível em: http://www.ufc.br/images/_files/a_universidade/cepe/resolucao_cepe_2013/resolucao08_cepe_2013_anexo22.pdf Acesso: 03 mar. 2022.
» http://www.ufc.br/images/_files/a_universidade/cepe/resolucao_cepe_2013/resolucao08_cepe_2013_anexo22.pdf -
VINHAIS, H. E. F. Estudo sobre o impacto da expansão das universidades federais no Brasil 2013. Tese (Doutorado) - Universidade de São Paulo, São Paulo, 2013. Disponível em: https://doi.org/10.11606/T.12.2013.tde-20012014-152929 Acesso em: 07 out. 2024.
» https://doi.org/10.11606/T.12.2013.tde-20012014-152929 -
WILHELML, M. F.; SCHLOSSERLL, M. T. S. Evasão no curso de licenciatura em geografia da Universidade Estadual do Oeste do Paraná (UNIOESTE): indagações e complexidades. Geografia Ensino & Pesquisa, Santa Maria, v. 23, 2019. Disponível em: https://doi.org/10.5902/2236499437672 Acesso em: 07 out. 2024.
» https://doi.org/10.5902/2236499437672
-
Editor de seção:
André Pires | Editor de Layout: Silmara Pereira da Silva Martins



Source: Prepared by the authors. Note: Courses on the vertical axis, names omitted for better visualization.
Source: Prepared by the author based on data provided by Prograd UFC.