Abstract
Objective: Causal inference from observational data remains a significant challenge for scholars and policymakers, particularly in assessing the impact of public health policies where randomization is often infeasible. The objective of this study was to evaluate the impact of the Yellow September campaign on suicide rates in Brazil using the synthetic control method (SCM).
Methods: A quasi-experimental design was applied using annual suicide rate data from 2000 to 2019. Socioeconomic and demographic variables were obtained from the World Bank and WHO datasets. The SCM was employed to construct a counterfactual scenario simulating suicide rates in the absence of the campaign, using Latin American countries as control units.
Results: The synthetic control model demonstrated a strong alignment between Brazil and its synthetic counterpart in the pre-intervention period. However, post-2015 analysis revealed a 9.2% increase in suicide rates in Brazil compared to the synthetic control. Brazil exhibited a post-/pre-intervention root mean squared prediction error (RMSPE) ratio of 4.18, the highest among all countries in the donor pool. However, placebo tests indicated that this observed increase was not statistically significant, suggesting the difference may reflect random variation or other factors unrelated to the campaign.
Conclusions: The Yellow September campaign had no statistically significant impact on reducing suicide rates in Brazil. While the observed increase in suicide rates may reflect improved case reporting or contextual influences, these findings underscore the complexity of evaluating mental health policies. Complementary strategies and further research are needed to better understand the campaign's effects and address the multifaceted nature of suicide prevention.
Keywords:
Suicide prevention; public health campaigns; health policy evaluation; quasi-experimental studies
Introduction
Awareness campaigns are widely used public health strategies aimed at promoting knowledge, reducing stigma, and encouraging behavioral change regarding complex health issues such as suicide.1 These initiatives frequently employ mass media, community engagement, and educational activities to disseminate information and foster open dialogue.2 Despite their widespread implementation, the evidence regarding their effectiveness in reducing suicide rates remains mixed. Some studies suggest positive outcomes, including increased awareness and help-seeking behaviors,3,4 while others indicate limited or no impact on actual suicide rates.2,5 Furthermore, awareness campaigns may inadvertently lead to unintended consequences, such as sensationalism or an increase in reporting bias.6 This highlights the necessity of rigorous evaluations to determine the real-world impact of such interventions.
In Brazil, the Yellow September (YS) campaign was launched in 2015 as a nationwide suicide prevention initiative. This initiative represents one of the most prominent public health efforts in Latin America aimed at increasing awareness and reducing suicide rates. Organized by the Brazilian Psychiatric Association (ABP) in partnership with the Federal Council of Medicine (CFM), the campaign seeks to raise public awareness, reduce stigma, and promote mental health services.7 Its activities include public lectures, educational workshops, media campaigns, and the symbolic display of yellow ribbons throughout the month of September.7 The campaign leverages mass media outreach, educational workshops, public lectures, and community engagement activities, often symbolized by the display of yellow ribbons during the month of September.8 By increasing the visibility of mental health discussions in public spaces, YS aims to normalize conversations about psychological distress, reduce feelings of isolation among at-risk individuals, and encourage early intervention.9
Despite its wide reach and visibility, there is limited empirical evidence assessing the effectiveness of YS in reducing suicide rates. The existing literature offers divergent findings, with some studies reporting no measurable impact,10–12 and others raising concerns about potential reporting artifacts or contextual confounders that complicate causal interpretation.13
This study addresses this gap by applying the Synthetic Control Method (SCM) – a quasi-experimental approach increasingly used in policy evaluation but still underutilized in public health campaign assessments. SCM allows for the construction of a counterfactual scenario by weighting a combination of control units to approximate the trajectory of the treated unit in the absence of the intervention. This method provides a robust framework for estimating causal effects in situations where randomized controlled trials are infeasible or unethical.
Therefore, the work aims to evaluate the impact of the YS campaign on suicide rates in Brazil using SCM. By employing this methodological approach, we seek to offer new insights into the effectiveness of awareness campaigns and contribute to the broader discussion on public mental health interventions.
Methods
Study design
This study adopts an ecological design with a quasi-experimental analytical approach to assess the impact of the YS campaign on suicide rates in Brazil. The analysis is based on aggregated, population-level data and employs the SCM, which allows for causal inference in observational contexts where randomization is not feasible. Initiated in 2015, YS represents a nationwide intervention aimed at raising awareness about suicide prevention and reducing stigma associated with mental health issues. The analysis focuses on annual suicide rate data spanning from 2000 to 2019, providing a comprehensive pre- and post-intervention time frame for robust comparisons. The paper examines national-level data while leveraging the synthetic control method to construct a counterfactual scenario, simulating suicide rates in the absence of the campaign.
Data source and variables
The data for this study were obtained from the Quality of Government (QoG) Institute database, which compiles internationally comparable indicators from reliable sources.14 The QoG dataset ensures consistency and validity of the data, facilitating robust analysis. The suicide rate data were sourced from the World Health Organization (WHO). Socioeconomic and demographic variables were retrieved from the World Bank (WB) database. These variables include indicators such as population, life expectancy at birth, fertility rate, gross domestic product (GDP) per capita, trade, unemployment, and infant mortality. Table 1 describes the characteristics of the variables.
Synthetic control method
Rationale for method selection
This technique was selected due to its methodological advantages in evaluating the causal effects of interventions in observational settings where randomized controlled trials are impractical or unethical. SCM allows for the construction of a synthetic counterfactual by optimally weighting control units to approximate the pre-intervention characteristics of the treated unit. This approach reduces bias by accounting for both observed and unobserved confounders that are time-invariant.
In public health research, SCM has gained recognition as a robust quasi-experimental technique, particularly for evaluating population-level interventions.15,16 Its ability to handle single-unit interventions, as in the case of Brazil's nationwide YS campaign, makes it particularly suitable for this analysis. Additionally, SCM's transparency in model construction and its visual interpretability enhance the credibility and communicability of the findings.17
Compared to other quasi-experimental methods, SCM offers distinct advantages. Whereas Difference-in-Differences (DiD) relies on the assumption of parallel trends between treated and control units, SCM constructs a synthetic counterfactual that closely matches pre-intervention characteristics and trends, making it suitable when parallel trends cannot be assumed.18 Matching methods, while useful for balancing observed covariates, often fail to account for time-varying unobserved confounders and do not leverage longitudinal outcome data as effectively.18 Interrupted Time Series (ITS), although powerful in modeling pre- and post-intervention trends within a single unit, lacks a robust comparison group, making it vulnerable to confounding from concurrent events.18 In contrast, SCM constructs a weighted combination of control units that best replicates the pre-intervention trajectory of the treated unit, providing a more credible counterfactual when randomized experiments are not feasible.19
Bayesian Structural Time Series (BSTS) models also provide causal inference tools, offering probabilistic estimates and handling complex temporal dynamics.20 However, BSTS requires strong prior assumptions and is sensitive to model specification, especially with limited data.20
Given its ability to construct a transparent and credible counterfactual in the absence of randomized trials, the SCM was deemed the most appropriate approach for this analysis. Its application is particularly relevant for evaluating nationwide interventions like YS, where traditional assumptions of other quasi-experimental methods are difficult to satisfy.
Statistical aspects
The mathematical formalization of the method was originally described by Abadie, Diamond, and Hainmueller.19 The equations consider the territorial units of interest and the pre- and post-intervention time points to define the estimators and differential matrices.
The first step of the model is to define the territorial units of interest. It is agreed that there are J + 1 regions, where the first receives the intervention and the J regions act as potential controls. It is assumed that the treatment region is uninterruptedly exposed to the intervention of interest after some period of initial intervention: .
Given this, this is the expected result for the region i, at the moment t, in the absence of intervention. On the contrary, it would be the expected result for the region i, at the moment t, if the region i is exposed to the intervention between periods .
Therefore, the expected effect of the intervention on the dependent variable in the region i can be defined as: .
Assuming it is calculated using a factorial model, we have: where δt a common factor not observed between the units; Zi a vector (1 × r) of covariates observed among units that are not affected by the intervention; θt a vector (R × 1) of unknown parameters; λt a vector (1 × F) of common unobservable factors; μi a (F × 1) factor vector of unknown regional factors, which corresponds to the weights on the total effect and εit transient observable shocks.
Considering a vector (J × 1) of weights , such that wj ≥ 0 for and . Each W value represents a potential synthetic control, that is, a particular average of the control regions. Therefore, the value of the dependent variable for each synthetic control indexed by W is:
Then, an unbiased estimator of α1t can be approximately defined as:
Whether it is a vector (k × 1) of characteristics of the pre-intervention treatment region, X0 it will be a matrix (k × J) of the same variables applied to the control units:
The vector W* is chosen in such a way that it minimizes the distance of each variable in the post-treatment period from the pre-treatment period. In such a way that, , where V is a symmetric and positive semidefinite matrix.
Statistical analysis
The parameter of interest is the intervention's effect, which is calculated as the average difference in the dependent variable between the treated units and the artificial control group. In turn, constructing the synthetic control group requires the use of data for all analysis units, both control and treatment. At a minimum, information on the dependent variable is needed at least one time point before and after the intervention.15 The greater the longitudinal data availability, the more reliable the counterfactual becomes, thereby enhancing the robustness of the results.
The synthetic control is estimated by weighting the control units in such a way that the distribution of the control group closely approximates the treated unit's distribution in the pre-intervention period.16 Moreover, the synthetic group seeks to minimize the pre-intervention differences in the predictors included between the treatment unit and the weighted average of the control units.16
Once the synthetic control is constructed, it is used to estimate what would have happened to the treatment group in the absence of the intervention.15 The weight of potential control units remains constant over time. The difference between this counterfactual trend and the observed values for the treatment unit represents the average effect of the intervention.15
The root mean squared prediction error (RMSPE) plays a critical role in evaluating the validity of the synthetic control as a counterfactual. A low pre-intervention RMSPE indicates that the synthetic control closely replicates the treated unit's trajectory prior to the intervention, ensuring a reliable counterfactual. The post-intervention RMSPE reflects how much the treated unit diverges from its synthetic control after the intervention. The RMSPE ratio – the post-intervention RMSPE divided by the pre-intervention RMSPE – standardizes this comparison; a high ratio suggests that the divergence is not due to poor pre-treatment fit or random fluctuations but may be attributable to the intervention.
To assess the sensitivity of the results to weight assignments, we conducted placebo tests in which the SCM was applied to each country in the donor pool as if it had received the intervention in 2015. These tests provide a distribution of placebo effects, against which the treatment effect for Brazil can be compared.
Parameters and analysis units
The dependent variable is the suicide rate and the independent variables used as controls include population, life expectancy at birth, fertility rate, urban population, GDP per capita, trade, unemployment, inflation, and infant mortality rate. These variables were selected based on their relevance as determinants of suicide rates and their availability across the analysis period. The study covers the years 2000 to 2019 and the intervention year is 2015.
The donor pool for constructing the synthetic control unit consisted of Latin American countries. Countries were selected based on multiple criteria: (1) geographic proximity, to ensure cultural and regional similarities; (2) comparable socioeconomic profiles, including indicators such as GDP per capita, unemployment rates, fertility rates, and life expectancy; (3) availability of reliable and complete suicide mortality data during the entire study period (2000–2019); and (4) the absence of national suicide prevention campaigns with broad media outreach comparable to YS in the selected donor countries during the period of analysis (2015–2019), to minimize potential confounding.
Ethical considerations
This study uses publicly available and aggregated data. As no individual-level or identifiable data are involved, ethical approval is not required. The research adheres to ethical standards by ensuring transparency and respecting data privacy.
Computational procedures
All statistical analyses were carried out using the R Statistical software version 4.0.5. The synthetic control was implemented using the Synth package.21 Materials for replication are publicly available at: https://osf.io/nvwx5/. By making these materials openly accessible, transparency in the procedures is enhanced, and the potential for reusing the original data and scripts for similar studies is increased.22
Results
The synthetic control analysis demonstrates a strong alignment between the treated unit, Brazil, and its synthetic counterpart prior to the intervention. Table 2 presents a summary of the variable characteristics for Brazil, the synthetic Brazil, and the average of Latin American countries in the donor pool. Key predictors such as population, life expectancy, fertility rate, GDP per capita, trade, unemployment, and infant mortality rates are well-matched between Brazil and its synthetic counterpart.
The weights assigned to the countries in the donor pool reflect the contribution of each unit to the synthetic control, as showed in Table 3. The largest weight is attributed to Mexico (0.72), followed by Colombia (0.22) and Haiti (0.06), with minimal or zero contributions from the remaining countries. This weighting scheme ensures that the synthetic unit closely replicates the pre-intervention characteristics of Brazil.
Post-intervention analysis in Figure 1 reveals a significant divergence between Brazil and its synthetic counterpart. Before 2015, the lines are closely aligned, indicating a good match between Brazil and its synthetic counterpart during the pre-intervention period. However, after 2015, the treated line rises above the synthetic line, suggesting that the actual suicide rate in Brazil increased relative to the counterfactual scenario estimated by the synthetic control. On average, there was a 9.2% increase in the suicide rate following the implementation of the YS campaign.
Table 4 presents the ratios between post- and pre-intervention RMSPE for Brazil and each country in the donor pool. Brazil exhibited a post/pre-intervention RMSPE ratio of 4.18, the highest among all countries analyzed. This result indicates a substantial deviation of Brazil's suicide mortality trajectory from its synthetic control following the implementation of the YS campaign. In contrast, most placebo countries displayed ratios substantially lower, suggesting that the divergence observed in Brazil is not commonly replicated among the control units.
The placebo test results in Figure 2 do not provide sufficient evidence to claim that the observed difference is statistically significant. The treated unit remains close to zero and follows a similar trajectory to the placebo units in this period pre-intervention. This indicates that the synthetic control model effectively fits the pre-intervention trends for Brazil, validating the model's reliability for counterfactual analysis. But the gap in suicide rates for the treated unit post-intervention closely aligns with the range of gaps observed in placebo units, suggesting that the impact of the YS campaign on suicide rates is not statistically distinguishable from random variation.
Gaps between observed and synthetic suicide mortality rates for Brazil and placebo countries.
Discussion
This study evaluated the impact of the YS campaign on suicide mortality rates in Brazil, employing the SCM. Our findings indicate that between 2015 and 2019, Brazil experienced a suicide mortality rate 9.2% higher than its synthetic control counterpart. However, placebo tests indicate that this observed increase is not statistically distinguishable from random variation, warranting cautious interpretation of the findings. Although initially counterintuitive, this result aligns with emerging evidence suggesting that awareness campaigns alone may not suffice to reduce suicide rates12 and may, in some cases, produce unintended effects.10,11,23
One plausible interpretation is that the YS campaign heightened public awareness and reduced stigma surrounding mental health, potentially increasing the reporting and detection of suicide cases without necessarily reducing their incidence. This visibility may have encouraged individuals, families, and healthcare professionals to recognize, report, and document suicide cases that might previously have gone unreported.24,25 Consequently, the observed increase in suicide mortality rates following the campaign could partially reflect improved detection and reporting practices, rather than an actual rise in incidence.
Nonetheless, in contexts where mental health services are limited, fragmented, or inaccessible, raising awareness alone may be insufficient to achieve meaningful reductions in suicide rates.26 Individuals identified as being at risk often face significant barriers to accessing timely and appropriate care, which can undermine the potential benefits of public health campaigns.27 In such settings, increasing visibility without parallel investments in mental health infrastructure may inadvertently leave vulnerable populations without adequate support, potentially exacerbating psychological distress or feelings of hopelessness.28
Alternative explanations for the observed increase in suicide rates must also be considered. Broader societal factors – including economic downturns, growing inequality, and persistent gaps in mental health service provision – may have contributed to rising suicide mortality during the study period, independently of the campaign.29,30 In Brazil, regional disparities in socioeconomic conditions and access to healthcare could have amplified these effects, particularly in vulnerable populations.31
The limitations of this study warrant careful consideration. First, although SCM provides a rigorous quasi-experimental framework, potential endogeneity remains a concern. Unobserved variables that simultaneously affect both the likelihood of campaign implementation and suicide rates may bias the estimates. Second, the findings may not be generalizable beyond the Brazilian context, given the unique cultural, social, and policy environments. Third, the presence of unobserved confounders – such as media coverage unrelated to the campaign or shifts in suicide methods – could have influenced the outcomes in ways not captured by the model. Finally, although the observed differences were not statistically distinguishable from random variation, the results contribute to advancing the scientific evidence on the effectiveness and limitations.32 This transparency is essential to provide a balanced understanding of the effectiveness of suicide prevention campaigns and to inform future public health strategies.
From a policy perspective, these results highlight the necessity of integrating awareness campaigns within comprehensive suicide prevention strategies. Public awareness efforts, such as YS, should be complemented by accessible mental health services, crisis helplines, and targeted interventions for high-risk groups. Additionally, evidence-based strategies – such as early intervention programs, school-based mental health education, and means restriction policies (e.g., limiting access to firearms and toxic substances) – should be prioritized to enhance the effectiveness of suicide prevention initiatives. A multi-layered approach is essential to address the complex determinants of suicide and achieve meaningful population-level outcomes.
Future research should employ mixed-methods approaches, combining quantitative evaluations like SCM with qualitative studies that explore individual experiences and perceptions of awareness campaigns. Moreover, longitudinal analyses examining regional variations and demographic-specific effects could offer deeper insights into the campaign's impact over time.
In conclusion, while the YS campaign represents an important initiative in promoting mental health awareness in Brazil, its isolated implementation appears insufficient to reduce suicide mortality rates at the population level. To achieve meaningful outcomes, public health strategies must adopt an integrated, multi-faceted approach grounded in evidence-based practices.
Acknowledgements
This work was supported by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES; grant no. 88887.966558/2024-00).
Data availability statement
The data that support this study are available in OSF at https://osf.io/nvwx5/.
References
- 1 Waters E, Doyle J, Jackson N, Howes F, Brunton G, Oakley A. Evaluating the effectiveness of public health interventions: the role and activities of the Cochrane Collaboration. J Epidemiol Community Health. 2006;60:285-9.
- 2 Tam MT, Wu JM, Zhang CC, Pawliuk C, Robillard JM. A Systematic review of the impacts of media mental health awareness campaigns on young people. Health Promot Pract. 2024;25:907-20.
- 3 Donovan RJ, Drane CF, Santini ZI, Jalleh G. Impact on help-seeking behaviours of a campaign perceived to decrease stigma and increase openness around mental health. Health Promot J Austr. 2024;35:1378-85.
- 4 Duthie G, Reavley N, Wright J, Morgan A. The impact of media-based mental health campaigns on male help-seeking: a systematic review. Health Promot Int. 2024;39:daae104.
- 5 Mohn C, Haga E, Nilsson HSW, Pirkis J, Mehlum L. Change in attitudes after a suicide prevention media campaign in the Mid-Norway region. BMC Psychiatry. 2024;24:444.
- 6 Organización Panamericana de la Salud (OPAS). Prevención del suicidio: un recurso para profesionales de los medios de comunicación. Actualización del 2023. Washington, D.C.: OPAS; 2023.
-
7 Associação Brasileira de Psiquiatria (ABP). Setembro amarelo - mês de prevenção ao suicídio [Internet]. 2024 [cited 2024 Oct 29]. www.setembroamarelo.com
» www.setembroamarelo.com - 8 Associação Brasileira de Psiquiatria (ABP). Setembro amarelo: uma campanha da ABP contra o suicídio! [Internet]. 2015 Sep 7 [cited 2025 Mar 17]. revistardp.org.br/abp/article/view/938
- 9 Ruckert MLT, Frizzo RP, Rigoli MM. Suicídio: a importância de novos estudos de posvenção no Brasil. Rev Bras Ter Cogn. 2019;15:85-91.
- 10 Cruz WGN, Jesuino TA, Moreno HF, Santos LG, Galvão-de Almeida A. Analysis of the impact of the Brazilian suicide prevention campaign "Yellow September": an ecological study. Trends Psychiatry Psychother. 2024;46:e20220564.
- 11 Damiano RF, Beiram L, Damiano BBF, Hoffmann MS, Moreira-Almeida A, Rück C, et al. Associations between a Brazilian suicide awareness campaign and suicide trends from 2000 to 2019: joinpoint and regression discontinuity analysis. J Affect Disord. 2024;365:459-65.
- 12 Corrêa Matias Pereira C, Najafi Moghaddam Gilani V, Nazif-Munoz JI. A brief research report of suicide rates in the Brazilian elderly over a 12-year period: the lack of association of the "Setembro Amarelo" campaign for suicide prevention. Front Psychiatry. 2024;15:1354030.
- 13 Souza FG de ME, de Souza Júnior SA, Ribeiro TT, de Araújo MB, Bisol LW, da Silva AG. Myths and facts about Yellow September and suicide prevention: a critical analysis of article by. J Affect Disord. 2024;372:182-3.
-
14 Dahlberg S, Sundström A, Holmberg S, Rothstein B, Alvarado Pachon N, Dalli CM, et al. The quality of government basic dataset, version Jan24 [Internet]. 2024 [cited 2024 Dec 10]. www.gu.se/en/quality-government/qog-data/data-downloads/basic-dataset
» www.gu.se/en/quality-government/qog-data/data-downloads/basic-dataset - 15 Bouttell J, Craig P, Lewsey J, Robinson M, Popham F. Synthetic control methodology as a tool for evaluating population-level health interventions. J Epidemiol Community Health. 2018;72:673-8.
- 16 Bonander C, Humphreys D, Degli Esposti M. Synthetic control methods for the evaluation of single-unit interventions in epidemiology: a tutorial. Am J Epidemiol. 2021;190:2700-11.
- 17 Abadie A. Using Synthetic controls: feasibility, data requirements, and methodological aspects. J Econ Lit. 2021;59:391-425.
-
18 Doudchenko N, Imbens GW. Balancing, regression, difference-in-differences and synthetic control methods: a synthesis [Internet]. 2016 [cited 2025 Mar 17]. www.nber.org/papers/w22791
» www.nber.org/papers/w22791 - 19 Abadie A, Diamond A, Hainmueller J. Synthetic control methods for comparative case studies: estimating the effect of California's tobacco control program. J Am Stat Assoc. 2010;105:493-505.
- 20 Brodersen KH, Gallusser F, Koehler J, Remy N, Scott SL. Inferring causal impact using Bayesian structural time-series models. Ann Appl Stat. 2015;9:247-74.
- 21 Abadie A, Diamond A, Hainmueller J. Synth: an R package for synthetic control methods in comparative case studies. J Stat Softw. 2011;42:1-17.
- 22 Figueiredo Filho D, Lins R, Domingos A, Janz N, Silva L. Seven reasons why: a user's guide to transparency and reproducibility. Bras Political Sci Rev. 2019;13:e0001.
- 23 Damiano RF, Beiram L, Damiano BBF, Miguel EC, Menezes PR, Salum GA. Letter in response to Matos e Souza et al. J Affect Disord. 2025;374:235-7.
- 24 Dumesnil H, Verger P. Public awareness campaigns about depression and suicide: a review. Psychiatr Serv. 2009;60:1203-13.
- 25 World Health Organization (WHO). Preventing suicide: a global imperative [Internet]. Geneva: WHO; 2014.
- 26 Campion J. Public mental health: key challenges and opportunities. BJPsych Int. 2018;15:51-4.
- 27 Das A, Sarkhel S, Brahma A, Akula V. Development of mental health human resources for mental well-being. Indian J Psychiatry. 2024;66:S391-402.
- 28 Wainberg ML, Scorza P, Shultz JM, Helpman L, Mootz JJ, Johnson KA, et al. Challenges and opportunities in global mental health: a research-to-practice perspective. Curr Psychiatry Rep. 2017;19:28.
- 29 Stack S. Contributing factors to suicide: Political, social, cultural and economic. Prev Med. 2021;152:106498.
- 30 Pirkis J, Bantjes J, Dandona R, Knipe D, Pitman A, Robinson J, et al. Addressing key risk factors for suicide at a societal level. Lancet Public Health. 2024;9:e816-24.
- 31 Araújo TM de, Torrenté M de ON de. Mental Health in Brazil: challenges for building care policies and monitoring determinants. Epidemiol Serv Saude. 2023;32:e2023098.
- 32 Gerber AS, Green DP, Nickerson D. Testing for publication bias in political science. Polit Anal. 2001;9:385-92.




