Open-access Time trend of Vigitel Brasil operation indicators (2006 to 2023)

ABSTRACT

OBJECTIVE  To analyze the time trend of indicators from the Vigitel data collection operation between 2006 and 2023.

METHODS  A methodological study that analyzed the temporal trend of three Vigitel operation indicators: eligibility rate, response rate and refusal rate, as well as the number of contacts answered according to days of the week and times of day. A series of linear regressions was used to identify temporal variations for the entire period (2006 to 2023) and for the final five years of this period (2018 to 2023).

RESULTS  During the 17 editions analyzed, there was a decrease in the eligibility rate (-3.26 pp/year; p < 0.001) with the greatest magnitude in the most recent period (-12.23 pp/year; p = 0.010). Although the inclusion of mobile phones in the register in 2023 has mitigated the drop in eligibility, especially in recent years, the reduction has remained high (-10.43 pp/year; p = 0.020). The response rate also fell over the entire period (-0.78; p < 0.001) and, together with the fall in eligibility, represents an obstacle to the sustainability of the survey. On the other hand, the refusal rate also fell (-0.23 pp/year; p = 0.030), indicating the consistent quality of the system’s operation. The number of contacts answered was concentrated on weekdays and between 9am and 8pm.

CONCLUSIONS  An analysis of the data from the Vigitel survey suggests that the methodology used has become fatigued, with reductions in eligibility and response rates over the years. Only the refusal rate, despite being far from its lowest levels, remains at satisfactory levels. The search for alternatives that make participation more convenient for the respondent is an important path for surveys of this nature, with a view to maintaining the quality of health surveillance in the country.

Methodological study; Health Survey; Public Health Surveillance; Risk Factors; Brazil

RESUMO

OBJETIVO  Analisar a tendência temporal de indicadores da operação de coleta dos dados do Vigitel entre 2006 e 2023.

MÉTODOS  Estudo metodológico que analisou a tendência temporal de três indicadores da operação do Vigitel: taxa de elegibilidade, taxa de resposta e taxa de recusa, além do número de contatos atendidos segundo dias da semana e horários. Uma série de regressões lineares foi empregada para identificar variações temporais para o período completo (2006 a 2023) e para o quinquênio final desse período (2018 a 2023).

RESULTADOS  Durante as 17 edições analisadas, observou-se uma diminuição na taxa de elegibilidade (-3,26 pp/ano; p < 0,001) com maior magnitude no período mais recente (-12,23 pp/ano; p = 0,010). Embora a inclusão de telefones móveis no cadastro em 2023 tenha atenuado a queda da elegibilidade, sobretudo nos últimos anos, a redução permaneceu elevada (-10,43 pp/ano; p = 0,020). A taxa de resposta também diminuiu no período completo (-0,78; p < 0,001) e, conjuntamente com a queda da elegibilidade, representa um obstáculo para a sustentabilidade do inquérito. Em contrapartida, a taxa de recusa também diminuiu (-0,23 pp/ano; p = 0,030), indicando a consistente qualidade na operação do sistema. O número de contatos atendidos se concentrou em dias da semana e horário entre 9h e 20h.

CONCLUSÕES  A análise dos dados da operação do inquérito Vigitel sugere uma fadiga da metodologia empregada, com reduções na elegibilidade e na taxa de resposta ao longo dos anos. Apenas a taxa de recusa que, a despeito de estar longe de seus menores níveis, permanece em patamares satisfatórios. A busca de alternativas que tornem a participação mais conveniente para o respondente é um caminho importante para inquéritos dessa natureza, visando a manutenção da qualidade da vigilância em saúde no País.

Estudo Metodológico; Inquérito de Saúde; Vigilância em Saúde Pública; Fatores de Risco; Brasil

INTRODUCTION

Improving global health systems requires continuous evaluation of solid, quality data, obtained through information systems and health surveys that are updated to accurately reflect the health conditions of the population1. In recent decades, through the efforts of international organizations, governments and research institutions, the use of health surveys in low- and middle-income countries has been expanded1. In Brazil, especially since the 1980s, primary data has been collected through population surveys, which are recognized as fundamental components of a national health information system3.

Among Brazilian health surveys, the Surveillance System for Risk and Protective Factors for Chronic Diseases by Telephone Survey (Vigitel) stands out for having been carried out since 2006, covering a representative sample of the adult population (≥ 18 years) living in all the capitals of the 26 Brazilian states and the Federal District6. The conception of Vigitel in the early 2000s was driven by the epidemiological panorama of the time, when analyses of historical mortality series in the capitals of the Brazilian states indicated a more than threefold increase in the proportion of deaths from chronic non-communicable diseases (CNCDs) between 1930 and 20067.

Since then, in 17 editions, Vigitel has been the survey responsible for continuously monitoring the temporal trend in the prevalence of CNCDs and their main risk factors in Brazilian capitals. The data collected annually remains a solid indicator of the evolution of health indicators, essential for monitoring actions to prevent and control CNCDs, such as the Strategic Action Plan for Tackling Chronic Non-Communicable Diseases in Brazil, 2011 to 20228, and its most current version, 2021 and 20309.

To provide an accurate picture of the health situation in Brazil, it is crucial that Vigitel be methodologically revised based on the observation of a possible drop in the quality of the information generated. It is worth noting that, since its conception, Vigitel’s sampling and data collection methodology has remained very similar to that adopted in its first edition, indicating its robustness over time. From 2006 to 2021, interviews were conducted exclusively by telephone10. However, the progressive reduction in landline telephone coverage in the country, especially in the capitals of the North and Northeast regions, pointed to the imminent need to incorporate a new data collection method into Vigitel11.

Data from household surveys in Brazil already pointed to this deterioration in landline coverage. In 2019, only 30.4% of households in Brazilian state capitals had a landline telephone, a significant reduction compared to 2013 (48.9%)12. These results also showed a disparity in coverage between the capitals, with only 9.2% in Macapá and 53.2% in Rio de Janeiro12. A previous study reported a decrease in the eligibility rate of landlines in the country13, making it unfeasible to carry out the survey exclusively using this mode. In response to this situation, in 2023, the Ministry of Health (MS) incorporated mobile phone interviews into the survey. However, to date, no studies have been carried out to provide a comprehensive analysis of Vigitel’s operation.

The aim of this study was therefore to analyze the time trend of the main indicators of the Vigitel data collection operation between 2006 and 2023.

METHODS

This is a methodological study based on information recorded during the operation of Vigitel between 2006 and 2023. All the information analyzed was provided by the company contracted to collect the data and sent weekly to the MS to monitor the survey’s progress over the years.

Background to Vigitel’s Sampling and Data Collection

Since 2006, Vigitel has interviewed approximately 54,000 individuals aged 18 or over every year, distributed in around 2,000 interviews in each of the capitals of the 26 Brazilian states and the Federal District. However, as of 2020, due to the COVID-19 pandemic, an emergency measure was implemented, reducing the sample to 1,000 individuals in each location10,14. In 2023, a further reduction was necessary due to administrative and operational issues, with a sample of around 800 individuals per capital6.

The survey sampling process is carried out in two stages. In the first, a predetermined number of 10,000 telephone lines are drawn in each city. Next, the selected lines in each city are resampled into replicas of 200 lines. With this, each replica should represent a similar proportion of lines by city region or telephone prefix to the total sample. Results of pilot studies carried out previously with mobile phone listings obtained by RDD (random digit dialing) indicated the low quality of mobile phone registrations. We therefore opted for replicates of 500 lines for this modal.

The second stage begins by identifying the lines drawn that are eligible for the system, i.e. active residential lines (methodology applied between 2006 and 2021). Lines assigned to companies, not in use or out of service were not considered eligible for the system. In addition, lines that do not answer six calls (between 2006 and 2008 up to 10 calls were made), which are made on different days and times, including Saturdays and Sundays and at night, and which probably correspond to unoccupied households, were also considered ineligible.

For each eligible landline, once the respondent has agreed to take part in the survey, the adult residents of the household are enumerated and then a random draw is made to select one of these adults for the interview. If the person drawn is present and available, the interview is carried out at the initial contact. Otherwise, the interview could be scheduled for a more convenient day and time. Refusal to take part in the study at each of these two moments is recorded and added up to make up the indicator of refusal to take part in the study.

A similar methodology was implemented for the operation with mobile phones (adopted from 2023), without the need for an intra-household lottery after confirming eligibility. This procedure is necessary in the case of landlines, since the line is owned by the household, but it is not necessary in the case of mobile phones, since ownership tends to be individual, and use is not shared between family members15.

Quality Indicators for the Register and System Performance

Throughout its editions, Vigitel has monitored the following system quality indicators: 1) Eligibility rate: percentage of lines considered eligible for Vigitel among the complete set of lines drawn; 2) Response rate: percentage of interviews actually carried out among the drawn lines considered eligible; and 3) Refusal rate: percentage of eligible lines that refused to participate in the study, including both refusals to interview and refusals to schedule.

Since different methodologies for calculating these rates in telephone surveys are available, Vigitel has opted to use the standard American Association for Public Research (AAPOR) #2 methodology since 2006. According to this methodology, the success and refusal rates are estimated considering only the lines identified as eligible, and lines with undefined status are assumed to be ineligible16.

In addition, information on the most effective days and times in the 2023 operation is included. A total of 60 interviewers took part in the operation, 40 from 8.45am to 3.15pm and 20 from 3.15pm to 9.30pm. At weekends, a single shift of interviewers worked from 9:20 to 15:40.

Data Organization and Analysis

All the data were organized and analyzed using Stata software® version 16.017.

Initially, data relating to the system’s three quality indicators, as well as the number of contacts answered according to days of the week and times of day, presented in the previous section, were estimated based on the operation’s microdata provided by the company responsible for collecting the data. This operation was carried out for each of the cities, for all the cities in the same geographical region (North, Northeast, Central-West, South, and Southeast) and for the entire set of cities studied by Vigitel.

Subsequently, a series of linear regressions were used to identify significant variations over the period studied. In each regression model, the Vigitel quality indicator in question was considered the outcome variable, while the year the survey was carried out was treated as the exposure variable. In this type of analysis, the angular coefficient of the regression model (Beta value) indicates the average variation of the indicator in percentage points per year (pp/year) and the P value identifies whether this variation was statistically significant (p < 0.05). This analysis of variation was carried out for the entire Vigitel period (from 2006 to 2023) and for the final five years of this period (from 2018 to 2023). In addition, an analysis of variation was carried out for a section of the complete period, excluding the years 2006, 2007, and 2008 (supplementary materiala), due to the changes that occurred in the operation of the survey in its initial years, which could bias the comparison in the complete set of the period. It should be noted that there is no evidence of error in the data for this period, however, until 2009 the rates were calculated manually, and after that were calculated automatically.

aSupplementary material available from: https://www.scidb.cn/en/detail?dataSetId=2282546d1f0e43feb9b79fa6bfc6cd8c

Given the inclusion of mobile phones in the survey in 2023, three additional analyses were carried out. The first considered only the landline telephones in Vigitel 2023, the second only the mobile telephones and the third combined landline and mobile telephones in a summary table.

Finally, the data on the days and times when the most interviews took place was organized in the form of tables containing heat maps. Once again, the presentations were divided between landline and mobile phone charts. These figures were tabulated in raw form, without considering the variation in the number of interviewers on the different days and times, as the number and distribution of interviewers was not constant during the operation. The Vigitel 2023 operation began with six interviewers, gradually increasing to 60 interviewers during the first 45 days of collection. The initial expectation was to have shifts of a similar size in terms of the number of interviewers. However, due to the low conversion of landline interviews, the possibility of reducing the number of workers on the second shift was indicated, making it possible to reduce the costs of the operation. Therefore, in the respective section, number and distribution of interviewers for the period February to April 2023 is also presented.

RESULTS

Eligibility rate

During the entire period studied, the eligibility rate of landlines drawn for inclusion in Vigitel varied between 72.5% and 5.3%. When considering only the 2023 operation, the mobile phone register had an eligibility rate almost three times higher than that of the landline register (21.2% versus 5.3%). The composition of an estimate involving both registries resulted in an eligibility rate of 11.3%, mitigating the sharp drop seen in recent years (Table 1).

Table 1
Percentage (%) of eligible telephone lines in Brazilian state capitals and the Federal District identified from the total number of lines drawn for system operation, by year. Vigitel Brasil, 2006 to 2023.

Considering only the landline registers for the 2023 operation, there was an average reduction in the eligibility rate of 3.26 pp/year (p < 0.001) in the period between 2006 and 2023. However, the reduction observed in the most recent period (2018 to 2023) was more than three times higher (12.23 pp/year; p = 0.010). When both registrations are considered, the reduction in the most recent period was 10.43 pp/year (p = 0.020). Although lower than in the case of landlines, this is still a significantly high figure (Table 1).

Considering landline registers for the whole period, reductions were seen mainly in the capitals of the Northeast (-3.58 pp/year; p < 0.001) and North (-3.52 pp/year; p < 0.001) regions of the country. In the most recent period, a greater magnitude of reduction was observed in the capitals of the Southeast region (-14.64 pp/year; p = 0.010) (Table 1).

Response Rate

Over the whole period, the response rate varied from 71.5% in 2006 to 58.7% in 2023, when only landline registers were considered (-0.78pp/year; p < 0.001). For the year 2023, the mobile phone registry had a response rate around 60% lower than the landline registry (24.1% versus 58.7%) (Table 2).

Table 2
Percentage (%) of response among eligible telephone lines drawn in Brazilian state capitals and the Federal District, by year. Vigitel Brasil, 2006 to 2023.

Regarding landline registers, the greatest reduction in the response rate was observed in the capitals of the northern region of the country (-1.05 pp/year; p < 0.001), with a stable trend in all regions in the most recent period (Table 2).

Refusal rate

Over the entire period analyzed, the refusal rate evolved favorably, with values ranging from 9.1% in 2006 to 5.3% in 2023, when considering only landlines. Similar refusal rates were also observed for mobile phones (4.1%). In both scenarios, the average reduction in the refusal rate over the period was around 0.23 pp/year (p = 0.030) (Table 3).

Table 3
Percentage (%) of refusals* among eligible telephone lines drawn in Brazilian state capitals and the Federal District, by year. Vigitel Brasil, 2006 to 2023.

Days and Times when Contacts are Most Frequent

In general, in 2023, Vigitel interviews were more concentrated on weekdays than on weekends, for both landlines and mobile phones (Figure 1). Considering the time of the calls, the highest number of contacts answered was between 9am and 8pm for both landlines and mobile phones (Figure 2). Data on the number of contacts answered is also described by day of the week (Figure 1) and time of day (Figure 2).

Figure 1
Number of contacts answered in Brazilian state capitals and the Federal District, by day of the week for landline and mobile phone registration. Vigitel Brasil, 2023.

Figure 2
Number of contacts answered in Brazilian state capitals and the Federal District, by time of day for landline and mobile phone registration. Vigitel Brasil, 2023.

DISCUSSION

This study analyzed the trend of Vigitel’s operational indicators over its 17 editions, revealing a challenging scenario. The eligibility rate fell over the entire period, and more sharply in recent years, when the reduction was almost four times that of the total period. Although the inclusion of mobile phones in the register in 2023 eased the drop in eligibility, the reduction still remained high. The response rate also fell over the whole period and, together with the fall in eligibility, represents an obstacle to the sustainability of the survey. On the other hand, the refusal rate fell over the whole period, showing the consistent quality of the system’s operation. In this context, our study provides an up-to-date and comprehensive analysis of important methodological issues for improving the quality and continuity of Vigitel.

Eligibility rates have fallen sharply since the start of the system, with values between 60% and 70% representing the trend between 2007 and 2017, followed by a sharp decline. The occurrence of ineligible lines, especially due to registration lags, should be accepted as a natural phenomenon, since the lines to be called throughout the operation are drawn at a single point in time. However, the intense reduction in the eligibility rate, from 72.5% to 5.3% in 2023 for landlines, exposes an important weakness of telephone surveys with a single data collection frame.

In Vigitel 2023, 61.3% of landlines and 54.5% of ineligible mobile lines corresponded to lines for which contact could not be made after six or more contact attempts on different days and at different times (data not shown). It should be noted that the methodology used to calculate the Vigitel rates (AAPOR #2) treats as ineligible those lines for which it was not possible to definitively identify their eligibility16. This increase in the number of ineligible lines coupled with low coverage can reduce the quality of the data collected, especially in regions with reduced coverage, such as the Northeast and North. These problems also have an impact on the cost of the survey, increasing the number of calls and screening attempts18,19.

It should be noted that even with the large drops seen in recent years, Vigitel has eligibility rates higher than those of the Behavioral Risk Factor Surveillance System (BRFSS), conducted by the Centers for Disease Control and Prevention (CDC) in the United States20. It should be noted, however, that the calculation of the eligibility rates applied in these surveys differs from one another. In Vigitel (AAPOR #2), individuals who answered the survey or not make up the denominator of the calculation. In the BRFSS (AAPOR #4), only those who did not complete the survey are included in the eligibility calculation, excluding those who did16. These methodological differences limit the comparability of these surveys.

Some factors are related to these results, such as the rapid increase in the use of cell phones and the Internet by the population18, also suggesting that telephone interviews (by voice) should coexist with Internet interviews (by data)21. Thus, some surveys are migrating to cell phones, the Internet, or adopting mixed modalities to compensate for the lack of landline coverage and non-response19,22. It is worth noting, however, that around 40% of ineligible lines fall into the categories of business or commercial line, out of service, non-existent, and empty holiday or vacation home, which would be more related to weaknesses in the quality of the register than to social movements beyond the control of the survey.

It is widely recognized that response rates in telephone surveys are decreasing over time, as “non-response” increases18. In this study, we found lower response rates for mobile phones compared to landlines. Vigitel 2023 was the first edition to include mobile phones in its register. Since 1984, the BRFSS has been considered the largest telephone survey system in the world. Since 2011, the BRFSS has adopted the use of cell phone interviews, complementing landline interviews, which has improved the validity, quality and representativeness of the survey. The survey scenario is even less favorable than Vigitel’s, since in 2022 the combined response rate (mobile + landline) was 45.1%23. Since mobile phones are equipped with caller ID and electronic processes to block unwanted calls, only interviews concluded on the first contact are usually successful, with a tiny proportion of appointments being converted into success19.

For 2023, the scenario identified suggests a reduction in the quality of the operation, possibly due to the short time available for implementation, the remote format of the collection, the gradual incorporation of new interviewers, the poor quality of the register used and the short period for collecting a significant volume of interviews (data collection took place over around four months - between December 2022 and April 2023). While the 2020 operations had around 40 interviewers, the 2023 operation required around 60.

Although telephone contact has become more difficult, considering the eligibility and success rates, Vigitel’s refusal rate is possibly the indicator that best expresses the quality of the system’s operation, and this indicator has evolved favorably, decreasing over the years. It is worth noting that in 13 of the 17 years under study, the refusal rate was below 5%, a very significant figure for surveys of this nature. The figures presented for the refusal rate refer both to refusal to schedule (obtained during the initial contact with the household, which does not even allow a resident to be drawn) and to refusal to take part in the interview (when a resident of the household has already been drawn and refuses to take part in the study). Contrary to these results, studies have shown an increase in refusal rates in telephone surveys, which is strongly associated with the initial screening of the interviewer18,19. Thus, the decrease in the refusal rate reveals the interviewer’s priority in obtaining the interviewee’s cooperation, which may be related to the experience acquired over more than a decade of interviewing, especially through interviewer training and supervision13.

An analysis of the Vigitel indicators allows us to conclude that the methodology used has become fatigued, with a reduction in eligibility and response rates. This is a common occurrence in population surveys, which must adapt periodically to social and technological changes21. At the moment, only the refusal rate has been saved, which, despite being far from its lowest levels, remains at satisfactory levels. It is noticeable that telephone surveys as the only method of data collection have become less convenient as exchanges of information via data (such as short message services - SMS and automatic messaging platforms) become more frequent21. The search for alternatives that make participation more convenient for the respondent is undoubtedly a constant path for surveys of this nature.

Based on these results, it is not possible to infer the magnitude of influence of each of these factors on the overall quality of the survey, and this conclusion is dependent on the occurrence of a new Vigitel operation under ideal planning and execution conditions, as happened in previous years13. It is also believed that the survey will benefit from the inclusion of a new means of collection, so the continuity of studies to keep up with the current technological dynamism will be important for maintaining the survey.

It is important to highlight the consolidation of VIGITEL as a surveillance system for risk and protective factors for CNCDs in Brazil, setting an example of a data collection initiative with good quality, low cost, and great agility, a pioneer in middle- and low-income countries. It should be emphasized that the sustainability of the data collection structure is a fundamental step in taking the system back in the right direction, as well as the continuity of studies capable of pointing out and incorporating improvements to its methodology.

REFERENCES

  • Funding:
    Brazilian Ministry of Health (MS - TED 37/2022).

Publication Dates

  • Publication in this collection
    30 June 2025
  • Date of issue
    2025

History

  • Received
    03 June 2024
  • Accepted
    12 Dec 2024
location_on
Faculdade de Saúde Pública da Universidade de São Paulo Avenida Dr. Arnaldo, 715, 01246-904 São Paulo SP Brazil, Tel./Fax: +55 11 3061-7985 - São Paulo - SP - Brazil
E-mail: revsp@usp.br
rss_feed Acompanhe os números deste periódico no seu leitor de RSS
Ir para o topo Reportar erro