The critical use of statistical inference in occupational epidemiology: essay

Abstract

Introduction  Statistical inference (SI) methods are widely used in epidemiological data analyses to evaluate random variation as a probable source of error in the results. However, the assumptions necessary for its use are frequently unmet.

Objective  To discuss assumptions for SI, implications of its use when individuals are non-randomly selected, different statistical approaches of epidemiological data and results extrapolation sceneries in occupational epidemiology.

Methods  Essay based in the specialized literature and in the authors’ academic experience.

Results  This essay presents situations of epidemiological research in which the SI procedures are used without the support of the assumptions for its appliance. Situations of generalization of epidemiological results are discussed, pinpointing when the validity of using SI can be questioned. The considerations about themes addressed in this essay aim to contribute to the critical management of data analysis and to the process of results extrapolation in occupational epidemiology research, delimiting the role of SI.

Conclusion  The use of statistical methods and of the report of research results should include essential provisos that allow the reader to evaluate its pertinence, possible assumptions violation, and the validity of the reasoning and of the inference used.

Epidemiology and Biostatistics; Sampling Studies; Confidence Intervals; Statistical Analysis; Data Interpretation, Statistical; Occupational Health

Resumo

Introdução  Métodos de inferência estatística (IE) são amplamente utilizados na análise de dados epidemiológicos para avaliação da variação amostral como provável fonte de erro dos resultados. Entretanto, pressupostos necessários para seu uso muitas vezes não são verificados.

Objetivo  Discutir pressupostos da IE, implicações do seu uso quando os indivíduos são selecionados de forma não aleatória, diferentes abordagens estatísticas de dados epidemiológicos e cenários de extrapolação de resultados em epidemiologia ocupacional.

Métodos  Ensaio com base em literatura especializada e na experiência acadêmica dos autores.

Resultados  São apresentadas situações da pesquisa epidemiológica nas quais os procedimentos de IE são utilizados sem apoio nos pressupostos para sua aplicação. São discutidas as situações de generalização de resultados epidemiológicos, assinalando-se quando a validade do uso da inferência estatística pode ser questionada. Ponderações sobre temas abordados neste artigo visam contribuir para a condução crítica da análise de dados e no processo de extrapolação dos resultados da pesquisa em epidemiologia ocupacional, delimitando o papel da IE.

Conclusão  O uso de métodos estatísticos e o relato de resultados de pesquisa devem ser feitos com as ressalvas indispensáveis que possibilitem ao leitor avaliar sua pertinência, possíveis violações de pressupostos e a validade do raciocínio e da inferência aplicadas.

Epidemiologia e Bioestatística; Amostragem; Intervalos de Confiança; Análise Estatística; Interpretação Estatística de Dados; Saúde do Trabalhador

Introduction

In epidemiological studies, the use of convenience samples or censuses is common, whether they are cross-sectional studies, cohort studies, or case-control studies. Although these are studies in which the selection of participants was not based on a probabilistic strategy, there are plenty of published studies that present the results of statistical tests and/or confidence intervals (CI) using a non-probabilistic sample. Considering the foundations of statistical inference (SI) as the set of assumptions necessary for its application and purpose, generalizing conclusions based on statistical tests and CIs implies compliance with these assumptions.

In this perspective, to cite just one of the renowned statisticians with an undeniable leading role in epidemiology, Greenland, in a 1990 publication in the journal Epidemiology, entitled Randomization, Statistics and Causal Inference¹, states that

In most epidemiologic studies, randomization and random sampling play little or no role in the assembly of study cohorts. I therefore conclude that probabilistic interpretations of conventional statistics are rarely justified, and that such interpretations may encourage misinterpretation of nonrandomized studies (p. 421).

Greenland and Poole² emphasize that “distortions that arise solely from the play of chance are traditionally called ‘random errors.’ It is these distortions, and only these distortions, that P-values and confidence intervals address.” (p. 115). These articles were preceded and followed by other contributions³-⁵, particularly the special edition of The American Statistician in 2019, with 43 articles. In the editorial of this special issue, entitled Moving to a World Beyond “p<0.05”⁶, the authors make it clear that CI results and p-values should be considered beyond binary “statistical significance”, expressed as a significant result or not. The validity of conclusions in epidemiological research depends on aspects other than SI methods. Valid scientific inferences involve considering different factors, including study design, the process of recruiting and selecting participants, the quality of the measurements, external evidence for the phenomenon under study, and the validity of the assumptions underpinning the analysis of the data⁵,⁷.

The sources of uncertainty in epidemiological studies, systematic errors (biases), and random errors require researchers to constantly seek to overcome them. The critical use of SI only concerns the procedures adopted to deal with random errors. However, although the assumptions of SI are well-defined⁸, misinterpretations of the results of hypothesis tests, using p-values and CIs, have long been mentioned⁹-¹¹. Some authors attribute the difficulty of using statistical methods to the complexity of the definitions of p-value, statistical significance, and CI². There seems to be a lack of knowledge, lack of understanding, or misunderstanding of the purpose and assumptions of SI.

This essay aims to: critically discuss the use of SI, especially in occupational epidemiology studies; evaluate the appropriateness of using statistics to approach epidemiological data, delimiting its different divisions - descriptive, analytical, and inferential; and present and discuss situations of generalization of epidemiological results.

Stages of essay construction

A critical evaluation of the use of SI procedures is the axis that structures the text. A few authors from the fields of statistics and epidemiology, especially in recent decades, have contributed their thoughts on the use of SI in epidemiological studies. Initially, the essay presents a section on the principles and purposes of SI. It then critically discusses situations commonly found in the analysis of epidemiological data, incorporated into the practice of scientific dissemination, peer review, and journals in the field, which are rarely questioned.

Once this set of SI situations has been presented, the essay develops an argument about the statistical approach to epidemiological data, with a view to delimiting its different divisions. It then characterizes the statistical approaches to epidemiological data as descriptive, analytical, and inferential, with the aim of contributing to the deconstruction of the conception that admits there are descriptive studies and analytical studies, but these would be analytical if, when presenting the measures of association resulting from the comparison of groups (for example, exposed and unexposed), they necessarily made use of SI. According to this conception, studies that do not use the generalization of data based on SI are not analytical and have compromised external validity. Finally, the essay presents different situations of generalization of epidemiological data, delimiting the role of SI.

SI principles and aims in Epidemiology

In epidemiology, studies of risk or exposure factors, or of the determinants of health conditions, or of the effects of interventions, require the adoption of methods that make it possible to generalize, with a reasonable margin of safety in the face of the limitations and uncertainties inherent in a population-based approach to health problems.

The set of methods available to support the argument in favor of a study hypothesis includes SI, i.e. inferential statistical procedures constitute an auxiliary axis to support the study hypothesis. The statistical argument, put in this way, would not be the most important or central in epidemiological reasoning and in the discussion of the results of epidemiological studies. Especially important for epidemiological studies is the magnitude of the numerical difference in the frequency measures of the comparison groups, or the magnitude of the ratio of the frequency measures, the directionality of the differences, or the ratio in the direction of the study hypothesis. However, once SI is applied, its assumptions must always be considered.

In any epidemiological study, it is essential for researchers to delimit the population of interest (target population), describe the actual population studied and how the individuals were selected to take part in the study. One of the reasons for this is to provide the scientific community with enough information to judge the merits of the study and the applicability of the results. Regarding the selection of individuals, the epidemiologist often does not have the necessary resources to study all the individuals in the population to which they wish to generalize their results. In these situations, they are led to study a part (sample) of this population.

In this essay, it is assumed that the population is a set of individuals who have at least one characteristic in common, and from whom we are interested in analyzing the behavior of certain other characteristics that vary between the subjects that make up the population. The target population is the population of interest, which may coincide with the population under investigation if all its components are studied. If this is not possible, the target population will still be the population of interest, but the population being investigated will consist of a sample drawn from it. A sample can be either probabilistic or non-probabilistic depending on the method for selecting individuals from the population.

Using probability theory and SI methods, the researcher will draw conclusions about the parameters of interest in the target population, based on the results obtained from the sample. To do this, they will adopt the most appropriate procedures so that all the characteristics of interest of the individuals who make up the target population are represented in the sample. Thus, they will have to apply probability sampling procedures, by which they will try to ensure that all the individuals in the population have an a priori known probability of being included in the sample.

This process implies that if numerous probabilistic samples were taken from the same target population, there would be a variation in the sample results, as many of these samples would present different results, since the individuals making up the target population are different, and each sample taken at random could contain a different set of individuals. As the epidemiologist will only be selecting one sample in their study, which could be any one of a number of samples that could be selected, the fact that the results obtained from different possible samples will vary between them will force them, faced with the only sample selected, to ask themself: what information will I be able to obtain about the parameters of the target population through the results obtained from this single sample studied? The process of SI - with the calculation of p-values or CIs, in the case of classic SI - has the sole purpose of helping the epidemiologist answer this question.

With hypothesis testing, in which the p-value is calculated, the epidemiologist will test whether or not a given statement about a population parameter is refuted by the results of the sample. For example, in a cross-sectional study, one could test the null hypothesis that the prevalence ratio is equal to 1, i.e. the hypothesis that there is no association between the independent variable and the outcome.

With the help of the CI, the epidemiologist will assess the uncertainty surrounding the point estimate obtained from the sample. The smaller the amplitude of the CI, the greater the precision with which the population parameter of interest is known. A 95% confidence interval means that in 100 replications, 95% of them would contain the population parameter. The binary interpretation of the CI should be avoided, dichotomized into significant or non-significant, a practice that is unfortunately quite common, based solely on checking whether the CI includes the null value⁶,¹⁰,¹²,¹³. We will return to this criticism later.

Common situations of questionable use of SI

Consider that without SI it is not possible to assess differences between the subgroups studied

Some epidemiologists argue that they need to use SI because without it they wouldn’t be able to assess whether or not there is a statistical difference in some outcome of interest between the subgroups studied (for example, the prevalence of hypertension among older and younger workers in the study population), even if it is a census survey or with a non-probabilistic sample. But why does the epidemiologist, in this case, think they should assess whether there is a statistical difference between the elderly and young subgroups?

Statistically evaluating a difference between subgroups involves evaluating the sampling distribution for the estimator, in the example in question, the difference in the prevalence of hypertension between older and younger workers, or evaluating the test statistic, in the case of the hypothesis test for comparing prevalence between subgroups. If the assumptions about the probabilistic model and probabilistic sampling are not met, what guarantee can there be for the results of the CI or hypothesis test? Clearly, there are limitations to the use of SI methods in these situations which need to be considered and discussed critically. In census studies, as the entire target population has been investigated and there has been no sampling, the numerical difference (not the statistical difference) observed between the prevalence of hypertension between the subgroups is the difference that exists in the target population, provided that the study does not contain significant biases. The relevance of this numerical difference obtained can be assessed in terms of its magnitude and epidemiological, clinical, and/or pathophysiological significance.

Consider that without the use of SI the article will not have scientific validity or will not be publishable

Take the example of a study with a probabilistic sample that concluded, based on SI, that the difference found between the incidences of lung cancer among smokers and non-smokers was not due to simple sampling variation of the results, so there was a statistically significant result. In this example, the difference in incidence exists, it is not zero, in the target population from which the sample was taken. At the end of the study, the researcher will be able to conclude that a statistical association has been observed between smoking and lung cancer. But concluding that smoking causes lung cancer would not be the result of a single study.

The evidence obtained from a specific study can only add to the body of evidence already available. And the relationship between smoking and lung cancer is a classic example of the long process of attributing causality, given the accumulated body of evidence thanks to the various epidemiological investigations that have allowed this causal relationship to be consolidated. It is therefore important to consider that the relevance of the evidence from a single study will depend on the appropriate study design; the biological, pathophysiological, and epidemiological plausibility of the hypotheses formulated and the results observed; the strength of the associations found; and the consistency of the results when comparing them with those found in other studies. In the long term, this process of assessing causality will continue, through systematic reviews of the literature and meta-analyses, considering the results of various studies of good scientific quality and assessing whether or not they meet the criteria for attributing causality¹⁴,¹⁵. In view of the above, SI procedures, when applicable, will be important in each specific study to address the influence of sampling variation on its results. It is therefore necessary to distinguish the process of SI from the process of causal inference in epidemiology, which involves many other assessments.

Another point to consider, which can contribute to authors’ misconceptions, is that many editors of scientific journals also mistakenly consider the use of SI procedures in articles to be essential for publication. According to Rothman, Greenland, and Lash¹³:

Many researchers still believe that it would be fruitless to submit for publication any paper that lacks statistical tests of significance. Their belief is not entirely ill-founded, because many journal editors and referees still rely on tests of significance as indicators of sophisticated and meaningful statistical analysis as well as the primary means of assessing sampling variability in a study (p. 151).

Again, SI procedures were developed to help the researcher carefully extrapolate the results obtained from the sample to a target population.

Not considering the specifics of using SI in a randomized clinical trial

In a randomized clinical trial (RCT), when the study is well planned and conducted, two draws are made: one to select possible study participants and another (called randomization) to separate the groups to be compared. If the first draw is not carried out, but only the second, the epidemiologist will still be faced with the following question: was the difference observed in the occurrence of the outcome between the compared groups due to the different interventions to which these groups were subjected? Note that if another randomization were carried out, the groups selected might not be exactly those obtained in the only randomization carried out and, consequently, the difference found in the occurrence of the outcome between the groups might not be exactly the same, i.e. if numerous randomizations were carried out, there would be a variation in the results observed. The epidemiologist should then ask themself: based on the difference obtained in the compared groups, which difference is more likely to be the real difference between the groups? Thus, SI can help them answer this question, concluding with the use of SI - CI or p-values - on the difference most likely to be the true difference between the groups. However, as the method of selecting possible participants in the target population was not probabilistic, i.e. there was not the first draw mentioned above, the epidemiologist will have to judge how appropriate it is to generalize the results of the study to the target population and to populations with comparable clinical conditions, noting possible limitations in doing so¹⁶,¹⁷.

Statistical approaches to epidemiological data: the necessary delimitation of each stage

Epidemiology is classically assumed to be based on three pillars: clinical, social sciences, and statistics¹⁸. And the role of statistics is well recognized, identifying three different statistical approaches to epidemiological data, according to the purpose of the study: descriptive, analytical, and inferential¹⁷.

Through the descriptive approach, it is possible to quantitatively characterize a reality or the individuals studied, using simple, relative, and accumulated frequencies; calculating measures of central tendency, dispersion, and position; graphs; and evaluating the shape of distribution curves. There are various types of statistical tools used to characterize the population studied¹⁷.

The analytical approach investigates the existence of relationships between the characteristics or attributes of individuals. Thus, the analytical approach makes it possible to obtain epidemiological measures of association between two variables, expressed as mean ratios, prevalence ratios, incidence ratios (relative risk), and odds ratios. Other common techniques of the analytical approach provide correlation and linear regression coefficients, to name a few of the most common¹³. An illustration of the appropriate use of the analytical approach can be seen in the following excerpt from a study: “The sampling plan was non-probabilistic, so inferential statistics were not calculated due to the inadequate estimation of the standard error in such a circumstance [...]. To assess correlation, [...] Spearman’s correlation coefficient was used”¹⁹, without using the p-value or CI.

The inferential approach will take place when, based on data obtained from a probabilistic sample, it is intended to statistically infer results for the target population, as already presented in the section about SI principles and aims in Epidemiology. In this case, many statistical tests are available and will be selected based on the specific hypothesis to be tested, both in terms of the type and characteristics of the data collected¹⁷,²⁰. In addition, the estimation theory will allow the CIs to be obtained.

Note that, in adopting these three different approaches, we are starting from the contribution of Silvany-Neto¹⁷, who maintains the delimitation of Descriptive Statistics, but distinguishes Analytical Statistics from Inferential Statistics. The most common delimitations in classic literature are Descriptive Statistics and Inferential Statistics. In this sense, the proposed distinction seems crucial to deconstruct misconceptions regarding the use of SI based on the understanding that a study is analytical only if it makes use of Inferential Statistics.

These approaches to epidemiological data are used in isolation or in combination, according to the objective and nature of each study. For example, the sick leave rate was 19.9 per 100 people-year in a cohort study of oil industry workers, obtained through a census²¹. Or the prevalence of low back pain in plastic industry workers, a study with a probabilistic sample, was described as 28.9%, followed by a 95% CI (CI 22.0-35.8)²². In the first case, a descriptive approach and in the second, descriptive and inferential approaches.

Furthermore, when the aim is to assess the existence, direction, and magnitude of an association between variables, i.e. in an analytical study, if the sample is non-probabilistic - a census or convenience sample - a measure of association will be obtained. In the case of the cohort described, women had a 79% higher absence rate than men (HRaj: 1.79)²¹. And in a cross-sectional study of prison guards, female workers had 2.77 times the prevalence of major depressive disorder of their male colleagues²³. In the case of an analytical study with a probabilistic sample and which also aims to investigate the association between variables, the CI will be added to the measure of association: for example, the prevalence of pain in the proximal upper extremities among those exposed to high psychological demand in the footwear industry is 63% higher when compared to workers under low demand (RPaj: 1.63, 95%CI: 1.29-2.06)²⁴. On the other hand, in the cohort with oil industry workers²¹ and in the survey among prison guards²³, an analytical approach more suited to census studies was used. The study in the footwear industry used both an analytical and inferential approach, investigating musculoskeletal disorders in a random sample²⁴.

Generalization issues in epidemiological studies

The sources of uncertainty in the scientific research process have been recognized as those related to sampling variation, which is the subject of this essay, and those resulting from systematic errors, as suggested by Altman and Bland²⁵. The first concerns the accuracy or reliability of the study, the role of chance, and the second source refers to biases related to the validity of the study or accuracy²⁰. Although this text discusses issues relating to the precision of the study and the various ways of approaching it, it is important to mention the validity of the study and its link to the sample selection process.

The validity of a study is presented, in theory, as the absence of systematic errors or biases, whether of information or selection, and confounding. It is preferable to assume that the role of the epidemiologist is not to eliminate any and all biases; however, it is their role to try to do so and, more than that, to assess, explain, and document the impact of imperfections that have not been overcome.

Once the validity of the study has been ensured, the internal validity of the sample or study population, the aim is to extrapolate the results. The validity of the study is therefore the primary objective of an investigation: “The usefulness of research lies primarily in the generalisation of the finding rather than in the information gained about those particular individuals “²⁶ (p. 409).

In epidemiology, a distinction is made between internal validity and external validity. The former refers to the degree of reliability of the conclusions reached for the sample or study population itself. As for external validity, which is the extrapolation of the results of the sample beyond it, Pereira²⁷ distinguishes as the first extrapolation that of the sample studied for the target population and, as the second extrapolation, of the population studied (sample or census) for other populations. He also points out that the second extrapolation is “based on subjective judgments”. He states that: “A margin of uncertainty always surrounds any extrapolation, the interpretation of which is a mixture of science and art”²⁷. Similarly, Rothman et al.²⁸ state about extrapolating results to other populations: “That process is uncertain, along with everything else in empirical Science” (p. 1012).

The process of extrapolating the results obtained in a study will take place in different research situations which will imply different types of extrapolations, which relate to the external validity of the study, as will be detailed below.

Referring to the different research situations and the basis for extrapolating their results makes it clear that the use of SI - which plays a leading role in health research - if its assumptions are taken into account, should be restricted to a single extrapolation situation, the extrapolation of the results of a probabilistic sample to the target population from which the sample was taken. In this situation, and assuming that the sample selection does not involve significant systematic error, it is possible to use probability theory and SI methods to extrapolate the results of the sample to the population, as presented in the section about SI principles and aims in Epidemiology. In other cases, the use of SI can be questioned.

In a study with a probabilistic sample, a statistical test will evaluate the p-value of the test, i.e. the probability of more extreme values occurring for the test statistic than those observed with the sample, assuming that the null hypothesis is true. If the p-value is small enough, it indicates, for example, that the difference in means is most likely independent of chance, given a previously set significance level (the so-called “alpha” value), or if this probability is not small enough to allow you to rule out chance as an explanation for the difference obtained. Just to illustrate, we can note the frequent use of an “alpha” of 5%, but there are more demanding situations in the control of this type I error (rejecting a true null hypothesis), which use an “alpha”, for example, of 1%. In addition to statistical tests, SI can be based on CIs, as already mentioned.

A relevant aspect concerns the interpretation of CIs²⁵. Many authors have now reinforced the need to overcome the binary interpretation of the CI, which we mentioned earlier, i.e. to conclude that the result is not statistically significant (the one that includes the “null value”, 1, for measures of association - risk ratios or odds ratios - or 0 for difference measures), or statistically significant (the one that does not include it). For Schmidt and Rothman¹⁰:

Using CIs to judge whether the null value is within or not within the interval is equivalent to using the CI as a significance test, rather than as a quantitative tool for determining the size of an effect and the precision with which it is measured (p. 1090).

Therefore, in this case, refusing to interpret the CI as a dichotomous indicator of statistical significance, the researcher will describe the practical implications of the values within the interval. This will make it clear that all values between the limits of the interval are compatible with the data, given the statistical assumptions used in its calculation. For example, a 95% CI between 0.99 and 4.20, with a relative risk (RR) of 1.8, between a certain outcome and exposure, will allow the researcher to conclude that it is much more likely that there is this association, than obtaining a 95% CI between 0.76 and 3.50, considering the same RR¹¹,¹².

Consider the assumptions of SI is the procedure envisaged by researchers who are critical of its indiscriminate use. According to Daniel and Cross²⁹, “in many instances in real-world applications it is impossible or impractical to use truly Random Samples” (p. 164) and they postulate, regarding the extrapolation of the results thus obtained that: “nonstatistical considerations must play a part in the generalization process “ (p. 165).

The success of this extrapolation process will depend on the degree of similarity between the populations being compared, when “there is no reason to believe that the material actually used is not representative of the population about which inferences are desired” (p. 165). And they reaffirm: “Again, generalizations must be made on the basis of nonstatistical considerations”²⁹ (p. 165).

It is important to emphasize once again that, even if a study complies with the assumptions of SI, it can incur systematic errors that can compromise its validity, interfering with the representativeness of the group initially defined as the target population: the researcher must evaluate these limitations. But the validity of a study - the chapter on biases, which should be reasonably well understood by epidemiologists - implies another subject for discussion, which is not the aim of this essay.

In addition, it is important to check the state of the art on the subject under study. Ultimately, in any of these situations, whether using a probabilistic sample, a convenience sample, or a census study, epidemiologists should seek support for their conclusions in knowledge produced in the medium and/or long term, through repeated studies in different locations and populations, using different study designs, and which provide consistent results⁵.

Conclusion

In this essay, the foundations for the application of SI were presented, which allow statistical approaches to epidemiological data to be organized as descriptive, analytical, and inferential. It is important to note that a study is analytical when it investigates associations between variables and not because it uses SI. This distinction is essential and timely for the necessary criticism of the use of SI.

When the sample is probabilistic, the results of the sample are extrapolated to the target population through the SI process. In the case of non-probabilistic samples - judicious or intentional - or even in censuses, the use of CIs or hypothesis tests can be questioned as to the possibility of inadequate results in such circumstances. Considerations about the limitations of these methods for making generalizations in these situations should guide the interpretation of the results.

It is hoped that this essay will help to stimulate the critical and parsimonious use of SI in epidemiological studies, recognizing its contribution to the causal inference process, but distinguishing SI from it. Furthermore, given the usual use of studies with non-probabilistic samples or censuses, as presented in this essay, it is necessary for epidemiologists to note possible limitations, adopting the necessary parsimony in interpreting their findings and the impact of using SI in these situations. This attitude should encourage debate about the scientific validity of epidemiological studies.

Acknowledgments

The authors would like to thank Professor Annibal Muniz Silvany Neto, an epidemiologist with an outstanding dedication to statistics, for his important contributions.

References

• ¹ Greenland S. Randomization, statistics, and causal inference. Epidemiol.1990 Nov;6(1):421-9. https://doi.org/10.1097/00001648-199011000-00003
  » https://doi.org/10.1097/00001648-199011000-00003
• ² Greenland S, Poole C. Problems in common interpretations of statistics in scientific articles, expert reports, and testimony. Jurimetrics. 2011;51(2):113-29.
• ³ Hahn GJ, Meeker WQ. Assumptions for statistical inference. Amer Stat. 1993;47(1):1-11. https://doi.org/10.1080/00031305.1993.10475924
  » https://doi.org/10.1080/00031305.1993.10475924
• ⁴ Strasak AM, Zaman Q, Pfeiffer KP, Göbel G, Ulmer H. Statistical errors in medical research: a review of common pitfalls. Swiss Med Wkly. 2007 Jan;137(3-4):44-9. https://doi.org/10.4414/smw.2007.11587
  » https://doi.org/10.4414/smw.2007.11587
• ⁵ Hubbard R, Haig BD, Parsa, RA. The limited role of formal statistical inference in scientific inference. Amer Stat. 2019;73(sup1):91-8. https://doi.org/10.1080/00031305.2018.1464947
  » https://doi.org/10.1080/00031305.2018.1464947
• ⁶ Wasserstein RL, Schirm AL, Lazar NA. Moving to a World Beyond “p < 0.05”. Amer Stat. 2019;73(sup1):1-19. https://doi.org/10.1080/00031305.2019.1583913
  » https://doi.org/10.1080/00031305.2019.1583913
• ⁷ Wasserstein RL, Lazar NA. The ASA statement on p-values: context, process, and purpose. Amer Stat. 2016; 70(2):129-133. https://doi.org/10.1080/00031305.2016.1154108
  » https://doi.org/10.1080/00031305.2016.1154108
• ⁸ DeGroot MH, Schervish, MJ. Probability and statistics. 4a ed. Boston: Addison-Wesley; 2012.
• ⁹ Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337-50. https://doi.org/10.1007/s10654-016-0149-3
  » https://doi.org/10.1007/s10654-016-0149-3
• ¹⁰ Schmidt M, Rothman KJ. Mistaken inference caused by reliance on and misinterpretation of a significance test. Int J Cardiol 2014 Dec;177(3):1089-90. https://doi.org/10.1016/j.ijcard.2014.09.205
  » https://doi.org/10.1016/j.ijcard.2014.09.205
• ¹¹ Szklo M. Quality of scientific articles. Rev Saude Publica. 2006;40(spec):30-5. https://doi.org/10.1590/s0034-89102006000400005
  » https://doi.org/10.1590/s0034-89102006000400005
• ¹² Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305-7. https://doi.org/10.1038/d41586-019-00857-9
  » https://doi.org/10.1038/d41586-019-00857-9
• ¹³ Rothman KJ, Greenland S, Lash TL. Precision and statistics in epidemiologic studies: modern epidemiology. 3a ed. Philadelphia: Lippincott Williams & Wilkins, 2008. P. 151.
• ¹⁴ Hill AB. The environment and disease: association or causation? Proc R Soc Med 1965 May;58(5):295-300.
• ¹⁵ Rothman KJ, Greenland S. Causation and causal inference in epidemiology. Am J Public Health. 2005;95 (Suppl 1): S144-50. https://doi.org/10.2105/AJPH.2004.059204
  » https://doi.org/10.2105/AJPH.2004.059204
• ¹⁶ Friedman LM, Furberg CD, DeMets D, Reboussin DM, Granger CB. Fundamentals of clinical trials. 5a ed. Switzerland: Springer Cham; 2015.
• ¹⁷ Silvany Neto, AM. Bioestatística sem segredos. Salvador: Annibal Silvany Neto; 2008.
• ¹⁸ Susser M. Epidemiology, health & society: selected papers. New York: Oxford University Press; 1987.
• ¹⁹ Oliveira JL, Camelier FWR, Jesus FR, Barreto RMA, Ferreira PP, Terse-Ramos R et al. Perfil ventilatório e capacidade funcional de pacientes com mucopolissacaridoses. Fisioter Bras. 2017;18(1):69-79. https://doi.org/10.33233/fb.v18i1.757
  » https://doi.org/10.33233/fb.v18i1.757
• ²⁰ Henneckens CH, Buring JE, Epidemiology in medicine. Boston: Little, Brown and Company; 1987.
• ²¹ Almeida CGDSTG, Silvany Neto AM, Fernandes RCP. Risk factors for sick leave in Brazilian oil industry workers. Int Arch Occup Environ Health. 2021 Aug Aug;94(6):1223-1231. https://doi.org/10.1007/s00420-021-01650-w
  » https://doi.org/10.1007/s00420-021-01650-w
• ²² Fernandes RCP, Assunção AA, Carvalho FM. Prevalence of musculoskeletal disorders among plastics industry workers. Cad Saude Publica. 2011 jan;27(1):78-86. https://doi.org/10.1590/S0102-311X2011000100008
  » https://doi.org/10.1590/S0102-311X2011000100008
• ²³ Santos SN, Santos KOB, Carvalho FM, Fernandes RCP. Major depressive disorder in detention officers. Rev Saude Publica. 2021 Apr 14;55:11. https://doi.org/10.11606/s1518-8787.2021055002507
  » https://doi.org/10.11606/s1518-8787.2021055002507
• ²⁴ Carvalho RLRB, Fernandes RCP, Lima VMC. Demandas psicológicas, baixo apoio social e repetitividade: fatores ocupacionais associados à dor musculoesquelética de trabalhadores da indústria de calçados. Rev Bras Saude Ocup. 2019;44:e6. https://doi.org/10.1590/2317-6369000001517
  » https://doi.org/10.1590/2317-6369000001517
• ²⁵ Altman DG, Bland JM. Uncertainty beyond sampling error. BMJ 2014;349: g7065. https://doi.org/10.1136/bmj.g7065
  » https://doi.org/10.1136/bmj.g7065
• ²⁶ Altman DG, Bland JM. Generalisation and extrapolation. BMJ.1998;317(7155):409-10. https://doi.org/10.1136/bmj.317.7155.409
  » https://doi.org/10.1136/bmj.317.7155.409
• ²⁷ Pereira MG. Epidemiologia: teoria e prática. Rio de Janeiro: Guanabara Koogan; 1995.
• ²⁸ Rothman KJ, Gallacher JEJ, Hatch EE. Why representativeness should be avoided. Int J Epidemiol. 2013;42(4):1012-4. https://doi.org/10.1093/ije/dys223
  » https://doi.org/10.1093/ije/dys223
• ²⁹ Daniel WW, Cross CL. Biostatistics: a foundation for analysis in the health sciences. 10th ed. Hoboken: John Wiley; 2013.

• Data availability:
  The entire data set supporting the results of this study has been published in the article itself.
• Presentation of the study at a scientific event:
  The authors inform us that the study has not been presented at a scientific event.

• Funding:
  The authors declare that the study was not funded.

Edited by

• Editor-in-Chief:
  Eduardo Algranti

Data availability

The entire data set supporting the results of this study has been published in the article itself.

Publication Dates

• Publication in this collection
  13 Jan 2025
• Date of issue
  2025

History

• Received
  28 Sept 2022
• rev-request
  09 Feb 2023
• Accepted
  14 Feb 2023