Occupational Self-Efficacy Scale - Short Form (OSS-SF): Invariance and Item Analysis via Rating Scale ModelABSTRACT
The present study aimed to gather new evidence of validity for the Occupational Self-Efficacy Scale - Short Form (OSS-SF). Exploratory factor analysis indicated a one-factor solution with satisfactory results, and confirmatory factor analysis endorsed the results obtained. The invariance analysis showed that the OSS-SF was equivalent across female and male participants and diverse career tenures of professional activity. Item Response Theory provided an understanding of the organization of the continuum represented by the occupational self-efficacy construct, as well as the item difficulty. It is concluded that the abbreviated version of the OSS-SF is an adequate measure to assess occupational self-efficacy of different groups and identify those with a low level of the latent trait.
KEYWORDS:
psychometry; psychological assessment; work
RESUMO
O presente estudo teve como objetivo reunir novas evidências de validade para a Occupational Self-Efficacy Scale - Short Form (OSS-SF). A análise fatorial exploratória indicou uma solução de um único fator com resultados satisfatórios, e a análise fatorial confirmatória corroborou os resultados. A análise de invariância mostrou a equivalência da OSS-SF entre participantes do sexo feminino e masculino, bem como entre diferentes trajetórias da atividade profissional. A Teoria de Resposta ao Item demonstrou a organização do continuum representado pelo construto de autoeficácia ocupacional, assim como a dificuldade dos itens. Conclui-se que a versão abreviada da OSS-SF é uma medida adequada para avaliar a autoeficácia ocupacional em diferentes grupos, permitindo identificar aqueles com baixo nível do traço latente.
PALAVRAS-CHAVE:
psicometria; avaliação psicológica; trabalho
The organizational environment in recent decades has become faster, more dynamic, more globalized, and consequently more uncertain, unpredictable, and fluid (Bakker & Demeroutti, 2017). Such changes have led organizations to rethink and reorganize their structure, including their employees, who need to keep up with labor demands, performing work with high levels of flexibility and adaptability (Bakker & Demerouti, 2017; Lim, 2023).
Changes in the organizational context, integrated with the complexity of the professionals’ relationships with their work, have motivated the development of studies and instruments that investigate and measure which characteristics of the occupational and professional environment reverberate on workers’ health conditions and levels of well-being (Bakker & Demerouti, 2017; Mastroianni & Storberg-Walker, 2014; Monteiro & Joseph, 2023). In this perspective, it is observed that the personal resource of occupational self-efficacy (OSE) has great relevance, as it enables job crafting or improvement in the work, regulating and readjusting the relationship between the individual and the task (Mielniczuk & Laguna, 2018).
Self-efficacy can be defined as the belief an individual has about his or her ability to perform and manage the required actions to achieve certain goals in different areas. Self-efficacy beliefs impact the activities that individuals engage in, the definition of their purposes and goals, as well as the level of stress they will endure throughout the development of their activities (Bandura, 1997). It is noteworthy that self-efficacy is a psychological construct with close contextual interdependence (Bandura, 1982). However, investigating self-efficacy in specific contexts enables a more precise and contextualized analysis of self-efficacy beliefs and how they influence behavior and outcomes within a particular environment (Bandura, 1977). Furthermore, analyzing self-efficacy in specific situations allows comprehension of its effects as a predictor of relevant behaviors in different situations, offering valuable insights into human behavior, performance, motivation, and resource management, providing a nuanced understanding of how individuals perceive and act in different contexts (Duan et al., 2024; Reyhing & Perren, 2023). This variable has gained strength in relation to the number of research studies developed in the work context, where it is, therefore, called occupational self-efficacy (Peixoto et al., 2023; Volmer et al., 2023).
Workers who exhibit high levels of occupational self-efficacy are successful in mobilizing new resources in stressful and high-demand situations, as occupational self-efficacy is positively associated with higher levels of intrinsic motivation, performance (Çetin & Aşkun, 2018), flow, and job satisfaction (Freitas et al., 2019), physical and psychological discomfort (Luna & Gondim, 2021), and lower levels of burnout (Freitas et al., 2016). Furthermore, workers who have higher levels of occupational self-efficacy tend to be more apt to make more assertive choices and not procrastinate, as well as worry and plan for the future and adverse situations arising from their jobs (Salvador & Ambiel, 2019). Such results provide evidence that professionals with higher levels of occupational self-efficacy tend to show less propensity to experience the process of health impairment and greater chances of developing the motivational process (Bakker & Demerouti, 2017; Mugiono et al., 2020; Nunes & Noronha, 2011; Rigotti et al., 2008; Schaufeli, 2017; Volmer et al., 2023).
As evidenced, research in Organizational and Work Psychology has dedicated efforts to increasing understanding of OSE and its relationship with other variables that make up this context (Luna & Gondim, 2021; Mugiono et al., 2020; Schaufeli, 2017; Shoji et al., 2016). Currently, different instruments can measure occupational self-efficacy, such as the Self-Efficacy Scale for Occupational Activities - EAAOc (Nunes & Noronha, 2011), Occupational Self-Efficacy Scale - OSS (Schyns & von Collani, 2002), and the Occupational Self-Efficacy Scale - Short Form - OSS-SF (Rigotti et al., 2008). Among the available instruments, the OSS-SF is the most widely used in the scientific environment (Damásio et al., 2014) because it has a reduced set of items and psychometric evidence that supports its safe application (Rigotti et al., 2008).
In the Brazilian context, the OSS-SF was adapted by Damásio et al. (2014) with a sample composed of 514 individuals (86.4% women), aged between 18 and 68 years (M = 37.91; SD = 10.78). The psychometric properties observed guarantee the use of the instrument in the Brazilian context, since the fit indices of the one-dimensional internal structure (with six items) was reproduced (χ²/df =1.64; RMSEA (90% C.I.) = .05 (.00 - .09); CFI = .98; TLI = .96). It is worth noting that in the study developed by Damásio et al. (2014), at a first moment, in the exploratory factor analysis, item one did not load according to the expectation of the original structure. The low factor loading of the first item was justified by the semantic complexity of the item, given that it presented two statements simultaneously, in order to question whether participants remain calm when facing difficulties and whether they trust their abilities.
In the confirmatory factor analysis, item 1 showed adequate factor loading, and the fit indices of the six-item model were higher than the five-item OSS-SF solution. Therefore, these results showed that the Brazilian version of the OSS-SF was similar to the version proposed by Rigotti et al. (2008). In addition, other studies that used the scale showed adequate adjustment and internal consistency indices for the one-dimensional solution of the OSS-SF (Freitas et al., 2016; Pimenta de Devotto et al., 2020; Salvador & Ambiel, 2019).
It is noteworthy that studies conducted nationally, as well as internationally, to evaluate the psychometric properties of the OSS-SF, indicate the efficiency and adequacy of the instrument to measure occupational self-efficacy (Merino-Soto et al., 2021; Peng et al., 2021; Rigotti et al., 2008). The results found in Brazil (e.g., Damásio et al., 2014; Freitas et al., 2016; Salvador & Ambiel, 2019) pointed to a unidimensional structure similar to the original version of the scale. Although initial studies already show evidence of validity based on internal structure as well as accuracy, further investigations are needed. Therefore, the present study aims to continue the search for validity evidence for the Brazilian version of the OSS-SF and, to do so, the analysis based on Item Response Theory (TRI) was used.
TRI refers to a mathematical model that expresses the relationship between observable variables, which are the behaviors emitted by individuals (represented through the items), and the latent traits (or non-observable variables), which represent the respondent's aptitude. Therefore, the answer given to the item demonstrates the level of latent trait that the individual possesses. TRI is structured from two basic assumptions: 1) a grouping of latent traits can predict the performance (probability of endorsement or not) of subjects on a test item, and 2) this relationship between the observed variable and the latent trait can be demonstrated from the Item Characteristic Curve (ICC). Thus, TRI makes it possible to calculate the level of aptitude, the level of difficulty, and discrimination of items, as well as linking the items to the subject's level of aptitude (Pasquali & Primi, 2003). Such insights enrich the interpretation of test outcomes, contribute to the refinement and optimization of assessment tools, allow the prediction of individual performance based on latent traits, and enhance the accuracy of assessments by linking observable behaviors to underlying aptitudes. Ultimately, TRI analysis fosters a more accurate and comprehensive understanding of individuals' abilities and traits.
Furthermore, it was also intended, in order to gather new validity evidence that contributes to operationalizing the OSE, to perform psychometric invariance analyses to assess the stability of the OSS-SF. Since such evidence can provide researchers with the possibility of comparing between groups, besides ensuring the use of an efficient tool (American Psychological Association, American Educational Research Association, & National Council on Measurement in Education [APA, AERA, & NCME], 1999). Therefore, the objective was to obtain validity evidence based on the internal structure through the invariance of the one-dimensional factor model, at different levels (configurational, metric, and scalar), between groups established according to gender and working time.
Method
Participants
Participants included 2,502 professionals aged 18 to 77 years (M = 39.9; SD = 11.17) and of both genders (60.9% female). Participants were from all regions of Brazil (2% were from the North, 11% from the Northeast, 4% from the Midwest, 53% from the Southeast, and 30% from the South). The average working time was 13.6 years, ranging from two to 53 years. Of the workers, 46% worked in the area of teaching or research (e.g., university professors, educational psychologists, researchers), 34% worked in the healthcare sector (e.g., doctors, psychologists, nurses), 11% were service providers (e.g., lawyers, accountants), 5% in the commerce sector (e.g., salespersons, managers), 2% worked in the industrial sector (e.g., engineers, architects, designers), and 2% in other areas (e.g., musicians, artists). Regarding education, 49.7% had complete postgraduate degrees, 27.5% had complete graduate degrees, and 9.3% had incomplete graduate degrees.
Instruments
Sociodemographic Questionnaire. Structured specifically for this study to gather pertinent information about the participants, such as age, gender, region, level of education, and area and time of work.
Occupational Self-Efficacy Scale - Short Form (OSS-SF). The OSS-SF (Rigotti et al., 2008) refers to a shortened form of the OSS (Schyns & von Collani, 2002). Translated and adapted for the Brazilian context by Damásio et al. (2014), it was named the Occupational Self-Efficacy Scale - Short Form (OSS-SF). The instrument is composed of six items (e.g., "I achieve the goals I set for myself in my work.") that are answered on a five-point Likert scale, ranging from 1 (strongly disagree) to 6 (strongly agree). The single-factor structure was ensured by confirmatory factor analysis χ²/df = 1.64, SRMR = .07, RMSEA (90% C.I.) = .05 (.01 - .09), CFI = .98, TLI = .96.).
Procedures
Ethical Aspects
The project was submitted to the Research Ethics Committee. After approval, the instruments were allocated on the online platform Google Forms, and the link was shared on the authors' social networks and their networks. Participants had to agree to the Informed Consent Form (ICF) and be 18 years old or older. They were then directed to the instruments, which were presented in the following order: sociodemographic questionnaire and OSS-SF. It was estimated that the form was completed in approximately 5 minutes.
Data Analysis Plan
To evaluate the internal structure of the OSS-SF, we used Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) procedures. For this purpose, the total sample was randomly divided into two subsamples (sample 1 and sample 2). As a factor retention method, parallel analysis (PA) based on Minimum Rank Factor Analysis (Timmerman & Lorezo-Seva, 2011) was employed. AFE was performed with the Unweighted Least Squares (ULS) estimation method, based on polychoric correlation matrices and items being stated as ordinal variables.
With sample 2 data, AFC was performed with Weighted Least Squares Mean and Variance-adjusted (WLSMV) estimation method. The fit of the data to the measurement model was evaluated from the fit indices recommended by Muthén and Muthén (2017), these being: WLSMV χ2, df, χ2/df, RMSEA, CFI, and TLI. The reference values commonly employed in the specialized literature were used as fitting parameters: χ2/df < 5, RMSEA < .08, CFI and TLI > .90.
Subsequently, the subsamples were regrouped to evaluate the invariance of the measurement model between groups established according to the gender of the participants and time of work experience (up to 10 years, between 10 and 20 years, and over 20 years). To do so, we used Confirmatory Factor Analysis Multigroup (CFA-MG) and estimation of configural, metric, and scalar models. The same adjustment indexes were considered. The assumption of between-group invariance was assessed using the variability of the CFI, with a threshold of ∆CFI ≤ .01 (Cheung & Resnvold, 2002). These analyses were performed with the support of the MPlus statistical package version 7.3 (Muthén & Muthén, 2017).
The Item Response Theory (IRT) was used to evaluate the item parameters, more specifically the Rating Scale Model (RSM) or Graded Response Model, which is part of the Rasch family of measurement and is configured as an annex of the Rasch dichotomous model for a sequence of graded responses. Considering the unidimensionality of the instrument (composed of one parameter), an estimation of the item difficulty levels, the fit indices (Infit/Outfit), the participants' total latent trait level, the item information curve, and the test information curve was performed (Linacre, 2015). The internal consistency of the scale was assessed through Cronbach’s alpha and Omega. Regarding the item fit indices (Infit/Outfit), values between .7 and 1.3 were considered good indicators (Bond & Fox, 2001). The analyses were performed using the statistical software R (R Core Team, 2022) and the MIRT package (Chalmers, 2012).
Results
Exploratory Factor Analysis (EFA)
The polycorrelation matrix proved to be appropriate for factor extraction (Kaiser-Meyer-Olkin, KMO = .85, Bartlett's test of sphericity χ2 [df:15] = 2,665.1; p < .001) and presented a single-factor solution (eigenvalue = 3.33; explained variance = 64.93%) through the parallel analysis method, corresponding to the theoretical proposal. Table 1 shows that the six items presented satisfactory factor loadings in the expected factor, with values between .634 (item 4) and .729 (item 6). The scale's internal consistency index, verified through the Cronbach alpha coefficient, was .83, and Omega was .90.
Confirmatory Factor Analysis (CFA)
To confirm the factor model observed in the EFA, a CFA was conducted with Sample 2. At first, the results showed adequate fit indices for the exploratory model, except for the RMSEA: χ²=216.126; df = 9, p < .001; RMSEA = .13, 90% C.I. .121-.153; CFI = .94; TLI = .90. To verify better indices, two errors were included in the model. As presented by Figure 1, fits classified as good were observed: χ² = 63.722, df = 7, p < .001; RMSEA = .08, C.I. = 90% .063-.100; CFI = .98; TLI = .96.
Confirmatory Factor Multigroup Analysis (CFA-MG)
With the observed results regarding the internal structure and reliability of the OSS-SF, the invariance of the factor model between men and women, as well as regarding working time, was tested at the configural, metric, and scalar levels. The outcomes of these analyses are presented in Table 2. Furthermore, it is possible to notice that the factorial model was invariant at all levels, evidencing the equivalence of the unidimensional model for evaluating men and women and workers with up to 10 years, between 11 and 20 years, and over 20 years of work, since the adjustment indexes proved adequate (∆CFI < 0.01).
Rasch Analysis - Rating Scale Model (RSM)
The psychometric properties of the instrument were assessed using the IRT RSM, and the results are presented in Table 3. Regarding the difficulty levels of the categories, item 1 presented the highest mean difficulty (M = -1.455), while item 3 presented the lowest mean (M = -1.237). It was verified that the fit indices (infit and outfit) indicate that the items demonstrate congruent behavior to the expectation of the model, with values considered as adequate. The chi-square (χ²) ranged between 46.598(43) - item 3 - and 100.216(50) - item 4 - and the RMSEA, in turn, ranged between .008 (item 5) and .022 (item 1).
Regarding the thresholds between Likert categories, it was expected that the items would present increasing difficulty, so that higher levels of self-efficacy would be associated with higher scores. The findings evidenced that items 1, 2, 4, 5, and 6 differentiated between very low and very high levels. However, there was an overlap between the second and third response categories for these items (Figure 2), so that the items did not discriminate against moderate scores. It should be noted that only item 3 differentiated the difficulty thresholds of the four categories evaluated (Figure 2).
In Figure 3, the association between the level of item difficulty and the level of latent trait (theta) demonstrated by the participants was presented. Thus, it can be observed that although the items can cover a significant magnitude of the latent trait level range, they are more informative for evaluating participants with a below-average level of theta. This can be understood as a potentiality of the instrument, given that it was developed with the aim of identifying individuals with low levels of perceived occupational self-efficacy.
Discussion
The present study aimed to search for new validity evidence for the OSS in its abbreviated version, based on different psychometric analyses. In general, the results indicated the adequacy of the one-factor structure with a good level of internal consistency, as well as a strong invariance of the model tested to measure OSE characteristics in different groups, such as males and females, and also to evaluate workers aged up to 10 years, between 11 and 20 years, and over 20 years. Thus, it is possible to infer that the measurement of OSE occurs in a similar way in these groups when the OSS-SF is used. Finally, from the RSM, the results indicated good quality of the items, besides demonstrating that the OSS-SF evaluates in a way that captures a higher level of information from individuals with a low level of latent trait, corroborating the potential of the instrument in identifying professionals who may benefit from deeper psychological processes and/or future psychosocial interventions.
It is worth mentioning that the accumulation of validity evidence is fundamental to guarantee an adequate interpretation of the scores estimated from a psychological measure, and for this, it is necessary to constantly investigate the psychometric aspects of an instrument, since such studies make it possible to accumulate evidence of how and how well the test measures the construct in question. The search for validity evidence can be evaluated in different ways, such as those based on the classical theory of test validation, as well as those focused on modern psychometrics with the application of TRI (APA, AERA & NCME, 1999; Pasquali & Primi, 2003).
In classical test theory (TCT), there is a search for validity evidence based on internal structure, which is divided into three basic aspects: factor structure, reliability, and measurement invariance (APA, AERA & NCME, 1999). Regarding factor structure, the results indicated, through the EFA, a single-factor structure for the OSS-SF, with satisfactory adjustment indexes, composed of six items in its abbreviated version, which presented high factor loadings. The values obtained corroborate those found by Damásio et al. (2014), with the exception of item one, which showed significant factor loadings in the present study. The CFA, in turn, endorsed the results obtained in the EFA, also presenting satisfactory adjustment indexes and, furthermore, through Cronbach's alpha and Omega values, demonstrated adequate internal consistency. The findings of the present study are similar to studies previously conducted in the Brazilian context with the use of the one-dimensional factor solution (Damásio et al., 2014; Freitas et al., 2016; Salvador & Ambiel, 2019).
Concerning factor invariance, the CFA-MG demonstrated equivalence of the OSS-SF between female and male subjects, as well as different times of professional performance in the three models tested (configurational, metric, and scalar). The results indicated that between one model and its subsequent one, there was no significant worsening of the fit indices (ΔCFI ≤ .01), thus evidencing strong invariance of the scale across different groups (Chen, 2007; Cheung & Rensvold, 2002). The values obtained demonstrate that the OSS-SF is configured as an instrument that can be applied in different samples, since such findings indicate that the parameters of the items show stability in subjects with different profiles, enabling comparison in an unbiased manner (Sass, 2011).
In addition to the TCT, the TRI provides information about the properties of the items, the latent trait of the respondents, as well as information regarding the interaction between the characteristics of individuals and item properties (Pasquali & Primi, 2003). Thus, the results obtained through RSM indicated good item quality, with Infit and Outfit fit indices within the range considered appropriate (Linacre, 2015).
From this result, it is possible to conclude that the abbreviated version of the OSS comprises items that fit the response pattern expected by the model. More specifically, in accordance with the infit statistic, which checks for possible discrepancies between items with difficulty close to the subject's latent trait level, as well as the outfit statistic, which checks for possible discrepancies in the response pattern of subjects with latent trait estimates that are located far from the difficulty level of the items, the items proved productive (Linacre, 2015). The quality of the items to compose the measurement model is also corroborated by the RMSEA statistic, with values less than.05, indicating a low magnitude of this discrepancy between the responses expected by the model and the empirically observed responses (Chalmers, 2012).
Regarding the level of difficulty of the items, from the level of difficulty of the first response option, one can specify the continuum presented by the items according to this psychometric property. Thus, it was observed that item 1 was estimated as the most easily endorsed by respondents, accompanied by item 6. Based on the content of the items, it can be inferred that this greater ease of endorsement is associated with the recognition of skills, obtained from previous experiences, to face most difficulties and demands in the work environment.
In accordance with Bandura´s (1997) proposal, the ability to interpret previous experiences corresponds to the greatest influence on the formation of self-efficacy beliefs. It provides legitimate information regarding the individual's behavior, indicating to him the possibility of obtaining or not success in a certain activity. In this direction, Nunes (2008) suggests that successful situations tend to intensify self-efficacy beliefs and, therefore, correspond to the cognitive resource of easiest access to the scale respondents. Thus, making the content more easily endorsed.
In sequence, on the difficulty level are allocated items 5 and 2, whose content concerns goal attainment and finding varied solutions to a problem. In accordance with the theoretical proposition, it appears that the content of these items is associated with the source of self-efficacy of verbal persuasion, which is characterized by the positive or negative impact of verbal evaluation (feedback) from others, and thus can increase or decrease the self-efficacy beliefs of an individual (Pajares, 2002). In view of this, when professionals receive information from the professionals around them regarding their ability to perform and the intended resolution, they tend to increase their efforts to obtain success when facing similar challenges.
Finally, items 3 and 4 presented a higher level of difficulty of endorsement by the participants, since the content reports the facing of any future obstacle that will be faced in the work environment. These contents are associated with the source of self-efficacy based on vicarious learning, which refers to observing other people performing a certain task and inferring one's own level of ability from comparison (Bandura, 1997). However, it is worth noting that this is a less efficient source of self-efficacy than the previous ones in maintaining effort and coping with work problems since it is based on observing other people's success or failure. For it is difficult to imagine a professional feeling able to "handle any obstacle" or feeling prepared for the professional future just based on previous personal experiences. In this sense, the intensity reflected in the content of the items translates statistically into their levels of difficulty.
The last source of self-efficacy was termed by Bandura (1997) as physiological indicators. These refer to the self-evaluation of anxiety, stress, and mood states during the performance of an activity. Although this source does not explicitly present from the ordering based on the difficulty of the items, it can be inferred that these physical and emotional reactions act as an indication of perception regarding the success or failure in the activity, as well as in the interpretation of external stimuli (Pajares & Olaz, 2008), therefore, it is possible to infer that this source permeates the others.
These findings shed light on an important discussion regarding the internal structure of self-efficacy instruments, which, although developed based on the theoretical framework of the four sources of self-efficacy from Social Cognitive Theory (SCT - Bandura, 1997), mostly present a unidimensional structure (e.g., Nunes & Noronha, 2011; Rigotti et al., 2008; Schyns & von Collani, 2002). For it is possible to infer that although the subjects' responses do not reflect a structure composed of four different factors, these theoretical nuances can be observed in the arrangement of a self-efficacy continuum based on the difficulty/content of the items.
Regarding the information curve of the test, the results indicated greater potential for the OSS-SF to identify respondents with low levels of the latent trait. The potential of a brief scale to discriminate professionals with low levels of occupational self-efficacy is relevant, as it allows for the recognition of those who are more susceptible to mental health decline (Damásio et al., 2014; Freitas et al., 2016; Rigotti et al., 2008), so that they may receive interventions to strengthen their OSE levels and, consequently, experience higher levels of organizational well-being.
Finally, we highlight the observations made from the inspection of the response characteristic curve, as shown in Figure 1, in which it is possible to identify the overlap of response categories 2 and 3 of the Likert scale. This suggests the appropriateness of adopting a 4-point response key. Discussions of this nature have been proposed in the psychometric literature (Asún et al., 2015; Colledani et al., 2024; Simms et al., 2019). In this regard, the understanding that the 4-point composite response scale is similar to a forced-choice Likert scale stands out since there is no neutral choice option. The absence of a neutral category forces the subject to issue their level of agreement regarding the content of the item (e.g., 1=Strongly disagree; 2= Somewhat disagree; 3= Somewhat agree; and 4= Agree), which may enable greater accuracy of the results (Asún et al., 2015). Therefore, a 4-point scale is preferred, as long as it does not affect the validity and accuracy of the instrument, since the presence of a midpoint (e.g., 3= Neither disagree nor agree) may skew the results (Colledani et al., 2024; Simms et al., 2019). Future studies should investigate the adequacy of the OSS-SF using a four-point response key.
It is therefore concluded that different psychometric models, such as Classical Test Theory and modern Item Response Theory, complement each other and add new knowledge in validating and understanding the benefits and weaknesses of the instrument in assessing latent variables (Lin et al., 2019). In this sense, TCT allowed the measurement of validity evidence based on the internal structure, reliability, and invariance of the instrument, while TRI expanded the understanding of the properties of the items, in addition to the greater potential for exploration regarding the continuum that represents occupational self-efficacy, and with this, greater knowledge regarding the properties of the Brazilian version of the OSS-SF.
Conclusions
Investigating the psychometric qualities of a measure enables the operationalization of the construct and guarantees an adequate interpretation of the obtained scores. Thus, the present study aimed to estimate new validity evidence based on the internal structure from the Classical Test Theory, as well as to perform new analyses based on Item Response Theory. The results obtained indicated, in general, that the OSS in its abbreviated version showed adequate factor structure and unidimensional internal consistency and strong invariance at the different configural, metric, and scalar levels. In addition, the RSM indicated good item quality. It is understood that researches that seek new validity evidence are important for the advancement of studies on psychological instruments and, consequently, for the area of Psychological Assessment in the Brazilian context, since the results indicate that the OSS-SF operationalizes the OSE.
The study's limitations include the analysis of a convenience sample, so that, although it had participants from the five regions of the country, the professionals were predominantly from the southeast region and had completed a college education. Therefore, it is suggested that future studies be carried out with a more comprehensive sample of professionals from the North and Midwest regions, and that efforts be dedicated to interviewing workers who had elementary and middle school education levels, while also acknowledging limitations regarding occupational types and the self-report nature of the measure. Expanding the evidence of the OSS-SF allows us to accumulate evidence to ensure that the instrument is beneficial and works properly when applied to different groups.
References
- American Psychological Association, American Educational Research Association, & National Council on Measurement in Education [APA, AERA & NCME] (1999). Standards for educational and psychological testing. American Educational Research Association.
-
Asún, R. A., Rdz-Navarro, K., & Alvarado, J. M. (2015). Developing multidimensional Likert scales using item factor analysis. Sociological Methods & Research, 45(1), 109-133. https://doi.org/10.1177/0049124114566716
» https://doi.org/10.1177/0049124114566716 -
Bakker, A. B., & Demerouti, E. (2017). Job demands-resources theory: Taking stock and looking forward. Journal of Occupational Health Psychology, 22(3), 273-285. https://doi.org/10.1037/ocp0000056
» https://doi.org/10.1037/ocp0000056 -
Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191-215. https://doi.org/10.1037/0033-295X.84.2.191
» https://doi.org/10.1037/0033-295X.84.2.191 -
Bandura, A. (1982). Self-efficacy mechanism in human agency.American Psychologist, 37(2), 122. https://doi.org/10.1037/0003-066X.37.2.122
» https://doi.org/10.1037/0003-066X.37.2.122 - Bandura, A. (1997). Self-efficacy: The exercise of control. Freeman.
- Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model Lawrence Erlbaum.
-
Çetin, F., & Aşkun, D. (2018). The effect of occupational self-efficacy on work performance through intrinsic work motivation. Management Research Review, 41(2), 186-201. https://doi.org/10.1108/MRR-03-2017-0062
» https://doi.org/10.1108/MRR-03-2017-0062 -
Chalmers, R. P. (2012). Mirt: A multidimensional item response theory package for the R environment.Journal of Statistical Software,48, 1-29. https://www.jstatsoft.org/v48/i06
» https://www.jstatsoft.org/v48/i06 -
Chen, F. F. (2007). Sensitivity of goodness-of-fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464-504. https://doi.org/10.1080/10705510701301834
» https://doi.org/10.1080/10705510701301834 -
Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233-255. https://doi.org/10.1207/S15328007SEM0902_5
» https://doi.org/10.1207/S15328007SEM0902_5 -
Colledani, D., González Pizzio, A. P., Devita, M., & Anselmi, P. (2024). Investigating the functioning of rating scales with Rasch models. Assessment, 1(1). https://doi.org/10.1177/10731911241245792
» https://doi.org/10.1177/10731911241245792 -
Damásio, B. F., de Freitas, C. P. P., & Koller, S. H. (2014). Occupational Self-Efficacy Scale-Short Form (OSS-SF): Adaptation and evidence of construct validity of the Brazilian version.Revista Brasileira de Orientação Profissional,15(1), 65-73. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1679-33902014000100008&lng=pt&tlng=en
» http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1679-33902014000100008&lng=pt&tlng=en -
Duan, S., Bissaker, K. & Xu, Z. (2024). Correlates of teachers’ classroom management self-efficacy: A systematic review and meta-analysis. Educational Psychology Review, 36(43). https://doi.org/10.1007/s10648-024-09881-2
» https://doi.org/10.1007/s10648-024-09881-2 -
Freitas, C. P. P., Damásio, B. F., Haddad, E. J., & Koller, S. H. (2019). Work-related flow inventory: Evidence of validity of the Brazilian version. Paidéia (Ribeirão Preto), 29, 1-8. https://doi.org/10.1590/1982-4327e2901
» https://doi.org/10.1590/1982-4327e2901 -
Freitas, C. P. P., Damasio, B. F., & Koller, S. H. (2016). Occupational self-efficacy in interventions for populations in social vulnerability scale. Psicologia: Teoria e Pesquisa, 32(4), 1-9. https://doi.org/10.1590/0102.3772e324224
» https://doi.org/10.1590/0102.3772e324224 -
Lim, W. M. (2023). The workforce revolution: Reimagining work, workers, and workplaces for the future. Global Business and Organizational Excellence, 42(4), 5-10. https://doi.org/10.1002/joe.22218
» https://doi.org/10.1002/joe.22218 -
Lin, C.-Y., Broström, A., Griffiths, M. D., & Pakpour, A. H. (2019). Psychometric evaluation of the Persian eHealth Literacy Scale (eHEALS) among elder Iranians with heart failure. Evaluation & the Health Professions, 43(4), 222-229. https://doi.org/10.1177/0163278719827997
» https://doi.org/10.1177/0163278719827997 - Linacre J. M. (2015). A user's guide to Winsteps Ministep: Rasch-model computer programs
-
Luna, A. F., & Gondim, S. M. G. (2021). Autoeficácia ocupacional, fatores de risco psicossocial do trabalho e mal-estar físico e psicológico. Revista Psicologia e Saúde, 13(3), 51-64. https://doi.org/10.20435/pssa.v13i3.972
» https://doi.org/10.20435/pssa.v13i3.972 -
Mastroianni, K., & Storberg-Walker, J. (2014). Do work relationships matter? Characteristics of workplace interactions that enhance or detract from employee perceptions of well-being and health behaviors. Health Psychology and Behavioral Medicine, 2(1), 798-819. https://doi.org/10.1080/21642850.2014.933343
» https://doi.org/10.1080/21642850.2014.933343 -
Merino-Soto, César, Lima-Mendoza, Sadith, Lozano-Huamán, Milagros, Calderón de la Cruz, Gustavo, & Juárez-García, Arturo. (2021). Escala de Autoeficácia Ocupacional - Breve (OSES): Exploración de sus propiedades psicométricas. Revista de la Asociación Española de Especialistas en Medicina del Trabajo, 30(2), 195-207. http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S3020-11602021000200195&lng=es&tlng=es
» http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S3020-11602021000200195&lng=es&tlng=es -
Mielniczuk, E., & Laguna, M. (2018). Positive affect mediates the relationship between self-efficacy and innovative behavior in entrepreneurs. Journal of Creative Behavior, 1-12. https://doi.org/10.1002/jocb.364
» https://doi.org/10.1002/jocb.364 -
Monteiro, E., & Joseph, J. (2023). A review on the impact of workplace culture on employee mental health and well-being. International Journal of Case Studies in Business, IT and Education (IJCSBE), 7(2), 291-317. https://doi.org/10.47992/IJCSBE.2581.6942.0274
» https://doi.org/10.47992/IJCSBE.2581.6942.0274 -
Mugiono, M., Purwoko, B. H., Soelton, M., & Yuvitasari, R. (2020). Recognizing how the job involvement, burnout, and self-efficacy work influences the work stress at international hospitals. In 4th International Conference on Management, Economics and Business (pp. 264-270). Atlantis Press. https://doi.org/ 10.2991/aebmr.k.200205.046
» https://doi.org/ 10.2991/aebmr.k.200205.046 - Muthén, L. K., & Muthén, B. O. (2017). Mplus: Statistical analysis with latent variables (Version 8) [User’s guide]. Muthén & Muthén.
-
Nunes, M. F. O. (2008). Funcionamento e desenvolvimento das crenças de autoeficácia: Uma revisão. Revista Brasileira de Orientação Profissional, 9(1), 29-42. https://www.redalyc.org/articulo.oa?id=203016830004
» https://www.redalyc.org/articulo.oa?id=203016830004 -
Nunes, M. F. O., & Noronha, A. P. P. (2011). Escala de autoeficácia para atividades ocupacionais: Estudo da estrutura interna e precisão.Avaliaçao Psicologica: Interamerican Journal of Psychological Assessment,10(1), 25-40. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712011000100004&lng=pt&tlng=pt
» http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712011000100004&lng=pt&tlng=pt -
Pajares, F. (2002). Overview of social cognitive theory and of self-efficacy. http://www.emory.edu/education/mfp/eff.html
» http://www.emory.edu/education/mfp/eff.html - Pajares, F., & Olaz, F. (2008). Teoria social cognitiva e autoeficácia: uma visão geral. In A. Bandura, R. G. Azzi, & &. S. Polydoro (Orgs.), Teoria social cognitiva: Conceitos básicos (pp. 97-114). Artmed.
-
Pasquali, L., & Primi, R. (2003). Fundamentos da teoria da resposta ao item: TRI.Avaliação Psicológica: Interamerican Journal of Psychological Assessment, 2(2), 99-110. http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712003000200002&lng=pt&tlng=pt
» http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712003000200002&lng=pt&tlng=pt -
Peixoto, E. M., Silva, M. V., Noronha, A. P. P., Chichekian, T., & Vallerand, R. J. (2023). Psychometric properties and measurement invariance of the Rigid and Flexible Persistence Scale in a Brazilian professional sample. Psicologia: Reflexão e Crítica, 36, 1-11. https://doi.org/10.1186/s41155-022-00240-0
» https://doi.org/10.1186/s41155-022-00240-0 -
Peng, J., Zhang, J., Zhou, X., Wan, Z., Yuan, W., Gui, J., & Zhu, X. (2021). Validation of the occupational self-efficacy scale in a sample of Chinese employees. Frontiers in Psychology, 12, 1-9. https://doi.org/10.3389/fpsyg.2021.755134
» https://doi.org/10.3389/fpsyg.2021.755134 -
Pimenta de Devotto, R., Freitas, C. P. P., & Wechsler, S. M. (2020). The role of job crafting on the promotion of flow and well-being. RAM - Revista de Administração Mackenzie, 21(1). https://doi.org/10.1590/1678-6971/eRAMD200113
» https://doi.org/10.1590/1678-6971/eRAMD200113 -
R Core Team. (2022). R: A language and environment for statistical computing R Foundation for Statistical Computing. https://www.R-project.org/
» https://www.R-project.org/ -
Reyhing, Y., & Perren, S. (2023). The situation matters! The effects of educator self-efficacy on interaction quality in child care. Journal of Research in Childhood Education, 37(4), 582-597. https://doi.org/10.1080/02568543.2022.2161678
» https://doi.org/10.1080/02568543.2022.2161678 -
Rigotti, T., Schyns, B., & Mohr, G. (2008). A short version of the occupational self-efficacy scale: Structural and construct validity across five countries. Journal of Career Assessment, 16(2), 238-255. https://doi.org/10.1177/1069072707305763
» https://doi.org/10.1177/1069072707305763 -
Salvador, A. P. V., & Ambiel, R. A. M. (2019). Adaptabilidade de carreira e autoeficácia ocupacional: Relações com variáveis de carreira.Avaliação Psicológica,18(3), 256-263. https://doi.org/ 0.15689/ap.2019.1803.16853.05
» https://doi.org/ 0.15689/ap.2019.1803.16853.05 -
Sass, D. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29(4), 347-363. http://doi.org/10.1177/0734282911406661
» http://doi.org/10.1177/0734282911406661 -
Schaufeli, W. B. (2017). Applying the job demands-resources model.Organizational Dynamics, 2(46), 120-132. https://doi.org/ 10.1016/j.orgdyn.2017.04.008
» https://doi.org/ 10.1016/j.orgdyn.2017.04.008 -
Schyns, B., & von Collani, G. (2002). A new occupational self-efficacy scale and its relation to personality constructs and organizational variables. European Journal of Work and Organizational Psychology, 11(2), 219-241. https://doi.org/10.1080/13594320244000148
» https://doi.org/10.1080/13594320244000148 -
Shoji, K., Cieslak, R., Smoktunowicz, E., Rogala, A., Benight, C. C., & Luszczynska, A. (2016). Associations between job burnout and self-efficacy: A meta-analysis. Anxiety, Stress, & Coping, 29(4), 367-386. https://doi.org/10.1080/10615806.2015.1058369
» https://doi.org/10.1080/10615806.2015.1058369 -
Simms, L. J., Zelazny, K., Williams, T. F., & Bernstein, L. (2019). Does the number of response options matter? Psychometric perspectives using personality questionnaire data.Psychological assessment,31(4), 557-566. https://doi.org/10.1037/pas0000648
» https://doi.org/10.1037/pas0000648 -
Timmerman, M. E. & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209-20. https://doi.org/10.1037/a0023353
» https://doi.org/10.1037/a0023353 -
Volmer, J., Spurk, D., Orth, M., & Göritz, A. (2023). Reciprocal effects of career adaptability and occupational self-efficacy: A three-wave longitudinal study with varying time lags. Journal of Career Assessment, 31(4), 665-685. https://doi.org/10.1177/10690727221140050
» https://doi.org/10.1177/10690727221140050
Research data is available on request from the corresponding author.
Note. OSE = Occupational Self-Efficacy
Note. ϴ = latent trait, P = response categories, P(ϴ) = probability of the response categories
Note. ϴ = latent trait, SE = standard error, I(ϴ) = test information function, SE(ϴ) = test standard error function