Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire
Job performance is considered the “ultimate dependent variable” in human resource management, turning its assessment into a capital issue. The present study analyzes the functioning of a brief 18-item self-report scale, the Individual Work Performance Questionnaire (IWPQ), which measures the main di...
| Autores: | , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2019 |
| País: | España |
| Institución: | Colegio Oficial de Psicólogos de Madrid |
| Repositorio: | Journal of Work and Organizational Psychology |
| OAI Identifier: | oai:journals.copmadrid.org:jwop/art/jwop2019a21 |
| Acceso en línea: | https://doi.org/10.5093/jwop2019a21 |
| Access Level: | acceso abierto |
| Palabra clave: | Job performance, Task performance, Contextual performance, Counterproductive work behaviors, Adaptation, Brief self-report scale. Desempeño laboral, Desempeño de tarea, Desempeño contextual, Conductas contraproductivas en el trabajo, Adaptación, Escala autoinformada breve. |
| id |
ES_e8c8cd4d05937ed43ba71c45ab77cb7d |
|---|---|
| oai_identifier_str |
oai:journals.copmadrid.org:jwop/art/jwop2019a21 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| dc.title.none.fl_str_mv |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire La evaluación del desempeño en el trabajo con escalas de autoinforme breves: el cuestionario de desempeño laboral individual |
| title |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire |
| spellingShingle |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire Fernández-del-Río, Elena Job performance, Task performance, Contextual performance, Counterproductive work behaviors, Adaptation, Brief self-report scale. Desempeño laboral, Desempeño de tarea, Desempeño contextual, Conductas contraproductivas en el trabajo, Adaptación, Escala autoinformada breve. |
| title_short |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire |
| title_full |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire |
| title_fullStr |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire |
| title_full_unstemmed |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire |
| title_sort |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance Questionnaire |
| dc.creator.none.fl_str_mv |
Fernández-del-Río, Elena Koopmans, Linda |
| author |
Fernández-del-Río, Elena |
| author_facet |
Fernández-del-Río, Elena Koopmans, Linda |
| author_role |
author |
| author2 |
Koopmans, Linda |
| author2_role |
author |
| dc.subject.none.fl_str_mv |
Job performance, Task performance, Contextual performance, Counterproductive work behaviors, Adaptation, Brief self-report scale. Desempeño laboral, Desempeño de tarea, Desempeño contextual, Conductas contraproductivas en el trabajo, Adaptación, Escala autoinformada breve. |
| topic |
Job performance, Task performance, Contextual performance, Counterproductive work behaviors, Adaptation, Brief self-report scale. Desempeño laboral, Desempeño de tarea, Desempeño contextual, Conductas contraproductivas en el trabajo, Adaptación, Escala autoinformada breve. |
| description |
Job performance is considered the “ultimate dependent variable” in human resource management, turning its assessment into a capital issue. The present study analyzes the functioning of a brief 18-item self-report scale, the Individual Work Performance Questionnaire (IWPQ), which measures the main dimensions of job performance (task performance, contextual performance, and counterproductive behaviors) in a wide variety of jobs. Participants were 368 employees who voluntarily answered a questionnaire including the IWPQ, other performance scales, and the NEO-FFI. Descriptive statistics, exploratory structural equation modeling, and correlations were performed. Results show that the IWPQ has a tridimensional structure with adequate reliability, exhibits significant associations with other measures of performance, and its association with personality traits is similar in terms of direction and strength of the correlations between other job performance measures and personality. We conclude that the IWPQ is an adequate measure of job performance but with emphasis on behaviors aimed toward organizations. |
| publishDate |
2019 |
| dc.date.none.fl_str_mv |
2019 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/published |
| format |
article |
| dc.identifier.none.fl_str_mv |
https://doi.org/10.5093/jwop2019a21 |
| url |
https://doi.org/10.5093/jwop2019a21 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.relation.none.fl_str_mv |
/*ref*/S. Adler, M. Campion, A. Colquitt, A. Grubb, K. Murphy, R. Ollander-Krane, E. D. Pulakos(2016) Getting rid of performance ratings: Genius or folly? A debate. Industrial and Organizational Psychology 9(), 219-252 DOI: https://doi.org/10.1017/iop.2015.106 /*ref*/L. R. Aiken(1980) Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement 40(), 955-959 DOI: https://doi.org/10.1177/001316448004000419 /*ref*/H. Aguinis(2013) . Performance managementPearson Prentice Hall (), DOI: https://doi.org/10.1177/001316448004000419 /*ref*/P. Alonso, S. Moscoso, D. Cuadrado(2015) Procedimientos de selección de personal en pequeñas y medianas empresas españolas [Personnel selection procedures in Spanish small and medium size organizations]. Journal of Work & Organizational Psychology 31(), 79-89 DOI: https://doi.org/10.1016/j.rpto.2015.04.002 /*ref*/T. Asparouhov, B. Muthen(2009) Exploratory structural equation modeling. Structural Equation Modeling 16(), 397-438 DOI: https://doi.org/10.1080/10705510903008204 /*ref*/C. Aubé, V. Rousseau, C. Mama, E. M. Morin(2009) Counterproductive behaviors and psychological well-being: The moderating effect of task interdependence. Journal of Business and Psychology 24(), 351-361 DOI: https://doi.org/10.1007/s10869-009-9113-5 /*ref*/M. A. Baloch, F. Meng, Z. Xu, Xu Z. Cepeda-Carrion, I. Danish, M. W. Bari(2017) Dark Triad, perceptions of organizational politics and counterproductive work behaviors: The moderating effect of political skills.. Frontiers in Psychology 8(), DOI: https://doi.org/10.3389/fpsyg.2017.01972 /*ref*/J. R. Barrada, Á. Castro, A. B. Correa, P. Ruiz-Gómez(2018) he tridimensional structure of sociosexuality: Spanish validation of the Revised Sociosexual Orientation Inventory. Journal of Sex & Marital Therapy 44(), 149-158 DOI: https://doi.org/10.1080/0092623X.2017.1335665 /*ref*/J. R. Barrada, J. F. Navas, C. M. Ruiz de Lara, J. Billieux, G. Devos, J. C. Perales(2019) Reconsidering the roots, structure, and implications of gambling motives: An integrative approach. PLoS ONE 14(2), DOI: https://doi.org/10.1371/journal.pone.0212695 /*ref*/M. R. Barrick, M. K. Mount(1991) The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology 44(), 1-26 DOI: https://doi.org/10.1111/j.1744-6570.1991.tb00688.x /*ref*/R. J. Bennett, S. L. Robinson(2000) Development of a measure of workplace deviance. Journal of Applied Psychology 85(), 349-360 DOI: https://doi.org/10.1037/0021-9010.85.3.349 /*ref*/C. M. Berry, D. S. Ones, P. R. Sackett(2007) Interpersonal deviance, organizational deviance, and their common correlates: A review and meta-analysis. Journal of Applied Psychology 92(), 410-424 DOI: https://doi.org/10.1037/0021-9010.92.2.410 /*ref*/A. J. Bishara, J. B. Hittner(2012) Testing the significance of a correlation with nonnormal data: Comparison of Pearson, Spearman, transformation, and resampling approaches. Psychological Methods 17(), 399-417 DOI: https://doi.org/10.1037/a0028087 /*ref*/A. J. Bishara, J. B. Hittner(2015) Reducing bias and error in the correlation coefficient due to nonnormality. Educational and Psychological Measurement 75(), 785-804 DOI: https://doi.org/10.1177/0013164414557639 /*ref*/W. C. Borman, L. A. Penner, T. D. Allen, S. J. Motowidlo(2001) Personality predictors of citizenship performance. International Journal of Selection and Assessment 9(), 52-69 DOI: https://doi.org/10.1177/0013164414557639 /*ref*/C. B. Bragg, N. A. Bowling(2018) Not all forms of misbehavior are created equal: Differential personality facet-counterproductive work behavior relations. International Journal of Selection and Assessment 26(), 27-35 DOI: https://doi.org/10.1111/ijsa.12200 /*ref*/T. A. Brown(2006) . Confirmatory factor analysis for applied researchGuilford Press (), DOI: https://doi.org/10.1111/ijsa.12200 /*ref*/M. J. Burke, A. S. Sarpy, P. E. Tesluk, K. Smith-Crowe(2002) General safety performance: A test of a grounded theoretical model. Personnel Psychology 55(), 429-457 DOI: https://doi.org/10.1111/j.1744-6570.2002.tb00116.x /*ref*/J. P. Campbell, B. M. Wiernik(2015) The modeling and assessment of work performance. Annual Review of Organizational Psychology and Organizational Behavior 2(), 47-74 DOI: https://doi.org/10.1146/annurev-orgpsych-032414-111427 /*ref*/V. S. Carlos, R. Gouveia(2016) Development and validation of a self-reported measure of job performance. Social Indicators Research 126(), 279-307 DOI: https://doi.org/10.1007/s11205-015-0883-z /*ref*/D. S. Chiaburu, I.-S. Oh, C. M. Berry, N. Li, R. G. Gardner(2011) The five-factor model of personality traits and organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology 96(), 1140-1166 DOI: https://doi.org/10.1037/a0024004 /*ref*/J. Cohen(1992) A power primer. Psychological Bulletin 112(), 155-159 DOI: https://doi.org/10.1037/0033-2909.112.1.155 /*ref*/P. Costa, R. R. McCrae(2008) . Inventario de personalidad NEO revisado (NEO PI-R). Inventario NEO reducido de cinco Factores (NEO- FFI)TEA (), DOI: https://doi.org/10.1037/0033-2909.112.1.155 /*ref*/L. J. Cronbach, C. G. Gleser(1957) . Psychological tests and personnel decisionsUniversity of Illinois Press (), DOI: https://doi.org/10.1037/0033-2909.112.1.155 /*ref*/R. S. Dalal(2005) A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology 90(), 1241-1255 DOI: https://doi.org/10.1037/0021-9010.90.6.1241 /*ref*/R. S. Dalal, M. Baysinger, B. J. Brummel, J. M. Lebreton(2012) The relative importance of employee engagement, other job attitudes, and trait affect as predictors of job performance. Journal of Applied Social Psychology 41(), 295-325 DOI: https://doi.org/10.1111/j.1559-1816.2012.01017.x /*ref*/M. C. Dávila, M. A. Finkelstein(2010) Predicting organizational citizenship behavior from the functional analysis and role identity perspectives: Further evidence in Spanish employees. The Spanish Journal of Psychology 13(), 277-283 DOI: https://doi.org/10.1017/S1138741600003851 /*ref*/A. S. DeNisi, K. R. Murphy(2017) Performance appraisal and performance management: 100 years of progress?. Journal of Applied Psychology 102(), 421-433 DOI: https://doi.org/10.1037/apl0000085 /*ref*/L. Díaz-Vilela, D. Díaz-Cabrera, R. Isla-Díaz, E. Hernández-Fernaud, C. Rosales-Sánchez(2012) Adaptación al español de la escala de desempeño cívico de Coleman y Borman (2000) y análisis de la estructura empírica del constructo [Spanish Adaptation of the Citizenship Performance Questionnaire by Coleman & Borman (2000) and an analysis of the empiric structure of the construct]. Journal of Work and Organizational Psychcology 28(), 135-149 DOI: https://doi.org/10.5093/tr2012a11 /*ref*/L. R. Fabrigar, D. T. Wegener, R. C. MacCallum, E. J. Strahan(1999) Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods 4(), 272-299 DOI: https://doi.org/10.1037/1082-989X.4.3.272 /*ref*/E. Fernández del Río, J. R. Barrada, P. J. Ramos-Villagrasa(2018) Bad behaviors at work: Spanish adaptation of the workplace deviance scale. Current Psychology (), DOI: https://doi.org/10.1037/1082-989X.4.3.272 /*ref*/S. Fine, M. Edward(2017) Breaking the rules, not the law: The potential risks of counterproductive work behaviors among overqualified employees. International Journal of Selection and Assessment 25(), 401-405 DOI: https://doi.org/10.1111/ijsa.12194 /*ref*/C. Fritz, S. Sonnentag(2006) Recovery, well-being, and performance-related outcomes: The role of workload and vacation experiences. Journal of Applied Psychology 91(), 936-945 DOI: https://doi.org/10.1037/0021-9010.91.4.936 /*ref*/L. E. Garrido, J. R. Barrada, J. A. Aguasvivas, A. Martínez-Molina, V. B. Arias, H. F. Golino, L. Rojo-Moreno(2018) Is small still beautiful for the Strengths and Difficulties Questionnaire? Novel findings using exploratory structural equation modeling. Assessment (), DOI: https://doi.org/10.1177/1073191118780461 /*ref*/M. J. Gorgievski, A. B. Bakker, W. B. Schaufeli(2010) Work engagement and workaholism: comparing the self-employed and salaried employees. The Journal of Positive Psychology 5(), 83-96 DOI: https://doi.org/10.1080/17439760903509606 /*ref*/B. J. Hoffman, C. A. Blair, J. P. Meriac, D. J. Woehr(2007) Expanding the criterion domain? A quantitative review of the OCB literature. Journal of Applied Psychology 92(), 555-566 DOI: https://doi.org/10.1037/0021-9010.92.2.555 /*ref*/L. Hu, P. M. Bentler(1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6(), 1-55 DOI: https://doi.org/10.1080/10705519909540118 /*ref*/G. M. Hurtz, J. J. Donovan(2000) Personality and job performance: The Big Five revisited. Journal of Applied Psychology 85(), 869-879 DOI: https://doi.org/10.1037//0021-9010.85.6.869 /*ref*/T. A. Judge, J. D. Kammeyer-Mueller(2012) General and specific measures in organizational behavior research: Considerations, examples, and recommendations for researchers. Journal of Organizational Behavior 33(), 161-174 DOI: https://doi.org/10.1002/job.764 /*ref*/L. Koopmans(2015) . Individual Work Performance Questionnaire instruction manualTNO Innovation for Life – VU University Medical Center (), DOI: https://doi.org/10.1002/job.764 /*ref*/L. Koopmans, C. M. Bernaards, V. H. Hildebrandt, H. C. W. De Vet, A. J. van der Beek(2013) Measuring individual work performance: Identifying and selecting indicators. Work 48(), 229-238 DOI: https://doi.org/10.3233/WOR-131659 /*ref*/L. Koopmans, C. M. Bernaards, V. H. Hildebrandt, H. C. W. De Vet, A. J. Van Der Beek(2014) Construct validity of the Individual Work Performance Questionnaire. Journal of Occupational and Environmental Medicine 56(), 331-337 DOI: https://doi.org/10.1097/JOM.0000000000000113 /*ref*/L. Koopmans, C. M. Bernaards(2016) Cross-cultural adaptation of the Individual Work Performance Questionnaire. Work (), 609-619 DOI: https://doi.org/10.3233/WOR-152237 /*ref*/L. Koopmans, C. M. Bernaards, V. H. Hildebrandt, W. B. Schaufeli, C. W. de Vet Henrica, A. J. van der Beek(2011) Conceptual frameworks of individual work performance. Journal of Occupational and Environmental Medicine 53(), 856-866 DOI: https://doi.org/10.1097/JOM.0b013e318226a763 /*ref*/L. Koopmans, C. Bernaards, V. Hildebrandt, S. van Buuren, A. J. van der Beek, H. C. W. de Vet(2013) Development of an individual work performance questionnaire. International Journal of Productivity and Performance Management 62(), 6-28 DOI: https://doi.org/10.1108/17410401311285273 /*ref*/C. H. Lawshe(1975) A quantitative approach to content validity. Personnel Psychology 28(), 563-575 DOI: https://doi.org/10.1111/j.1744-6570.1975.tb01393.x /*ref*/K. Lee, N. J. Allen(2002) Organizational citizenship behavior and workplace deviance: The role of affect and cognitions. The Journal of Applied Psychology 87(), 131-142 DOI: https://doi.org/10.1037/0021-9010.87.1.131 /*ref*/J. A. Lepine, A. Erez, D. E. Johnson(2002) The nature and dimensionality of organizational citizenship behavior: A critical review and meta-analysis. The Journal of Applied Psychology 87(), 52-65 DOI: https://doi.org/10.1037/0021-9010.87.1.52 /*ref*/S. B MacKenzie, P. M. Podsakoff, R. Fetter(1991) Organizational citizenship behavior and objective productivity as determinants of managerial evaluations of salespersons’ performance. Organizational Behavior and Human Decision Processes 50(), 123-150 DOI: https://doi.org/10.1016/0749-5978(91)90037-T /*ref*/J. Meedovi, B. Petrovi(2015) The Dark Tetrad: Structural properties and location in the personality space. Journal of Individual Differences 36(), 228-236 DOI: https://doi.org/10.1027/1614-0001/a000179 /*ref*/M. Morf, A. Feierabend, B. Staffelbach(2017) Task variety and counterproductive work behavior. Journal of Managerial Psychology 32(), 581-592 DOI: https://doi.org/10.1108/JMP-02-2017-0048 /*ref*/J. Muñiz, P. Elosua, R. K. Hambleton(2013) Directrices para la traducción y adaptación de los test: segunda edición [International Test Commission guidelines for test translation and adaptation: Second edition]. Psicothema 25(), 151-157 DOI: https://doi.org/10.7334/psicothema2013.24 /*ref*/K. R. Murphy(2008) Explaining the weak relationship between job performance and ratings of job performance. Industrial and Organizational Psychology (), DOI: https://doi.org/10.1111/j.1754-9434.2008.00030.x /*ref*/L. K. Muthén, B. O. Muthén(2015) . Mplus user’s guide. Seventh editionMuthén & Muthén (), DOI: https://doi.org/10.1111/j.1754-9434.2008.00030.x /*ref*/G. Navarro-Carrillo, A. M. Beltrán-Morillas, I. Valor-Segura, F. Expósito(2017) What is behind envy? Approach from a psychosocial perspective [¿Qué se esconde detrás de la envidia? Aproximación desde una perspectiva psicosocial]. Revista de Psicología Social 32(), 217-245 DOI: https://doi.org/10.1080/02134748.2017.1297354 /*ref*/D. W. Organ, J. B. Paine(1999) . International review of industrial and organizational psychologyJohn Wiley & Sons Ltd. 14(), 337-368 DOI: https://doi.org/10.1080/02134748.2017.1297354 /*ref*/N. P. Podsakoff, S. W. Whiting, P. M. Podsakoff, B. D. Blume(2009) Individual- and organizational-level consequences of organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology 94(), 122-141 DOI: https://doi.org/10.1037/a0013079 /*ref*/E. D. Pulakos, S. Arad, M. A. Donovan, K. E. Plamondon(2000) Adaptability in the workplace: Development of a taxonomy of adaptive performance. Journal of Applied Psychology 85(), 612-624 DOI: https://doi.org/10.1037/0021-9010.85.4.612 /*ref*/(2019) . R: A language and environment for statistical computingR Foundation for Statistical Computing (), DOI: https://doi.org/10.1037/0021-9010.85.4.612 /*ref*/U. Rehman, M. G. Shahnawaz(2018) Machiavellianism, job autonomy, and counterproductive work behaviour among Indian managers. Journal of Work and Organizational Psychology 34(), 83-88 DOI: https://doi.org/10.5093/jwop2018a10 /*ref*/M. Rhemtulla, P. É. Brosseau-Liard, V. Savalei(2012) When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods 17(), 354-373 DOI: https://doi.org/10.1037/a0029315 /*ref*/S. L. Robinson, R. J. Bennett(1995) A typology of deviant workplace behaviors: A multidimensional scaling study. Academy of Management Journal 38(), 555-572 DOI: https://doi.org/10.2307/256693 /*ref*/K.-A. Rogers, E. K. Kelloway(1997) Violence at work: Personal and organizational outcomes. Journal of Occupational Health Psychology 2(), 63-71 DOI: https://doi.org/10.1037/1076-8998.2.1.63 /*ref*/M. Rotundo, P. R. Sackett(2002) The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach. Journal of Applied Psychology 87(), 66-80 DOI: https://doi.org/10.1037//0021-9010.87.1.66 /*ref*/P. R. Sackett(2002) The structure of counterproductive work behaviors: Dimensionality and relationships with facets of job performance. International Journal of Selection and Assessment 10(), 5-11 DOI: https://doi.org/10.1111/1468-2389.00189 /*ref*/P. R. Sackett, C. J. DeVore(2001) . Handbook of industrial, work, and organizational psychologySAGE Publications (), DOI: https://doi.org/10.4135/9781848608320.n9 /*ref*/P. R. Sackett, F. Lievens(2008) Personnel selection. Annual Review of Psychology 59(), 419-450 DOI: https://doi.org/10.1146/annurev.psych.59.103006.093716 /*ref*/J. F. Salgado(2002) The Big Five personality dimensions and counterproductive behaviors. International Journal of Selection and Assessment 10(), 117-125 DOI: https://doi.org/10.1111/1468-2389.00198 /*ref*/J. F. Salgado(2003) Predicting job performance using FFM and non-FFM personality measures. Journal of Occupational and Organizational Psychology 76(), 323-346 DOI: https://doi.org/10.1348/096317903769647201 /*ref*/J. F. Salgado, Á. L. Cabal(2011) Evaluación del desempeño en la administración pública del Principado de Asturias: Análisis de las propiedades psicométricas [Performance appraisal in the public administration of the Principality of Asturias: An analysis of psychometric properties]. Journal of Work and Organizational Psychology 27(), 75-91 DOI: https://doi.org/10.1348/096317903769647201 /*ref*/J. F. Salgado, S. Moscoso, J. I. Sanchez, P. Alonso, B. Choragwicka, A. Berges(2015) Validity of the five-factor model and their facets: The impact of performance measure and facet residualization on the bandwidth-fidelity dilemma. European Journal of Work and Organizational Psychology 24(), 325-349 DOI: https://doi.org/10.1080/1359432X.2014.903241 /*ref*/D. Sánchez-Carracedo, J. R. Barrada, G. López-Guimerà, J. Fauquet, C. A. Almenara, E. Trepat(2012) Analysis of the factor structure of the Sociocultural Attitudes Towards Appearance Questionnaire (SATAQ-3) in Spanish secondary-school students through exploratory structural equation modeling. Body Image 9(), 163-171 DOI: https://doi.org/10.1016/j.bodyim.2011.10.002 /*ref*/S. E. Scullen, M. K. Mount, M. Goff(2000) Understanding the latent structure of job performance ratings. Journal of Applied Psychology 85(), 956-970 DOI: https://doi.org/10.1037/0021-9010.85.6.956 /*ref*/E. Selenko, A. Mäkikangas, S. Mauno, U. Kinnunen(2013) How does job insecurity relate to self-reported job performance? Analysing curvilinear associations in a longitudinal sample. Journal of Occupational and Organizational Psychology 86(), 522-542 DOI: https://doi.org/10.1111/joop.12020 /*ref*/C. Viswesvaran, D. S. Ones(2017) . The Blackwell handbook of personnel selectionWiley (), 354-375 DOI: https://doi.org/10.1002/9781405164221.ch16 /*ref*/J. M. Werner(1994) Dimensions that make a difference: Examining the impact of in-role and extrarole behaviors on supervisory ratings. Journal of Applied Psychology 79(), 98-107 DOI: https://doi.org/10.1037/0021-9010.79.1.98 /*ref*/D. J. Woehr(2008) On the relationship between job performance and ratings of job performance: What do we really know?. Industrial and Organizational Psychology 1(), 161-166 DOI: https://doi.org/10.1111/j.1754-9434.2008.00031.x |
| dc.rights.none.fl_str_mv |
https://creativecommons.org/licenses/by-nc-nd/4.0/deed.es_ES Copyright © 2026. Colegio Oficial de Psicólogos de Madrid info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-nd/4.0/deed.es_ES Copyright © 2026. Colegio Oficial de Psicólogos de Madrid |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf text/html application/xml |
| dc.publisher.none.fl_str_mv |
Colegio Oficial de Psicologos de Madrid |
| publisher.none.fl_str_mv |
Colegio Oficial de Psicologos de Madrid |
| dc.source.none.fl_str_mv |
reponame:Journal of Work and Organizational Psychology instname:Colegio Oficial de Psicólogos de Madrid |
| instname_str |
Colegio Oficial de Psicólogos de Madrid |
| reponame_str |
Journal of Work and Organizational Psychology |
| collection |
Journal of Work and Organizational Psychology |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869422971798421504 |
| spelling |
Assessing Job Performance Using Brief Self-report Scales: The Case of the Individual Work Performance QuestionnaireLa evaluación del desempeño en el trabajo con escalas de autoinforme breves: el cuestionario de desempeño laboral individualFernández-del-Río, ElenaKoopmans, LindaJob performance, Task performance, Contextual performance, Counterproductive work behaviors, Adaptation, Brief self-report scale.Desempeño laboral, Desempeño de tarea, Desempeño contextual, Conductas contraproductivas en el trabajo, Adaptación, Escala autoinformada breve.Job performance is considered the “ultimate dependent variable” in human resource management, turning its assessment into a capital issue. The present study analyzes the functioning of a brief 18-item self-report scale, the Individual Work Performance Questionnaire (IWPQ), which measures the main dimensions of job performance (task performance, contextual performance, and counterproductive behaviors) in a wide variety of jobs. Participants were 368 employees who voluntarily answered a questionnaire including the IWPQ, other performance scales, and the NEO-FFI. Descriptive statistics, exploratory structural equation modeling, and correlations were performed. Results show that the IWPQ has a tridimensional structure with adequate reliability, exhibits significant associations with other measures of performance, and its association with personality traits is similar in terms of direction and strength of the correlations between other job performance measures and personality. We conclude that the IWPQ is an adequate measure of job performance but with emphasis on behaviors aimed toward organizations.El desempeño laboral es considerado la “variable dependiente definitiva” en recursos humanos, convirtiendo su evaluación en algo crucial. El presente estudio analiza el funcionamiento de una escala autoinformada breve de 18 ítems, el Individual Work Performance Questionnaire (IWPQ), que mide las principales dimensiones del desempeño laboral (desempeño de tarea, desempeño contextual y comportamientos contraproductivos en el trabajo) en una amplia variedad de trabajos. Los participantes fueron 368 empleados que voluntariamente completaron un cuestionario que incluía el IWPQ, otras escalas de desempeño y el NEO-FFI. Se llevaron a cabo estadísticos descriptivos, modelos exploratorios de ecuaciones estructurales y correlaciones. Los resultados muestran que el IWPQ tiene una estructura tridimensional con una fiabilidad adecuada, mostrando asociaciones significativas con el resto de medidas de desempeño. En cuanto a los factores de personalidad, el IWPQ muestra correlaciones similares a las de los otros instrumentos de desempeño analizados. Se concluye que el IWPQ es un instrumento adecuado para medir de manera breve y autoinformada el desempeño laboral, pero con énfasis en los comportamientos dirigidos hacia la organización.Colegio Oficial de Psicologos de Madrid2019info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedapplication/pdftext/htmlapplication/xmlhttps://doi.org/10.5093/jwop2019a21reponame:Journal of Work and Organizational Psychologyinstname:Colegio Oficial de Psicólogos de MadridInglés/*ref*/S. Adler, M. Campion, A. Colquitt, A. Grubb, K. Murphy, R. Ollander-Krane, E. D. Pulakos(2016) Getting rid of performance ratings: Genius or folly? A debate. Industrial and Organizational Psychology 9(), 219-252 DOI: https://doi.org/10.1017/iop.2015.106/*ref*/L. R. Aiken(1980) Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement 40(), 955-959 DOI: https://doi.org/10.1177/001316448004000419/*ref*/H. Aguinis(2013) . Performance managementPearson Prentice Hall (), DOI: https://doi.org/10.1177/001316448004000419/*ref*/P. Alonso, S. Moscoso, D. Cuadrado(2015) Procedimientos de selección de personal en pequeñas y medianas empresas españolas [Personnel selection procedures in Spanish small and medium size organizations]. Journal of Work & Organizational Psychology 31(), 79-89 DOI: https://doi.org/10.1016/j.rpto.2015.04.002/*ref*/T. Asparouhov, B. Muthen(2009) Exploratory structural equation modeling. Structural Equation Modeling 16(), 397-438 DOI: https://doi.org/10.1080/10705510903008204/*ref*/C. Aubé, V. Rousseau, C. Mama, E. M. Morin(2009) Counterproductive behaviors and psychological well-being: The moderating effect of task interdependence. Journal of Business and Psychology 24(), 351-361 DOI: https://doi.org/10.1007/s10869-009-9113-5/*ref*/M. A. Baloch, F. Meng, Z. Xu, Xu Z. Cepeda-Carrion, I. Danish, M. W. Bari(2017) Dark Triad, perceptions of organizational politics and counterproductive work behaviors: The moderating effect of political skills.. Frontiers in Psychology 8(), DOI: https://doi.org/10.3389/fpsyg.2017.01972/*ref*/J. R. Barrada, Á. Castro, A. B. Correa, P. Ruiz-Gómez(2018) he tridimensional structure of sociosexuality: Spanish validation of the Revised Sociosexual Orientation Inventory. Journal of Sex & Marital Therapy 44(), 149-158 DOI: https://doi.org/10.1080/0092623X.2017.1335665/*ref*/J. R. Barrada, J. F. Navas, C. M. Ruiz de Lara, J. Billieux, G. Devos, J. C. Perales(2019) Reconsidering the roots, structure, and implications of gambling motives: An integrative approach. PLoS ONE 14(2), DOI: https://doi.org/10.1371/journal.pone.0212695/*ref*/M. R. Barrick, M. K. Mount(1991) The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology 44(), 1-26 DOI: https://doi.org/10.1111/j.1744-6570.1991.tb00688.x/*ref*/R. J. Bennett, S. L. Robinson(2000) Development of a measure of workplace deviance. Journal of Applied Psychology 85(), 349-360 DOI: https://doi.org/10.1037/0021-9010.85.3.349/*ref*/C. M. Berry, D. S. Ones, P. R. Sackett(2007) Interpersonal deviance, organizational deviance, and their common correlates: A review and meta-analysis. Journal of Applied Psychology 92(), 410-424 DOI: https://doi.org/10.1037/0021-9010.92.2.410/*ref*/A. J. Bishara, J. B. Hittner(2012) Testing the significance of a correlation with nonnormal data: Comparison of Pearson, Spearman, transformation, and resampling approaches. Psychological Methods 17(), 399-417 DOI: https://doi.org/10.1037/a0028087/*ref*/A. J. Bishara, J. B. Hittner(2015) Reducing bias and error in the correlation coefficient due to nonnormality. Educational and Psychological Measurement 75(), 785-804 DOI: https://doi.org/10.1177/0013164414557639/*ref*/W. C. Borman, L. A. Penner, T. D. Allen, S. J. Motowidlo(2001) Personality predictors of citizenship performance. International Journal of Selection and Assessment 9(), 52-69 DOI: https://doi.org/10.1177/0013164414557639/*ref*/C. B. Bragg, N. A. Bowling(2018) Not all forms of misbehavior are created equal: Differential personality facet-counterproductive work behavior relations. International Journal of Selection and Assessment 26(), 27-35 DOI: https://doi.org/10.1111/ijsa.12200/*ref*/T. A. Brown(2006) . Confirmatory factor analysis for applied researchGuilford Press (), DOI: https://doi.org/10.1111/ijsa.12200/*ref*/M. J. Burke, A. S. Sarpy, P. E. Tesluk, K. Smith-Crowe(2002) General safety performance: A test of a grounded theoretical model. Personnel Psychology 55(), 429-457 DOI: https://doi.org/10.1111/j.1744-6570.2002.tb00116.x/*ref*/J. P. Campbell, B. M. Wiernik(2015) The modeling and assessment of work performance. Annual Review of Organizational Psychology and Organizational Behavior 2(), 47-74 DOI: https://doi.org/10.1146/annurev-orgpsych-032414-111427/*ref*/V. S. Carlos, R. Gouveia(2016) Development and validation of a self-reported measure of job performance. Social Indicators Research 126(), 279-307 DOI: https://doi.org/10.1007/s11205-015-0883-z/*ref*/D. S. Chiaburu, I.-S. Oh, C. M. Berry, N. Li, R. G. Gardner(2011) The five-factor model of personality traits and organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology 96(), 1140-1166 DOI: https://doi.org/10.1037/a0024004/*ref*/J. Cohen(1992) A power primer. Psychological Bulletin 112(), 155-159 DOI: https://doi.org/10.1037/0033-2909.112.1.155/*ref*/P. Costa, R. R. McCrae(2008) . Inventario de personalidad NEO revisado (NEO PI-R). Inventario NEO reducido de cinco Factores (NEO- FFI)TEA (), DOI: https://doi.org/10.1037/0033-2909.112.1.155/*ref*/L. J. Cronbach, C. G. Gleser(1957) . Psychological tests and personnel decisionsUniversity of Illinois Press (), DOI: https://doi.org/10.1037/0033-2909.112.1.155/*ref*/R. S. Dalal(2005) A meta-analysis of the relationship between organizational citizenship behavior and counterproductive work behavior. Journal of Applied Psychology 90(), 1241-1255 DOI: https://doi.org/10.1037/0021-9010.90.6.1241/*ref*/R. S. Dalal, M. Baysinger, B. J. Brummel, J. M. Lebreton(2012) The relative importance of employee engagement, other job attitudes, and trait affect as predictors of job performance. Journal of Applied Social Psychology 41(), 295-325 DOI: https://doi.org/10.1111/j.1559-1816.2012.01017.x/*ref*/M. C. Dávila, M. A. Finkelstein(2010) Predicting organizational citizenship behavior from the functional analysis and role identity perspectives: Further evidence in Spanish employees. The Spanish Journal of Psychology 13(), 277-283 DOI: https://doi.org/10.1017/S1138741600003851/*ref*/A. S. DeNisi, K. R. Murphy(2017) Performance appraisal and performance management: 100 years of progress?. Journal of Applied Psychology 102(), 421-433 DOI: https://doi.org/10.1037/apl0000085/*ref*/L. Díaz-Vilela, D. Díaz-Cabrera, R. Isla-Díaz, E. Hernández-Fernaud, C. Rosales-Sánchez(2012) Adaptación al español de la escala de desempeño cívico de Coleman y Borman (2000) y análisis de la estructura empírica del constructo [Spanish Adaptation of the Citizenship Performance Questionnaire by Coleman & Borman (2000) and an analysis of the empiric structure of the construct]. Journal of Work and Organizational Psychcology 28(), 135-149 DOI: https://doi.org/10.5093/tr2012a11/*ref*/L. R. Fabrigar, D. T. Wegener, R. C. MacCallum, E. J. Strahan(1999) Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods 4(), 272-299 DOI: https://doi.org/10.1037/1082-989X.4.3.272/*ref*/E. Fernández del Río, J. R. Barrada, P. J. Ramos-Villagrasa(2018) Bad behaviors at work: Spanish adaptation of the workplace deviance scale. Current Psychology (), DOI: https://doi.org/10.1037/1082-989X.4.3.272/*ref*/S. Fine, M. Edward(2017) Breaking the rules, not the law: The potential risks of counterproductive work behaviors among overqualified employees. International Journal of Selection and Assessment 25(), 401-405 DOI: https://doi.org/10.1111/ijsa.12194/*ref*/C. Fritz, S. Sonnentag(2006) Recovery, well-being, and performance-related outcomes: The role of workload and vacation experiences. Journal of Applied Psychology 91(), 936-945 DOI: https://doi.org/10.1037/0021-9010.91.4.936/*ref*/L. E. Garrido, J. R. Barrada, J. A. Aguasvivas, A. Martínez-Molina, V. B. Arias, H. F. Golino, L. Rojo-Moreno(2018) Is small still beautiful for the Strengths and Difficulties Questionnaire? Novel findings using exploratory structural equation modeling. Assessment (), DOI: https://doi.org/10.1177/1073191118780461/*ref*/M. J. Gorgievski, A. B. Bakker, W. B. Schaufeli(2010) Work engagement and workaholism: comparing the self-employed and salaried employees. The Journal of Positive Psychology 5(), 83-96 DOI: https://doi.org/10.1080/17439760903509606/*ref*/B. J. Hoffman, C. A. Blair, J. P. Meriac, D. J. Woehr(2007) Expanding the criterion domain? A quantitative review of the OCB literature. Journal of Applied Psychology 92(), 555-566 DOI: https://doi.org/10.1037/0021-9010.92.2.555/*ref*/L. Hu, P. M. Bentler(1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 6(), 1-55 DOI: https://doi.org/10.1080/10705519909540118/*ref*/G. M. Hurtz, J. J. Donovan(2000) Personality and job performance: The Big Five revisited. Journal of Applied Psychology 85(), 869-879 DOI: https://doi.org/10.1037//0021-9010.85.6.869/*ref*/T. A. Judge, J. D. Kammeyer-Mueller(2012) General and specific measures in organizational behavior research: Considerations, examples, and recommendations for researchers. Journal of Organizational Behavior 33(), 161-174 DOI: https://doi.org/10.1002/job.764/*ref*/L. Koopmans(2015) . Individual Work Performance Questionnaire instruction manualTNO Innovation for Life – VU University Medical Center (), DOI: https://doi.org/10.1002/job.764/*ref*/L. Koopmans, C. M. Bernaards, V. H. Hildebrandt, H. C. W. De Vet, A. J. van der Beek(2013) Measuring individual work performance: Identifying and selecting indicators. Work 48(), 229-238 DOI: https://doi.org/10.3233/WOR-131659/*ref*/L. Koopmans, C. M. Bernaards, V. H. Hildebrandt, H. C. W. De Vet, A. J. Van Der Beek(2014) Construct validity of the Individual Work Performance Questionnaire. Journal of Occupational and Environmental Medicine 56(), 331-337 DOI: https://doi.org/10.1097/JOM.0000000000000113/*ref*/L. Koopmans, C. M. Bernaards(2016) Cross-cultural adaptation of the Individual Work Performance Questionnaire. Work (), 609-619 DOI: https://doi.org/10.3233/WOR-152237/*ref*/L. Koopmans, C. M. Bernaards, V. H. Hildebrandt, W. B. Schaufeli, C. W. de Vet Henrica, A. J. van der Beek(2011) Conceptual frameworks of individual work performance. Journal of Occupational and Environmental Medicine 53(), 856-866 DOI: https://doi.org/10.1097/JOM.0b013e318226a763/*ref*/L. Koopmans, C. Bernaards, V. Hildebrandt, S. van Buuren, A. J. van der Beek, H. C. W. de Vet(2013) Development of an individual work performance questionnaire. International Journal of Productivity and Performance Management 62(), 6-28 DOI: https://doi.org/10.1108/17410401311285273/*ref*/C. H. Lawshe(1975) A quantitative approach to content validity. Personnel Psychology 28(), 563-575 DOI: https://doi.org/10.1111/j.1744-6570.1975.tb01393.x/*ref*/K. Lee, N. J. Allen(2002) Organizational citizenship behavior and workplace deviance: The role of affect and cognitions. The Journal of Applied Psychology 87(), 131-142 DOI: https://doi.org/10.1037/0021-9010.87.1.131/*ref*/J. A. Lepine, A. Erez, D. E. Johnson(2002) The nature and dimensionality of organizational citizenship behavior: A critical review and meta-analysis. The Journal of Applied Psychology 87(), 52-65 DOI: https://doi.org/10.1037/0021-9010.87.1.52/*ref*/S. B MacKenzie, P. M. Podsakoff, R. Fetter(1991) Organizational citizenship behavior and objective productivity as determinants of managerial evaluations of salespersons’ performance. Organizational Behavior and Human Decision Processes 50(), 123-150 DOI: https://doi.org/10.1016/0749-5978(91)90037-T/*ref*/J. Meedovi, B. Petrovi(2015) The Dark Tetrad: Structural properties and location in the personality space. Journal of Individual Differences 36(), 228-236 DOI: https://doi.org/10.1027/1614-0001/a000179/*ref*/M. Morf, A. Feierabend, B. Staffelbach(2017) Task variety and counterproductive work behavior. Journal of Managerial Psychology 32(), 581-592 DOI: https://doi.org/10.1108/JMP-02-2017-0048/*ref*/J. Muñiz, P. Elosua, R. K. Hambleton(2013) Directrices para la traducción y adaptación de los test: segunda edición [International Test Commission guidelines for test translation and adaptation: Second edition]. Psicothema 25(), 151-157 DOI: https://doi.org/10.7334/psicothema2013.24/*ref*/K. R. Murphy(2008) Explaining the weak relationship between job performance and ratings of job performance. Industrial and Organizational Psychology (), DOI: https://doi.org/10.1111/j.1754-9434.2008.00030.x/*ref*/L. K. Muthén, B. O. Muthén(2015) . Mplus user’s guide. Seventh editionMuthén & Muthén (), DOI: https://doi.org/10.1111/j.1754-9434.2008.00030.x/*ref*/G. Navarro-Carrillo, A. M. Beltrán-Morillas, I. Valor-Segura, F. Expósito(2017) What is behind envy? Approach from a psychosocial perspective [¿Qué se esconde detrás de la envidia? Aproximación desde una perspectiva psicosocial]. Revista de Psicología Social 32(), 217-245 DOI: https://doi.org/10.1080/02134748.2017.1297354/*ref*/D. W. Organ, J. B. Paine(1999) . International review of industrial and organizational psychologyJohn Wiley & Sons Ltd. 14(), 337-368 DOI: https://doi.org/10.1080/02134748.2017.1297354/*ref*/N. P. Podsakoff, S. W. Whiting, P. M. Podsakoff, B. D. Blume(2009) Individual- and organizational-level consequences of organizational citizenship behaviors: A meta-analysis. Journal of Applied Psychology 94(), 122-141 DOI: https://doi.org/10.1037/a0013079/*ref*/E. D. Pulakos, S. Arad, M. A. Donovan, K. E. Plamondon(2000) Adaptability in the workplace: Development of a taxonomy of adaptive performance. Journal of Applied Psychology 85(), 612-624 DOI: https://doi.org/10.1037/0021-9010.85.4.612/*ref*/(2019) . R: A language and environment for statistical computingR Foundation for Statistical Computing (), DOI: https://doi.org/10.1037/0021-9010.85.4.612/*ref*/U. Rehman, M. G. Shahnawaz(2018) Machiavellianism, job autonomy, and counterproductive work behaviour among Indian managers. Journal of Work and Organizational Psychology 34(), 83-88 DOI: https://doi.org/10.5093/jwop2018a10/*ref*/M. Rhemtulla, P. É. Brosseau-Liard, V. Savalei(2012) When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods 17(), 354-373 DOI: https://doi.org/10.1037/a0029315/*ref*/S. L. Robinson, R. J. Bennett(1995) A typology of deviant workplace behaviors: A multidimensional scaling study. Academy of Management Journal 38(), 555-572 DOI: https://doi.org/10.2307/256693/*ref*/K.-A. Rogers, E. K. Kelloway(1997) Violence at work: Personal and organizational outcomes. Journal of Occupational Health Psychology 2(), 63-71 DOI: https://doi.org/10.1037/1076-8998.2.1.63/*ref*/M. Rotundo, P. R. Sackett(2002) The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach. Journal of Applied Psychology 87(), 66-80 DOI: https://doi.org/10.1037//0021-9010.87.1.66/*ref*/P. R. Sackett(2002) The structure of counterproductive work behaviors: Dimensionality and relationships with facets of job performance. International Journal of Selection and Assessment 10(), 5-11 DOI: https://doi.org/10.1111/1468-2389.00189/*ref*/P. R. Sackett, C. J. DeVore(2001) . Handbook of industrial, work, and organizational psychologySAGE Publications (), DOI: https://doi.org/10.4135/9781848608320.n9/*ref*/P. R. Sackett, F. Lievens(2008) Personnel selection. Annual Review of Psychology 59(), 419-450 DOI: https://doi.org/10.1146/annurev.psych.59.103006.093716/*ref*/J. F. Salgado(2002) The Big Five personality dimensions and counterproductive behaviors. International Journal of Selection and Assessment 10(), 117-125 DOI: https://doi.org/10.1111/1468-2389.00198/*ref*/J. F. Salgado(2003) Predicting job performance using FFM and non-FFM personality measures. Journal of Occupational and Organizational Psychology 76(), 323-346 DOI: https://doi.org/10.1348/096317903769647201/*ref*/J. F. Salgado, Á. L. Cabal(2011) Evaluación del desempeño en la administración pública del Principado de Asturias: Análisis de las propiedades psicométricas [Performance appraisal in the public administration of the Principality of Asturias: An analysis of psychometric properties]. Journal of Work and Organizational Psychology 27(), 75-91 DOI: https://doi.org/10.1348/096317903769647201/*ref*/J. F. Salgado, S. Moscoso, J. I. Sanchez, P. Alonso, B. Choragwicka, A. Berges(2015) Validity of the five-factor model and their facets: The impact of performance measure and facet residualization on the bandwidth-fidelity dilemma. European Journal of Work and Organizational Psychology 24(), 325-349 DOI: https://doi.org/10.1080/1359432X.2014.903241/*ref*/D. Sánchez-Carracedo, J. R. Barrada, G. López-Guimerà, J. Fauquet, C. A. Almenara, E. Trepat(2012) Analysis of the factor structure of the Sociocultural Attitudes Towards Appearance Questionnaire (SATAQ-3) in Spanish secondary-school students through exploratory structural equation modeling. Body Image 9(), 163-171 DOI: https://doi.org/10.1016/j.bodyim.2011.10.002/*ref*/S. E. Scullen, M. K. Mount, M. Goff(2000) Understanding the latent structure of job performance ratings. Journal of Applied Psychology 85(), 956-970 DOI: https://doi.org/10.1037/0021-9010.85.6.956/*ref*/E. Selenko, A. Mäkikangas, S. Mauno, U. Kinnunen(2013) How does job insecurity relate to self-reported job performance? Analysing curvilinear associations in a longitudinal sample. Journal of Occupational and Organizational Psychology 86(), 522-542 DOI: https://doi.org/10.1111/joop.12020/*ref*/C. Viswesvaran, D. S. Ones(2017) . The Blackwell handbook of personnel selectionWiley (), 354-375 DOI: https://doi.org/10.1002/9781405164221.ch16/*ref*/J. M. Werner(1994) Dimensions that make a difference: Examining the impact of in-role and extrarole behaviors on supervisory ratings. Journal of Applied Psychology 79(), 98-107 DOI: https://doi.org/10.1037/0021-9010.79.1.98/*ref*/D. J. Woehr(2008) On the relationship between job performance and ratings of job performance: What do we really know?. Industrial and Organizational Psychology 1(), 161-166 DOI: https://doi.org/10.1111/j.1754-9434.2008.00031.xhttps://creativecommons.org/licenses/by-nc-nd/4.0/deed.es_ESCopyright © 2026. Colegio Oficial de Psicólogos de Madridinfo:eu-repo/semantics/openAccessoai:journals.copmadrid.org:jwop/art/jwop2019a212026-06-23T12:47:17Z |
| score |
15,228081 |