A Critical Approach to the Construct -Related Validity of Assessment Centers: A Study of the Assessment Center of the National Iranian Oil Company

Salimi, Majid; Shokrkon, Hossein; Montakhab Yeganeh, Mohammad

doi:10.22034/ijpb.2021.261549.1211

[Home ] [Archive]

[ فارسی ]

International Journal of Psychology (IPA)

Main Menu

Home

Journal Information

Articles archive

For Authors

For Reviewers

Registration

Contact us

Site Facilities

reviewer

Word Cloud

In Publish

Designing and Testing a Model of the Relationship between Transformational Leadership, Job Involvement as well as Health Literacy and Quality of Work Life: Mediating Role of Perceived Organizational Support between Transformational Leadership and Quality of Work Life
Raziyeh Abedini Velamdehy, Nasrin Arshadi ^*, Kioumars Beshlideh

The Effect of Inclusive Leadership on Change-Oriented Organizational Citizenship Behavior and Benevolent Rule-Breaking: The Mediating Role of Trust in the Leader
Fatemeh Latifat ^*, Abdolzahra Naami, Seyed Esmaeil Hashemi

Effectiveness of the Promoting Adult Resilience (PAR) Program on Resilience Resources and Positive Adaptation in Hospital Staff: A Natural Experiment Amid the War
Saba Gheysari, Kioumars Beshlideh ^*, Abdolkazem Neisi, nasrin arshadi

Examining the Efficacy of Metacognitive Training Interventions in Enhancing Behavioral Regulation, Attentional Control, Working Memory, and Reducing Impulsivity among Adolescents with Attention Deficit/Hyperactivity Disorder (ADHD): A Randomized Controlled Trial
Shima Tamannaeifar, Ghazale Raei Dehaghi, Farhad Mohammadi Masiri ^*

Search in website

Receive site information

Journal Statistics

Last 3 years statistics:

Submitted articles: 228

Accepted articles: 59

Rejected articles: 98

Published articles: 58

Acceptance rate: 25.88

Rejection rate: 42.98

Average Time to Accept: 178 days

Average Time to First Review: 20.8 days

Average Time to Publish: 76.9 days

____

Volume 14, Issue 2 (4-2020)

ijpb 2020, 14(2): 218-247

Back to browse issues page

A Critical Approach to the Construct -Related Validity of Assessment Centers: A Study of the Assessment Center of the National Iranian Oil Company

Majid Salimi ^*

, Hossein Shokrkon

, Mohammad Montakhab Yeganeh

Training and Development Manager of the National Iranian Oil Company

Abstract: (5319 Views)

The purpose of this study was to investigate the construct validity of the Assessment and Development Center of Managers in the National Iranian Oil Company. The research method is based on the applied purpose and in terms of data collection is descriptive-analytical. The statistical sample includes data obtained from the evaluation of 384 managers who were selected from the database of managers by simple random sampling. These data were analyzed based on the multi-trait-multi-method and the confirmatory factor analysis. The overall fit indices showed that the data fit the model very well. The examination of the individual parameter estimates indicates that while both dimension and exercise factors contribute to the ratings, the estimates for exercise parameters are substantially larger in all cases than estimates for dimension factors (mean parameter estimates of .39 and .68 for the dimension and exercise parameters, respectively). In addition, inter-correlations among the latent dimension factors ranged from 0.98 to 1.0, providing evidence of no discriminant validity across dimensions. Thus, these results support and are consistent with those obtained from the correlational analyses. To put it short, the exercise factors are the primary determinants of assessment center ratings.

Keywords: assessment centers, construct validity, discriminant validity, dimension, exercise

Full-Text [PDF 774 kb] (2442 Downloads)

Type of Study: Research | Subject: General
Received: 2020/12/10 | Revised: 2021/08/9 | Accepted: 2021/08/14 | Published: 2021/09/10

References

1. Arthur, Jr., W., Woehr, D. J., & Maldegen, R. (2000). Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical reexamination of the assessment center construct-related validity paradox. Journal of Management, 26(4), 813-835. https://doi.org/10.1177/014920630002600410 [DOI:10.1016/S0149-2063(00)00057-X]

2. Arthur, W., Day, E. A., & Woehr, D. J. (2008). Mend it, don't end it: An alternate view of assessment center construct-related validity evidence. Industrial and Organizational Psychology, 1(1), 105-111. [DOI:10.1111/j.1754-9434.2007.00019.x]

3. Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A conceptual analysis of the inferential and evidential bases. Journal of Applied Psychology, 74(3), 478. [DOI:10.1037/0021-9010.74.3.478]

4. Bowler, M. C., & Woehr, D. J. (2006). A meta-analytic evaluation of the impact of dimension and exercise factors on assessment center ratings. Journal of Applied Psychology, 91(5), 1114. [DOI:10.1037/0021-9010.91.5.1114] [PMID]

5. Brannick, M. T. (2008). Back to basics of test construction and scoring. Industrial and Organizational Psychology, 1(1), 131-133. [DOI:10.1111/j.1754-9434.2007.00025.x]

6. Buckett, A., Becker, J. R., Melchers, K. G., & Roodt, G. (2020). How different indicator-dimension ratios in assessment center ratings affect evidence for dimension factors. Frontiers in Psychology, 11, 459. [DOI:10.3389/fpsyg.2020.00459] [PMID]

7. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81. [DOI:10.1037/h0046016] [PMID] [PMCID]

8. Connelly, B. S., Ones, D. S., Ramesh, A., & Goff, M. (2008). A pragmatic view of assessment center exercises and dimensions. Industrial and Organizational Psychology, 1(1), 121-124. [DOI:10.1111/j.1754-9434.2007.00022.x]

9. Dunnette, M. D. (1971). The assessment of managerial talent. In P. McReynolds (Ed.), Advances in psychological assessment (Vol. 2). Palo Alto, CA: Science and Behavior Books.

10. Hardison, C. M., & Sackett, P. R. (2004). Assessment center criterion-related validity: A meta-analytic update. Unpublished Manuscript. [DOI:10.1037/e518632013-042]

11. Hennessy, J., Mabey, B., & Warr, P. (1998). Assessment centre observation procedures: An experimental comparison of traditional, checklist and coding methods. International Journal of Selection and Assessment, 6(4), 222-231. [DOI:10.1111/1468-2389.00093]

12. Howard, A. (1997). A reassessment of assessment centers: Challenges for the 21st century. Journal of Social Behavior and Personality, 12(5), 13-52.

13. Howard, A. (2008). Making assessment centers work the way they are supposed to. Industrial and Organizational Psychology, 1(1), 98-104. [DOI:10.1111/j.1754-9434.2007.00018.x]

14. International Task Force on Assessment Center Guidelines. (2000). Guidelines and ethical considerations for assessment center operations. International Journal of Selection and Assessment, 17(3), 243-253. [DOI:10.1111/j.1468-2389.2009.00467.x]

15. Kleinmann, M., & Ingold, P. V. (2019). Toward a better understanding of assessment centers: A conceptual review. Annual Review of Organizational Psychology and Organizational Behavior, 6, 349-372. doi:10.1146/annurev-orgpsych-012218-014955. [DOI:10.1146/annurev-orgpsych-012218-014955]

16. Kleinmann, M., & Köller, O. (1997). Construct validity of assessment centers: Appropriate use of confirmatory factor analysis and suitable construction principles. Journal of Social Behavior and Personality, 12(5), 65.

17. Kolk, N. J., Born, M. P., & Der Flier, H. V. (2003). The transparent assessment centre: The effects of revealing dimensions to candidates. Applied Psychology, 52(4), 648-668. [DOI:10.1111/1464-0597.00156]

18. Lance, C. E. (2008). Why assessment centers do not work the way they are supposed to. Industrial Organizational Psychology: Perspectives on Science and Practice, 1, 84-97. [DOI:10.1111/j.1754-9434.2007.00017.x]

19. Lance, C. E., Lambert, T. A., Gewin, A. G., Lievens, F., & Conway, J. M. (2004). Revised estimates of dimension and exercise variance components in assessment center postexercise dimension ratings, Journal of Applied Psychology, 89(2), 377. [DOI:10.1037/0021-9010.89.2.377] [PMID]

20. Lievens, F. (2002). Trying to understand the different pieces of the construct validity puzzle of assessment centers: An examination of assessor and assessee effects. Journal of Applied Psychology, 87(4), 675. [DOI:10.1037/0021-9010.87.4.675] [PMID]

21. Lievens, F. (2008). What does exercise-based assessment really mean?. Industrial and Organizational Psychology, 1(1), 112-115. [DOI:10.1111/j.1754-9434.2007.00020.x]

22. Lievens, F., & Klimoski, R. J. (2001). Understanding the assessment center process: Where are we now?.

23. Lievens, F., & Sackett, P. R. (2007). Situational judgment tests in high-stakes settings: Issues and strategies with generating alternate forms. Journal of Applied Psychology, 92(4), 1043. [DOI:10.1037/0021-9010.92.4.1043] [PMID]

24. Marsh, H. W., & Grayson, D. (1995). Latent variable models of multitrait-multimethod data.

25. Moses, J. (2008). Assessment centers work, but for different reasons. Industrial and Organizational Psychology, 1(1), 134-136. [DOI:10.1111/j.1754-9434.2007.00026.x]

26. Moses, J. L. & Byham,W. C. (Eds.). (1977). Applying the assessment center method. New York: Pergamon. [DOI:10.1016/B978-0-08-019581-0.50006-2]

27. Robie, C., Osburn, H. G., Morris, M. A., Etchegaray, J. M., & Adams, K. A. (2000). Effects of the rating process on the construct validity of assessment center dimension evaluations. Human Performance, 13(4), 355-370. [DOI:10.1207/S15327043HUP1304_2]

28. Rupp, D. E., Thornton, G. C., & Gibbons, A. M. (2008). The construct validity of the assessment center method and usefulness of dimensions as focal constructs. Industrial and Organizational Psychology, 1(1), 116-120. [DOI:10.1111/j.1754-9434.2007.00021.x]

29. Sackett, P. R., & Dreher, G. F. (1982). Constructs and assessment center dimensions: Some troubling findings. Journal of Applied Psychology, 67, 401-410. [DOI:10.1037/0021-9010.67.4.401]

30. Sackett, P. R., & Dreher, G. F. (1984). Situation specificity of behavior and assessment center validation strategies: A rejoinder to Neidig and Neidig. Journal of Applied Psychology, 69, 187-190. [DOI:10.1037/0021-9010.69.1.187]

31. Sackett, P. R., & Tuzinski, K. (2001). The role of dimensions in assessment center judgment. In M. London (Ed.), How people evaluate others in organizations. Mahwah, NJ: Erlbaum.

32. Sackett, P. R., Shewach, O. R., & Keiser, H. N. (2017). Assessment centers versus cognitive ability tests: Challenging the conventional wisdom on criterion-related validity. Journal of Applied Psychology, 102, 1435-1447. [DOI:10.1037/apl0000236] [PMID]

33. Schneider, J. R., & Schmitt, N. (1992). An exercise design approach to understanding assessment center dimension and exercise constructs. Journal of Applied Psychology, 77(1), 32. [DOI:10.1037/0021-9010.77.1.32]

34. Schuler, H. (2008). Improving assessment centers by the trimodal concept of personnel assessment. Industrial and Organizational Psychology, 1(1), 128-130. [DOI:10.1111/j.1754-9434.2007.00024.x]

35. Tett, R. P., & Burnett, D. D. (2003). A personality trait-based interactionist model of job performance. Journal of Applied Psychology, 88(3), 500. [DOI:10.1037/0021-9010.88.3.500] [PMID]

36. Thornton, G. C., III, & Byham,W. C. (1982). Assessment centers and managerial performance. New York: Academic. https://doi.org/10.1016/B978-0-12-690620-2.50008-8 https://doi.org/10.1016/B978-0-12-690620-2.50015-5 https://doi.org/10.1016/B978-0-12-690620-2.50009-X https://doi.org/10.1016/B978-0-12-690620-2.50016-7 https://doi.org/10.1016/B978-0-12-690620-2.50013-1 https://doi.org/10.1016/B978-0-12-690620-2.50010-6 https://doi.org/10.1016/B978-0-12-690620-2.50007-6 https://doi.org/10.1016/B978-0-12-690620-2.50011-8 https://doi.org/10.1016/B978-0-12-690620-2.50006-4 https://doi.org/10.1016/B978-0-12-690620-2.50012-X [DOI:10.1016/B978-0-12-690620-2.50014-3]

37. Thornton, G. C., III, & Mueller-Hanson, R. A. (2004). Developing organizational simulations: A guide for practitioners and students. Mahwah, NJ: Erlbaum. [DOI:10.4324/9781410609458]

38. Wirz, A., Melchers, K. G., Kleinmann, M., Lievens, F., Annen, H., Blum, U., & Ingold, P. V. (2020). Do overall dimension ratings from assessment centers show external construct-related validity?. European Journal of Work and Organizational Psychology, 29(3), 405-420. [DOI:10.1080/1359432X.2020.1714593]

39. Woehr, D. J., & Arthur, Jr., W. (2003). The construct-related validity of assessment center ratings: A review and meta-analysis of the role of methodological factors. Journal of Management, 29(2), 231-258. [DOI:10.1177/014920630302900206]

Send email to the article author

‎ 10.22034/ijpb.2021.261549.1211

‎ 20.1001.1.20081251.2020.14.2.8.2

Mendeley

Zotero

RefWorks

Salimi M, Shokrkon H, Montakhab Yeganeh M. A Critical Approach to the Construct -Related Validity of Assessment Centers: A Study of the Assessment Center of the National Iranian Oil Company. ijpb 2020; 14 (2) :218-247
URL: http://ijpb.ir/article-1-305-en.html

Rights and permissions
	This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Volume 14, Issue 2 (4-2020)

Back to browse issues page

Persian site map - English site map - Created in 0.22 seconds with 49 queries by YEKTAWEB 4772