An Evaluation Model for In-Service Virtual Training Programs for Organizational Employees
Keywords:
Evaluation, Virtual Training, In-service TrainingAbstract
Purpose: The purpose of this study was to develop an evaluation model for in-service virtual training programs for organizational employees based on the Kirkpatrick model and to identify the key components and indicators across the levels of reaction, learning, behavior, and results.
Methodology: This study was conducted using a qualitative approach with an applied objective. The study population consisted of 20 university faculty members and experts in educational management and organizational training who were selected through purposive convenience sampling. Data were collected using semi-structured interviews designed according to the theoretical framework of the Kirkpatrick model. Content validity was assessed using the Content Validity Ratio (CVR) and Content Validity Index (CVI), and all 83 indicators were retained due to acceptable validity values. Reliability was evaluated through inter-coder agreement, yielding a reliability coefficient of 93%. Data analysis was performed using open and axial coding procedures.
Findings: The findings revealed that 73 initial concepts were extracted during the open coding stage. At the reaction level, three major components were identified, including learners’ satisfaction with the virtual learning environment, quality of interaction with instructors, and content attractiveness. At the learning level, employee empowerment, acquisition of new knowledge and skills, ability to use digital tools, and attitude change toward online learning were recognized as key components. At the behavior level, the use of newly acquired skills in the workplace, changes in task performance methods, and training-based professional interactions were emphasized. Finally, at the results level, organizational productivity, improvement of service quality, and reduction of training costs were identified as the main outcomes, indicating the effectiveness of virtual training programs in improving organizational performance.
Conclusion: The findings demonstrated that virtual in-service training programs can serve as an effective tool for improving employees’ knowledge, skills, and organizational performance. The proposed model based on the Kirkpatrick framework provides organizations with a comprehensive approach for evaluating virtual training effectiveness and supports the continuous improvement of educational processes within organizations.
Downloads
References
Abdolahi, M. J., Barmayehvar, B., Marjani, T., & Esmaeilabadi, R. (2023). Investigating the relationship between human resource development and quality in building projects. Journal of Financial Management of Property and Construction, 28(2), 260-271. https://doi.org/10.1108/jfmpc-06-2021-0035
Abich, J. I., Parker, J., Murphy, J. S., & Eudy, M. (2021). A review of the evidence for training effectiveness with virtual reality technology. Virtual Reality, 25(4), 919-933. https://doi.org/10.1007/s10055-020-00498-8
Adeniyi, I. S., Al Hamad, N. M., Adewusi, O. E., Unachukwu, C. C., Osawaru, B., Chilson, O. U., & David, I. O. (2024). Reviewing online learning effectiveness during the COVID-19 pandemic: A global perspective. International Journal of Scientific Research Archive, 11, 1676-1685. https://doi.org/10.30574/ijsra.2024.11.1.0282
Al-Mughairi, A. M. (2018). The evaluation of training and development of employees: The case of a national oil and gas industry Brunel University London].
Al-Omari, Z., Alomari, K., & Aljawarneh, N. (2020). The role of empowerment in improving internal process, customer satisfaction, learning and growth. Management Science Letters, 10(4), 841-848. https://doi.org/10.5267/j.msl.2019.10.013
Barani, S., Asadi, E., & Ahmadi, M. (2021). Identifying facilitating and inhibiting factors in conducting electronic in-service training courses for employees: Case study: Shahid Sattari Aviation University.
Bluestone, J., Johnson, P., Fullerton, J., Carr, C., Alderman, J., & BonTempo, J. (2013). Effective in-service training design and delivery: Evidence from an integrative literature review. Human Resources for Health, 11, 1-26. https://doi.org/10.1186/1478-4491-11-51
Dehghanzadeh, A., Fishman Ziyari, E., Moazami, M., & Kavousi, E. (2022). Providing a model for evaluating the effectiveness of virtual in-service training courses for employees in Tejarat Bank. Sociology of Education, 8(1), 1-14. https://doi.org/10.61838/kman.soe.756
Habibi, H., Hamrahzadeh, M., & Jafariyann, V. (2020). Pathology of organizational training management: A phenomenological study. Journal of New Approach in Educational Administration, 11(4), 55-76. https://sid.ir/paper/1000138/en
Hashem, M., Sfeir, E., Hejase, H. J., & Hejase, A. J. (2022). Effect of online training on employee engagement during the COVID-19 era. Asian Business Research, 7(5), 10-40. https://doi.org/10.20849/abr.v7i5.1294
Khalil, M. K., & Elkhider, I. A. (2016). Applying learning theories and instructional design models for effective instruction. Advances in Physiology Education. https://doi.org/10.1152/advan.00138.2015
Kirkpatrick, J. (2015). An introduction to the new world Kirkpatrick model. https://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/Introduction-to-The-New-World-Kirkpatrick%C2%AE-Model.pdf
Mahtalab, Z., Hoseini, S. M., Mosa Farkhani, E., & Makarem, A. (2021). Assessing the quality of in-service training for employees of Mashhad comprehensive health centers. Horizon of Medical Education Development, 12(2), 33-44. https://doi.org/10.22038/hmed.2020.51109.1071
Matović, S. (2025). Effectiveness of in-service teacher training for developing necessary digital competencies in modern education. Istraživanja u pedagogiji, 15(2), 408-418. https://doi.org/10.5937/IstrPed2502408M
Maunula, M., & Harju-Luukkainen, H. (2026). Enhancing teacher well-being through in-service training: A study on the impact of the Teacher Classroom Management (TCM) method. In A Nordic perspective on teacher well-being in educational contexts (1st ed.) (pp. 18). Routledge. https://doi.org/10.4324/9781003589938-17
Mohammadi, H. (2021). Measure the effectiveness of on-the-job training courses: Case study: Mazandaran Regional Water Company. Knowledge Retrieval and Semantic Systems, 8(27), 177-203. https://doi.org/10.22054/jks.2020.48013.1257
Mohammadi, M. S., Sohrabi, R., & Saraji, F. (2019). Identifying and ranking barriers to the effectiveness of e-learning for employees: Case study of Kurdistan University of Medical Sciences. Educational Technology (Technology and Education), 14(3), 671-680. https://doi.org/10.22061/jte.2019.4173.2016
Muller Queiroz, A. C., Moreira Nascimento, A., Tori, R., Brashear Alejandro, T., Veloso de Melo, V., de Souza Meirelles, F., & da Silva Leme, M. I. (2018). Immersive virtual environments in corporate education and training. https://doi.org/10.1111/jcal.1259
Mwakabenga, R. J. (2025). Sustainable In-Service Teacher Training Programmes in Tanzania: Harnessing Mentorship for Continued Professional Learning. Papers in Education and Development, 1(aop), 1-22. https://doi.org/10.1163/26650746-20250005
Puripunyavanich, M. (2025). Measuring the effectiveness of an online teacher training program for in-service secondary school teachers: Trainees' reactions and learning. LEARN Journal: Language Education and Acquisition Research Network, 18(1), 816-857. https://doi.org/10.70730/YVSD7249
Rahmati, M. H., Hosseini Fard, S. M., & Alimadadi, A. (2014). Investigating effectiveness of in-service training in the public sector. Interdisciplinary Journal of Management Studies, 7(2), 305-327. https://doi.org/10.22059/ijms.2014.36619
Ramezani Ardi, E., Zaraii Zavaraki, E., Nili, M., & Aliabadi, K. (2018). Developing and assessment of competencies for steering and development of in-service e-learning in government organizations. Quarterly of Educational Measurement, 9(34), 173-194. https://doi.org/10.22054/jem.2019.38313.1867
Ramezani Limaee, A., Sobhani, A., Mohammad Davoudi, A. H., & Taghipour Zahir, A. (2022). Presenting a structural model to promote effective in-service training of administrative staff of Islamic Azad University: Case study: Tehran Province. Educational Development of Jundishapur, 13(0), 75-86. https://doi.org/10.22118/edc.2021.309703.1904
Ramezani Limaei, A., Sobhani, A., Mohammad Davoodi, A. H., & Taghipoor Zahir, A. (2021). Designing a conceptual model for improving the effectiveness of in-service training of the administrative employees of the Islamic Azad University. Journal of Human Resource Management Research, 131(43), 225-260. https://sid.ir/paper/1046191/en
Shakeri Kalilabad, F., Gholami, H., Salehian, M., & Bagherian Frotageh, F. (2022). In-service training courses, an important approach for empowerment and development of organizational human capital. Horizon of Medical Education Development, 13(4), 49-60. https://doi.org/10.22038/hmed.2021.58941.1166
Talebi, S. (2019). Standardization and designing the questionnaire for assessing the effectiveness of electronic in-service curriculum among educational managers. Journal of New Approaches in Educational Administration, 9(36), 91-106. https://doi.org/20.1001.1.20086369.1397.9.36.5.0
Tarazi, M., Parsa Moein, K., & Afkaneh, S. (2020). Identifying the affecting criteria and components on the effectiveness of in-service training courses for employees of the Refah Bank. Career and Organizational Counseling, 12(42), 95-120. https://doi.org/10.52547/jcoc.12.1.95
Torres Martín, C., Acal, C., El Homrani, M., & Mingorance Estrada, Á. C. (2021). Impact on the virtual learning environment due to COVID-19. Sustainability, 13(2), 582. https://doi.org/10.3390/su13020582
Urbancová, H., Vrabcová, P., Hudáková, M., & Petrů, G. J. (2021). Effective training evaluation: The role of factors influencing the evaluation of effectiveness of employee training and development. Sustainability, 13(5), 2721. https://doi.org/10.3390/su13052721
Varshney, D. (2022). Understanding virtual employee onboarding (VEO): The new normal and beyond. Emirati Journal of Business, Economics and Social Studies, 1(1), 58-80. https://doi.org/10.54878/EJBESS.171
Yaqoot, E. S., Wan Mohd Noor, W. S., & Mohd Isa, M. F. (2017). Factors influencing training effectiveness: Evidence from public sector in Bahrain. Acta Universitatis Danubius. Œconomica, 13(2). https://www.ceeol.com/search/article-detail?id=546806
Yazdani, F. (2014). Measuring the effectiveness of the in-service virtual education system for educators. Information and Communication Technology in Educational Sciences, 5(2), 97-122. https://dor.isc.ac/dor/20.1001.1.20080441.1394.9.4.1.0
Yousefi, Z., Imani, M. N., & Jahed, H. (2022). Presenting a model for improving the effectiveness of in-service training courses for employees of the central headquarters of the Ministry of Health and Medical Education. Medicine and Purification, 31(3), 199-215. https://dor.isc.ac/dor/20.1001.1.16082397.1401.31.3.1.2
Downloads
Published
Submitted
Revised
Accepted
Issue
Section
License
Copyright (c) 2026 Shima Valikhani (Author); Masoumeh Oladian; Reza Hassanavi (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
