00:00 Kirkpatrick model for evaluation 1:13 Level 1 : Reaction Overall satisfactions and engagement Survey, interviews, focus group 2:05 Level 2 :Learning New Knowledge, skills or attitude Quizzes / tests, interviews, focus groups, discussions, observation, and more. 3:09 Level 3 :Impact Are people performing the critical tasks? Surveys, observation, work review, KPls 4:30 Level 4 :Results Which results do we aim to achieve? What do people need to do differently? What knowledge and skills do they need? How do we design an attractive intervention? 8:33 Conclusion
Devlin -- I've worked on projects where there's an interest in knowing an ROI to justify the cost of say, narration or video vs just Powerpoint, but there's NO interest in looking at level 2 or 3. Maybe it's compliance or other training where the ROI is a little squishier. Management is only interested in justifying the cost and cares less about the long-term results. How would Kirkpatrick address this? Perhaps that kind of ROI is better evaluated with just infographics on the value of eLearning? Thx.
I think that Kirkpatrick would say that those efforts are unlikely to produce any real results. If the stakeholders are not interested in improving business metrics or actually changing human behavior, then there are few cases where the efforts will make a difference for the audience or the organizaton.
@@DevlinPeckYT Thanks for the reply. I think that for compliance and other BAU training there’s no interest in metrics.. only in compliance. No behavior change if the employees are already in compliance, so it comes down to evaluating the return on the training investment every year to keep the status quo.
Hi Travis! I appreciate the support. I referenced this book when I was making this vid to make sure I didn't miss / misrepresent anything: www.kirkpatrickpartners.com/product/Kirkpatricks-Four-Levels-of-Training-Evaluation/
I love how this guy seems annoyed by his own presentation. I could hear him sigh...... hahahaha dude, we're not forcing you to do this, you want to present this information to us 😅🤣🤣🤣🤣🤣
👀👀👀. I must have caught myself on a bad day with this one haha. Definitely happy to present the info but it can be rough some times…especially in these earlier days when I was newer to TH-cam 😂
The Kirkpatrick model consists of four evaluation levels-Reaction, Learning, Behavior, and Results-where Reaction measures participant satisfaction through surveys, Learning assesses knowledge acquisition via quizzes, Behavior evaluates the application of skills on the job through observations, and Results measures the impact on business metrics, all of which require careful planning that starts with identifying desired results and works backward to define necessary behaviors and skills, while also allowing for iterative adjustments during training to ensure effectiveness.
This old model lacks human capital factors, for instance, level four should also include employee retention (as an example: The customer received high level service and needs are met for the employee - Maslow hierarchy of needs)
This is one area where I think a lot of educators would have some experience, especially if there were in any area of curriculum develop or accreditation studies, but it could be helpful to have some more real world examples. It seems to be hard to find those in the courses I've done on LinkedIn and such. Thanks again for the refresher!
Your videos have been so helpful! I have an ID interview tomorrow! I'm a Learning and Development specialist but haven't focused on the ID side of things. Thank you so much for helping me to prepare for the interview in understanding how I can add to my learning.
I am moving to the USA. I have experience as trainer and university professor. Do you recommend to offer to design a course to a local business, in my home country, for free in order to build my portfolio?
That could work, Fabian! But you could also create a concept project so that you do not need to rely on unavailable SMEs, unreasonable requests from clients, etc.
Thanks for the summary. Useful content and I do agree that many instructional designers avoid Levels 3 and 4, but this is where true value lies. It's not as complicated as it may seem. We should be included in planning and done right you're able to measure training impact - leading to improved learning interventions. It's also important to periodically measure your evaluation methods so that you continually improve them.
Thanks for the vid. Briefly,... Level 3 & 4 are needed for just in time training, however they don't immediately apply to capacity building training, which rely mostly on level 1 & 2 evals.
Hi Devlin! Is the designing part in the planning process (the 4th point: designing attractive intervention) related to the reaction level (i.e. Level 1)? If yes, then how?
Thanks @Devlin for knowledge in only 9 minutes You have been able to summarize a model perfectly. Thanks! I continue having a little doubt. How I could show that level 3 and 4 It is a consequence of a training? when It has runned time for example 6 months or 1 year . Thanks in advance!
Good question, Cynthia! I would suggest looking into the success case method for a program that has already been running for a while. This model works best when baked into the strategy and design from the beginning 😃. You could also look at performance and metrics both before the training was implemented and after it was implemented.
@@DevlinPeckYT thank for your answering.. Before test and post test...I am exploring it more. Thanks for your information and congrats for great conversation with Cathy Moore. I am reading this remarkable book... I follow you!! greetings from Peru
Thank u for the sharing Devlin. I'm wondering any other factors influencing training effectiveness score..for example virtual learning vs physical learning, virtual 2 hours vs full 8 hours..do u have any view? I've been asked to this question a lot by the leaders..
Hello hello! Thanks for checking out the video. Those items would be great things to ask your intended audience (and collect data about how they’re received, how they impact learning, whether people use those new skills on the job at the same rate, etc).
@@DevlinPeckYT thank u for the idea..would be a good start to do survey..the learning platform have just swift from physical to virtual since COVID pandemic. thanks again yaa!
Level 3 actually requires additional time from SME persons. They should later send the results of whether something has changed in terms of user behavior. In most situations they are very busy and this is rarely delivered unfortunately. In most cases, you will not get these results because of the organization you are in. This also applies to the entire level.
This content is very timely as this is our topic in my Masters Degree in Human Resource Management. Thank you very much for sharing your personal thoughts on the planning perspective. Will subscribe and hoping to learn more from you. Keep rocking!
Hi...great explanation and so helpful.can you tell ACTIVITIES for this each 4 level that helps to language learning trainees or teacher? Thanks in advance.
@@sindhuvelmurgan361 Level 1 pertains to how the audience reacted to the learning experience, level 2 pertains to how much they learned, level 3 pertains to whether or not they're using what they learned in a real-world context, and level 4 pertains to the impact that their newfound langauge skills are having on the organization that they're working for :D
@@DevlinPeckYT How about content on the AECT standards? My portfolio for my MS in LDT will be judged against it this semester for graduation but NO employer ever mentions it in the job listings which mean they do not know it or its skill over credential or both.
Great videos. It's beyond me as an educator, why we can't graduate better knowledgeable, skilled, and intelligent students with all the tools we have in the United States. This then tells me that it has to be a discipline issue.
I liked how you talked about working backwards from Level 4 to Level 1. It made so much more sense.
Me too, working backwards makes so much more sense as I'm working on Behavioral competency e-learning project
wow tthis was so clear and precise it helped me form my training and development poa
Glad to hear it! Thanks Shubham 😃
This was very crisp and informative. Also, thank you for sharing the amazing list of books :)
You’re welcome! Thanks for watching, Pooja 😊
Great refresher, and yes many times we stop at level 2 but should try to get to levels 3 or 4.
For sure. Thanks for checking out the video :)
Grab the FREE interactive "Become an ID" checklist: www.devlinpeck.com/become-an-id-checklist
Next is Philips ROI - Level 5. Information that truly demonstrates value to an organization and what executives will want to hear.
Great Summary!
Thanks!
well explained.
Thanks 😃
00:00 Kirkpatrick model for evaluation
1:13 Level 1 : Reaction
Overall satisfactions and engagement
Survey, interviews, focus group
2:05 Level 2 :Learning
New Knowledge, skills or attitude
Quizzes / tests, interviews, focus groups, discussions, observation, and more.
3:09 Level 3 :Impact
Are people performing the critical tasks?
Surveys, observation, work review, KPls
4:30 Level 4 :Results
Which results do we aim to achieve?
What do people need to do differently?
What knowledge and skills do they need?
How do we design an attractive intervention?
8:33 Conclusion
Thanks for sharing this, Andres!
Devlin -- I've worked on projects where there's an interest in knowing an ROI to justify the cost of say, narration or video vs just Powerpoint, but there's NO interest in looking at level 2 or 3. Maybe it's compliance or other training where the ROI is a little squishier. Management is only interested in justifying the cost and cares less about the long-term results. How would Kirkpatrick address this? Perhaps that kind of ROI is better evaluated with just infographics on the value of eLearning? Thx.
I think that Kirkpatrick would say that those efforts are unlikely to produce any real results. If the stakeholders are not interested in improving business metrics or actually changing human behavior, then there are few cases where the efforts will make a difference for the audience or the organizaton.
@@DevlinPeckYT Thanks for the reply. I think that for compliance and other BAU training there’s no interest in metrics.. only in compliance. No behavior change if the employees are already in compliance, so it comes down to evaluating the return on the training investment every year to keep the status quo.
@@catboy721 Yeah, compliance training is notoriously one of those "checklist" training items. "Did employees complete it? Then we're in the clear!"
Yes, the same doubts! Thanks for displaying it!
@@cynthiareid6416 you’re welcome!
I liked and subscribed please share youre references. espicially google scholar
Hi Travis! I appreciate the support. I referenced this book when I was making this vid to make sure I didn't miss / misrepresent anything: www.kirkpatrickpartners.com/product/Kirkpatricks-Four-Levels-of-Training-Evaluation/
I love how this guy seems annoyed by his own presentation. I could hear him sigh...... hahahaha dude, we're not forcing you to do this, you want to present this information to us 😅🤣🤣🤣🤣🤣
👀👀👀. I must have caught myself on a bad day with this one haha. Definitely happy to present the info but it can be rough some times…especially in these earlier days when I was newer to TH-cam 😂
Thank so much! I work in professional development and this 10min video has given me so many ideas as to how we can improve our needs analysis!!
Wonderful! I’m glad to hear it 😃
extremely helpful video. crisp and engaging. thank you!
🙂
The Kirkpatrick model consists of four evaluation levels-Reaction, Learning, Behavior, and Results-where Reaction measures participant satisfaction through surveys, Learning assesses knowledge acquisition via quizzes, Behavior evaluates the application of skills on the job through observations, and Results measures the impact on business metrics, all of which require careful planning that starts with identifying desired results and works backward to define necessary behaviors and skills, while also allowing for iterative adjustments during training to ensure effectiveness.
Any weaknesses of Kirkpatrick model when applying to prepare a Training for nurse learners? Thanks
This old model lacks human capital factors, for instance, level four should also include employee retention (as an example: The customer received high level service and needs are met for the employee - Maslow hierarchy of needs)
Thanks John! I can see how that would be an important organizational factor to consider :D
This is one area where I think a lot of educators would have some experience, especially if there were in any area of curriculum develop or accreditation studies, but it could be helpful to have some more real world examples. It seems to be hard to find those in the courses I've done on LinkedIn and such. Thanks again for the refresher!
You're welcome! Thanks for the insight, Jessie. Glad that you can relate to evaluation as well.
You explain in less than 10 mins what others take hours to. Thank you.
You’re welcome! Thanks for checking out the video 😃
Your videos have been so helpful! I have an ID interview tomorrow! I'm a Learning and Development specialist but haven't focused on the ID side of things. Thank you so much for helping me to prepare for the interview in understanding how I can add to my learning.
You’re welcome! Good luck and please let us know how it goes 😃
Kirkpatrick's Model of Evaluation,..
Checked!! Thank you.. off to the next video...
Keep up the good work, Jackie :D
thanks for your presentation. can you help me how to get that book?
Sometimes the BEST videos are the ones with less views and likes.
Thanks Fernando!
I am moving to the USA. I have experience as trainer and university professor. Do you recommend to offer to design a course to a local business, in my home country, for free in order to build my portfolio?
That could work, Fabian! But you could also create a concept project so that you do not need to rely on unavailable SMEs, unreasonable requests from clients, etc.
This was SO helpful. I was trying to find the best way to summarize evaluations for my students and this video is perfect. Thank you!!
You're welcome! Glad to hear that it helped :)
Great content, thanks! I'm a fresh Instructional Designer and your videos are super useful for quick overviews!
That’s wonderful! Glad to hear it 😃. If there’s other content that you think would be helpful (but don’t see here on the channel), please let me know!
Thanks for the summary. Useful content and I do agree that many instructional designers avoid Levels 3 and 4, but this is where true value lies. It's not as complicated as it may seem. We should be included in planning and done right you're able to measure training impact - leading to improved learning interventions. It's also important to periodically measure your evaluation methods so that you continually improve them.
Excellent information and explained clearly. Thanks.
Glad it was helpful!
Thank you so much!! I just realized that I evaluated only the level one!
That’s very common! The higher up the levels that you get with the evaluation, the more valuable that the data will be 😃
Hope you can add information on criticisms and limitations of the model
Thanks for the suggestion, Patricia!
Thanks for the vid.
Briefly,... Level 3 & 4 are needed for just in time training, however they don't immediately apply to capacity building training, which rely mostly on level 1 & 2 evals.
You’re welcome! Thanks for commenting 😃
Makes me proud.
Haha because you have the same last name? I’d be proud too 😃
Hi Devlin! Is the designing part in the planning process (the 4th point: designing attractive intervention) related to the reaction level (i.e. Level 1)? If yes, then how?
Hi! Yes, exactly. If people find that it's appealing / attractive / engaging, then that's a level 1 reaction :D
@@DevlinPeckYT Thank you, Devlin!
Thank you for a wonderful presentation. Very well explained. 👍
You're very welcome! Thank you for checking it out :)
Thank you, Devlin. Awesome.
You’re welcome! Thanks for watching 😃
Hi Devlin, I have done an ID course but this provides greater clarity to the theory.
I’m glad that it helped!
You have given the essence of the model. Great video! Looking forward to a video on SAM model for both simple and complex projects
Thanks so much, Lakshmi! Glad to hear that you’re interested in a SAM video.
Thanks @Devlin for knowledge in only 9 minutes You have been able to summarize a model perfectly. Thanks! I continue having a little doubt.
How I could show that level 3 and 4 It is a consequence of a training? when It has runned time for example 6 months or 1 year . Thanks in advance!
Good question, Cynthia! I would suggest looking into the success case method for a program that has already been running for a while. This model works best when baked into the strategy and design from the beginning 😃. You could also look at performance and metrics both before the training was implemented and after it was implemented.
@@DevlinPeckYT thank for your answering.. Before test and post test...I am exploring it more. Thanks for your information and congrats for great conversation with Cathy Moore. I am reading this remarkable book... I follow you!! greetings from Peru
@@cynthiareid6416 yes, and good idea! Thanks for supporting the content 😊
Hi
Hi there! Thanks for watching :)
Thank u for the sharing Devlin. I'm wondering any other factors influencing training effectiveness score..for example virtual learning vs physical learning, virtual 2 hours vs full 8 hours..do u have any view? I've been asked to this question a lot by the leaders..
Hello hello! Thanks for checking out the video. Those items would be great things to ask your intended audience (and collect data about how they’re received, how they impact learning, whether people use those new skills on the job at the same rate, etc).
@@DevlinPeckYT thank u for the idea..would be a good start to do survey..the learning platform have just swift from physical to virtual since COVID pandemic. thanks again yaa!
@@MsNedy you’re welcome! 😊
Explained so well. Thank you so much.
This video helped me understand evaluation and the Kirkpatrick model so much.
Super clear and crisp . Thank you :-)
You’re welcome! Thanks for watching and commenting 😃
Level 3 actually requires additional time from SME persons. They should later send the results of whether something has changed in terms of user behavior. In most situations they are very busy and this is rarely delivered unfortunately. In most cases, you will not get these results because of the organization you are in. This also applies to the entire level.
Thanks for the insights! Gathering information for Level 3 can be tricky for sure.
Learning just got easier
Glad that you found your way here!
This content is very timely as this is our topic in my Masters Degree in Human Resource Management. Thank you very much for sharing your personal thoughts on the planning perspective. Will subscribe and hoping to learn more from you. Keep rocking!
So true, easily the best evaluation model extant
So says Randall Kirkpatrick
It’s a good one! 😃
Amazing
Thanks Priyanka!
Thanks.
You’re welcome!
Really helpful summary, thank you
Hi...great explanation and so helpful.can you tell ACTIVITIES for this each 4 level that helps to language learning trainees or teacher? Thanks in advance.
Thanks so much! And I am not sure I understand the question. Are you asking about how to assess language learning?
@@DevlinPeckYT thanks for the reply and yes, like how to assess and example that might suits for language learners.
@@sindhuvelmurgan361 Level 1 pertains to how the audience reacted to the learning experience, level 2 pertains to how much they learned, level 3 pertains to whether or not they're using what they learned in a real-world context, and level 4 pertains to the impact that their newfound langauge skills are having on the organization that they're working for :D
@@DevlinPeckYT Thanks alot.
@@sindhuvelmurgan361 you’re welcome 😃
Thanks so much for this summary!
You're welcome! Thanks for watching :)
Did you make your intro on AE?
Good question. I actually got a template from Envato and then just added my logo to it in AE 😃. And got a separate music file from Envato as well.
@@DevlinPeckYT Nice!
@@flagshipbuilds thanks Faizan!
Hey man, really good production here. Congratz and thanks.
You're welcome. Thanks for checking it out :)
Ch15 right here.
Haha I'm not sure I understand this reference, Faizan :D
@@DevlinPeckYT Lol that was brief. Its from the textbook Leaving ADDIE for SAM.
@@flagshipbuilds ok, good to know. Going to record some SAM content soon!
@@DevlinPeckYT How about content on the AECT standards? My portfolio for my MS in LDT will be judged against it this semester for graduation but NO employer ever mentions it in the job listings which mean they do not know it or its skill over credential or both.
@@flagshipbuilds Good suggestion! Thank you :D
Great videos. It's beyond me as an educator, why we can't graduate better knowledgeable, skilled, and intelligent students with all the tools we have in the United States. This then tells me that it has to be a discipline issue.
Thanks so much! And I'm sure everyone is trying their best :D