As a new learner to the field I found this video very helpful! Some videos I've seen on the Kirkpatrick model try to cram in too much information. I like how this was a general overview of the model and even noted that limitations were not going to be discussed in this video. Short, sweet, and to the point. thank you!👌👌
Your videos have been so helpful! I have an ID interview tomorrow! I'm a Learning and Development specialist but haven't focused on the ID side of things. Thank you so much for helping me to prepare for the interview in understanding how I can add to my learning.
00:00 Kirkpatrick model for evaluation 1:13 Level 1 : Reaction Overall satisfactions and engagement Survey, interviews, focus group 2:05 Level 2 :Learning New Knowledge, skills or attitude Quizzes / tests, interviews, focus groups, discussions, observation, and more. 3:09 Level 3 :Impact Are people performing the critical tasks? Surveys, observation, work review, KPls 4:30 Level 4 :Results Which results do we aim to achieve? What do people need to do differently? What knowledge and skills do they need? How do we design an attractive intervention? 8:33 Conclusion
This is one area where I think a lot of educators would have some experience, especially if there were in any area of curriculum develop or accreditation studies, but it could be helpful to have some more real world examples. It seems to be hard to find those in the courses I've done on LinkedIn and such. Thanks again for the refresher!
Thanks for the summary. Useful content and I do agree that many instructional designers avoid Levels 3 and 4, but this is where true value lies. It's not as complicated as it may seem. We should be included in planning and done right you're able to measure training impact - leading to improved learning interventions. It's also important to periodically measure your evaluation methods so that you continually improve them.
Thanks for the vid. Briefly,... Level 3 & 4 are needed for just in time training, however they don't immediately apply to capacity building training, which rely mostly on level 1 & 2 evals.
The Kirkpatrick model consists of four evaluation levels-Reaction, Learning, Behavior, and Results-where Reaction measures participant satisfaction through surveys, Learning assesses knowledge acquisition via quizzes, Behavior evaluates the application of skills on the job through observations, and Results measures the impact on business metrics, all of which require careful planning that starts with identifying desired results and works backward to define necessary behaviors and skills, while also allowing for iterative adjustments during training to ensure effectiveness.
This content is very timely as this is our topic in my Masters Degree in Human Resource Management. Thank you very much for sharing your personal thoughts on the planning perspective. Will subscribe and hoping to learn more from you. Keep rocking!
Thanks @Devlin for knowledge in only 9 minutes You have been able to summarize a model perfectly. Thanks! I continue having a little doubt. How I could show that level 3 and 4 It is a consequence of a training? when It has runned time for example 6 months or 1 year . Thanks in advance!
Good question, Cynthia! I would suggest looking into the success case method for a program that has already been running for a while. This model works best when baked into the strategy and design from the beginning 😃. You could also look at performance and metrics both before the training was implemented and after it was implemented.
@@DevlinPeckYT thank for your answering.. Before test and post test...I am exploring it more. Thanks for your information and congrats for great conversation with Cathy Moore. I am reading this remarkable book... I follow you!! greetings from Peru
This old model lacks human capital factors, for instance, level four should also include employee retention (as an example: The customer received high level service and needs are met for the employee - Maslow hierarchy of needs)
Hi...great explanation and so helpful.can you tell ACTIVITIES for this each 4 level that helps to language learning trainees or teacher? Thanks in advance.
@@sindhuvelmurgan361 Level 1 pertains to how the audience reacted to the learning experience, level 2 pertains to how much they learned, level 3 pertains to whether or not they're using what they learned in a real-world context, and level 4 pertains to the impact that their newfound langauge skills are having on the organization that they're working for :D
Hi Devlin! Is the designing part in the planning process (the 4th point: designing attractive intervention) related to the reaction level (i.e. Level 1)? If yes, then how?
I am moving to the USA. I have experience as trainer and university professor. Do you recommend to offer to design a course to a local business, in my home country, for free in order to build my portfolio?
That could work, Fabian! But you could also create a concept project so that you do not need to rely on unavailable SMEs, unreasonable requests from clients, etc.
Thank u for the sharing Devlin. I'm wondering any other factors influencing training effectiveness score..for example virtual learning vs physical learning, virtual 2 hours vs full 8 hours..do u have any view? I've been asked to this question a lot by the leaders..
Hello hello! Thanks for checking out the video. Those items would be great things to ask your intended audience (and collect data about how they’re received, how they impact learning, whether people use those new skills on the job at the same rate, etc).
@@DevlinPeckYT thank u for the idea..would be a good start to do survey..the learning platform have just swift from physical to virtual since COVID pandemic. thanks again yaa!
@@DevlinPeckYT How about content on the AECT standards? My portfolio for my MS in LDT will be judged against it this semester for graduation but NO employer ever mentions it in the job listings which mean they do not know it or its skill over credential or both.
Devlin -- I've worked on projects where there's an interest in knowing an ROI to justify the cost of say, narration or video vs just Powerpoint, but there's NO interest in looking at level 2 or 3. Maybe it's compliance or other training where the ROI is a little squishier. Management is only interested in justifying the cost and cares less about the long-term results. How would Kirkpatrick address this? Perhaps that kind of ROI is better evaluated with just infographics on the value of eLearning? Thx.
I think that Kirkpatrick would say that those efforts are unlikely to produce any real results. If the stakeholders are not interested in improving business metrics or actually changing human behavior, then there are few cases where the efforts will make a difference for the audience or the organizaton.
@@DevlinPeckYT Thanks for the reply. I think that for compliance and other BAU training there’s no interest in metrics.. only in compliance. No behavior change if the employees are already in compliance, so it comes down to evaluating the return on the training investment every year to keep the status quo.
Great videos. It's beyond me as an educator, why we can't graduate better knowledgeable, skilled, and intelligent students with all the tools we have in the United States. This then tells me that it has to be a discipline issue.
Hi Travis! I appreciate the support. I referenced this book when I was making this vid to make sure I didn't miss / misrepresent anything: www.kirkpatrickpartners.com/product/Kirkpatricks-Four-Levels-of-Training-Evaluation/
I love how this guy seems annoyed by his own presentation. I could hear him sigh...... hahahaha dude, we're not forcing you to do this, you want to present this information to us 😅🤣🤣🤣🤣🤣
👀👀👀. I must have caught myself on a bad day with this one haha. Definitely happy to present the info but it can be rough some times…especially in these earlier days when I was newer to TH-cam 😂
Level 3 actually requires additional time from SME persons. They should later send the results of whether something has changed in terms of user behavior. In most situations they are very busy and this is rarely delivered unfortunately. In most cases, you will not get these results because of the organization you are in. This also applies to the entire level.
As a new learner to the field I found this video very helpful! Some videos I've seen on the Kirkpatrick model try to cram in too much information. I like how this was a general overview of the model and even noted that limitations were not going to be discussed in this video. Short, sweet, and to the point. thank you!👌👌
I liked how you talked about working backwards from Level 4 to Level 1. It made so much more sense.
Me too, working backwards makes so much more sense as I'm working on Behavioral competency e-learning project
You explain in less than 10 mins what others take hours to. Thank you.
You’re welcome! Thanks for checking out the video 😃
Kirkpatrick's Model of Evaluation,..
Checked!! Thank you.. off to the next video...
Keep up the good work, Jackie :D
Thank so much! I work in professional development and this 10min video has given me so many ideas as to how we can improve our needs analysis!!
Wonderful! I’m glad to hear it 😃
wow tthis was so clear and precise it helped me form my training and development poa
Glad to hear it! Thanks Shubham 😃
Your videos have been so helpful! I have an ID interview tomorrow! I'm a Learning and Development specialist but haven't focused on the ID side of things. Thank you so much for helping me to prepare for the interview in understanding how I can add to my learning.
You’re welcome! Good luck and please let us know how it goes 😃
extremely helpful video. crisp and engaging. thank you!
🙂
Excellent information and explained clearly. Thanks.
Glad it was helpful!
This video helped me understand evaluation and the Kirkpatrick model so much.
00:00 Kirkpatrick model for evaluation
1:13 Level 1 : Reaction
Overall satisfactions and engagement
Survey, interviews, focus group
2:05 Level 2 :Learning
New Knowledge, skills or attitude
Quizzes / tests, interviews, focus groups, discussions, observation, and more.
3:09 Level 3 :Impact
Are people performing the critical tasks?
Surveys, observation, work review, KPls
4:30 Level 4 :Results
Which results do we aim to achieve?
What do people need to do differently?
What knowledge and skills do they need?
How do we design an attractive intervention?
8:33 Conclusion
Thanks for sharing this, Andres!
Great refresher, and yes many times we stop at level 2 but should try to get to levels 3 or 4.
For sure. Thanks for checking out the video :)
This was very crisp and informative. Also, thank you for sharing the amazing list of books :)
You’re welcome! Thanks for watching, Pooja 😊
This is one area where I think a lot of educators would have some experience, especially if there were in any area of curriculum develop or accreditation studies, but it could be helpful to have some more real world examples. It seems to be hard to find those in the courses I've done on LinkedIn and such. Thanks again for the refresher!
You're welcome! Thanks for the insight, Jessie. Glad that you can relate to evaluation as well.
Thank you so much!! I just realized that I evaluated only the level one!
That’s very common! The higher up the levels that you get with the evaluation, the more valuable that the data will be 😃
Thanks for the summary. Useful content and I do agree that many instructional designers avoid Levels 3 and 4, but this is where true value lies. It's not as complicated as it may seem. We should be included in planning and done right you're able to measure training impact - leading to improved learning interventions. It's also important to periodically measure your evaluation methods so that you continually improve them.
So true, easily the best evaluation model extant
So says Randall Kirkpatrick
It’s a good one! 😃
This was SO helpful. I was trying to find the best way to summarize evaluations for my students and this video is perfect. Thank you!!
You're welcome! Glad to hear that it helped :)
Hi Devlin, I have done an ID course but this provides greater clarity to the theory.
I’m glad that it helped!
Great content, thanks! I'm a fresh Instructional Designer and your videos are super useful for quick overviews!
That’s wonderful! Glad to hear it 😃. If there’s other content that you think would be helpful (but don’t see here on the channel), please let me know!
Explained so well. Thank you so much.
Thanks for the vid.
Briefly,... Level 3 & 4 are needed for just in time training, however they don't immediately apply to capacity building training, which rely mostly on level 1 & 2 evals.
You’re welcome! Thanks for commenting 😃
Thank you, Devlin. Awesome.
You’re welcome! Thanks for watching 😃
Next is Philips ROI - Level 5. Information that truly demonstrates value to an organization and what executives will want to hear.
Grab the FREE interactive "Become an ID" checklist: www.devlinpeck.com/become-an-id-checklist
Learning just got easier
Glad that you found your way here!
Super clear and crisp . Thank you :-)
You’re welcome! Thanks for watching and commenting 😃
You have given the essence of the model. Great video! Looking forward to a video on SAM model for both simple and complex projects
Thanks so much, Lakshmi! Glad to hear that you’re interested in a SAM video.
Thank you for a wonderful presentation. Very well explained. 👍
You're very welcome! Thank you for checking it out :)
Really helpful summary, thank you
Thanks so much for this summary!
You're welcome! Thanks for watching :)
Sometimes the BEST videos are the ones with less views and likes.
Thanks Fernando!
Great Summary!
Thanks!
The Kirkpatrick model consists of four evaluation levels-Reaction, Learning, Behavior, and Results-where Reaction measures participant satisfaction through surveys, Learning assesses knowledge acquisition via quizzes, Behavior evaluates the application of skills on the job through observations, and Results measures the impact on business metrics, all of which require careful planning that starts with identifying desired results and works backward to define necessary behaviors and skills, while also allowing for iterative adjustments during training to ensure effectiveness.
This content is very timely as this is our topic in my Masters Degree in Human Resource Management. Thank you very much for sharing your personal thoughts on the planning perspective. Will subscribe and hoping to learn more from you. Keep rocking!
Makes me proud.
Haha because you have the same last name? I’d be proud too 😃
Hey man, really good production here. Congratz and thanks.
You're welcome. Thanks for checking it out :)
Thanks.
You’re welcome!
Hope you can add information on criticisms and limitations of the model
Thanks for the suggestion, Patricia!
well explained.
Thanks 😃
Thanks @Devlin for knowledge in only 9 minutes You have been able to summarize a model perfectly. Thanks! I continue having a little doubt.
How I could show that level 3 and 4 It is a consequence of a training? when It has runned time for example 6 months or 1 year . Thanks in advance!
Good question, Cynthia! I would suggest looking into the success case method for a program that has already been running for a while. This model works best when baked into the strategy and design from the beginning 😃. You could also look at performance and metrics both before the training was implemented and after it was implemented.
@@DevlinPeckYT thank for your answering.. Before test and post test...I am exploring it more. Thanks for your information and congrats for great conversation with Cathy Moore. I am reading this remarkable book... I follow you!! greetings from Peru
@@cynthiareid6416 yes, and good idea! Thanks for supporting the content 😊
Amazing
Thanks Priyanka!
Any weaknesses of Kirkpatrick model when applying to prepare a Training for nurse learners? Thanks
This old model lacks human capital factors, for instance, level four should also include employee retention (as an example: The customer received high level service and needs are met for the employee - Maslow hierarchy of needs)
Thanks John! I can see how that would be an important organizational factor to consider :D
thanks for your presentation. can you help me how to get that book?
Hi...great explanation and so helpful.can you tell ACTIVITIES for this each 4 level that helps to language learning trainees or teacher? Thanks in advance.
Thanks so much! And I am not sure I understand the question. Are you asking about how to assess language learning?
@@DevlinPeckYT thanks for the reply and yes, like how to assess and example that might suits for language learners.
@@sindhuvelmurgan361 Level 1 pertains to how the audience reacted to the learning experience, level 2 pertains to how much they learned, level 3 pertains to whether or not they're using what they learned in a real-world context, and level 4 pertains to the impact that their newfound langauge skills are having on the organization that they're working for :D
@@DevlinPeckYT Thanks alot.
@@sindhuvelmurgan361 you’re welcome 😃
Hi Devlin! Is the designing part in the planning process (the 4th point: designing attractive intervention) related to the reaction level (i.e. Level 1)? If yes, then how?
Hi! Yes, exactly. If people find that it's appealing / attractive / engaging, then that's a level 1 reaction :D
@@DevlinPeckYT Thank you, Devlin!
I am moving to the USA. I have experience as trainer and university professor. Do you recommend to offer to design a course to a local business, in my home country, for free in order to build my portfolio?
That could work, Fabian! But you could also create a concept project so that you do not need to rely on unavailable SMEs, unreasonable requests from clients, etc.
Thank u for the sharing Devlin. I'm wondering any other factors influencing training effectiveness score..for example virtual learning vs physical learning, virtual 2 hours vs full 8 hours..do u have any view? I've been asked to this question a lot by the leaders..
Hello hello! Thanks for checking out the video. Those items would be great things to ask your intended audience (and collect data about how they’re received, how they impact learning, whether people use those new skills on the job at the same rate, etc).
@@DevlinPeckYT thank u for the idea..would be a good start to do survey..the learning platform have just swift from physical to virtual since COVID pandemic. thanks again yaa!
@@MsNedy you’re welcome! 😊
Ch15 right here.
Haha I'm not sure I understand this reference, Faizan :D
@@DevlinPeckYT Lol that was brief. Its from the textbook Leaving ADDIE for SAM.
@@flagshipbuilds ok, good to know. Going to record some SAM content soon!
@@DevlinPeckYT How about content on the AECT standards? My portfolio for my MS in LDT will be judged against it this semester for graduation but NO employer ever mentions it in the job listings which mean they do not know it or its skill over credential or both.
@@flagshipbuilds Good suggestion! Thank you :D
Did you make your intro on AE?
Good question. I actually got a template from Envato and then just added my logo to it in AE 😃. And got a separate music file from Envato as well.
@@DevlinPeckYT Nice!
@@flagshipbuilds thanks Faizan!
Devlin -- I've worked on projects where there's an interest in knowing an ROI to justify the cost of say, narration or video vs just Powerpoint, but there's NO interest in looking at level 2 or 3. Maybe it's compliance or other training where the ROI is a little squishier. Management is only interested in justifying the cost and cares less about the long-term results. How would Kirkpatrick address this? Perhaps that kind of ROI is better evaluated with just infographics on the value of eLearning? Thx.
I think that Kirkpatrick would say that those efforts are unlikely to produce any real results. If the stakeholders are not interested in improving business metrics or actually changing human behavior, then there are few cases where the efforts will make a difference for the audience or the organizaton.
@@DevlinPeckYT Thanks for the reply. I think that for compliance and other BAU training there’s no interest in metrics.. only in compliance. No behavior change if the employees are already in compliance, so it comes down to evaluating the return on the training investment every year to keep the status quo.
@@catboy721 Yeah, compliance training is notoriously one of those "checklist" training items. "Did employees complete it? Then we're in the clear!"
Yes, the same doubts! Thanks for displaying it!
@@cynthiareid6416 you’re welcome!
Great videos. It's beyond me as an educator, why we can't graduate better knowledgeable, skilled, and intelligent students with all the tools we have in the United States. This then tells me that it has to be a discipline issue.
Thanks so much! And I'm sure everyone is trying their best :D
I liked and subscribed please share youre references. espicially google scholar
Hi Travis! I appreciate the support. I referenced this book when I was making this vid to make sure I didn't miss / misrepresent anything: www.kirkpatrickpartners.com/product/Kirkpatricks-Four-Levels-of-Training-Evaluation/
Hi
Hi there! Thanks for watching :)
I love how this guy seems annoyed by his own presentation. I could hear him sigh...... hahahaha dude, we're not forcing you to do this, you want to present this information to us 😅🤣🤣🤣🤣🤣
👀👀👀. I must have caught myself on a bad day with this one haha. Definitely happy to present the info but it can be rough some times…especially in these earlier days when I was newer to TH-cam 😂
Level 3 actually requires additional time from SME persons. They should later send the results of whether something has changed in terms of user behavior. In most situations they are very busy and this is rarely delivered unfortunately. In most cases, you will not get these results because of the organization you are in. This also applies to the entire level.
Thanks for the insights! Gathering information for Level 3 can be tricky for sure.