I'm disappointed but not surprised by this video. Yes it captures the basics of writing test questions. But it lacks the first step. Make sure your test and its questions are related to measurable objectives. Each question should relate back to one or more objectives. "Easy" or "hard" isn't the point once you have objectives. The point is did the individual meet or not meet a learning objective - not an outcome - but an objective and yes there is a difference. When we simply create questions based on content tests can be lengthy, empirically troublesome, unjust, unfair and more simply - they may not meet your goal: to measure knowledge and attitudes. Also in creating multiple choice questions you need to be mindful of creating questions just to create questions. Inexperienced test makers will do this. More is better. Hard is better than easy. There is no real easy or hard. If your questions are hard but not progressively more difficult you will simply exhaust your learner. But if your test questions correctly assess what the learner knew before and after the "treatment" (the transfer of knowledge through e-Learning, lecture, group study, reading, etc.) then the test indeed is easy for learners who have fulfilled the objectives for each learning item. If your learning has no objectives and there is no logical tie between the knowledge they must learn and the test question - the question and the test maker fail - not the learner. Creating multiple choice questions can be very difficult. But if your learning objectives don't flow from the inception to the end the test will be impossibly difficult and suddenly test takers and those with profound knowledge of skills other than the knowledge at hand will succeed while average learners will fail. Good up front analysis and excellent learning objectives are absolutely essential for the multiple choice test. Other test item types don't require the kind of logic and reasoning that multiple choice does. Also remember that succeeding at multiple choice items doesn't mean your learners have actually learned. Vary the test items so that all learners have an opportunity to achieve a passing score without excelling at any one type of test item (or its logic). This will help to insure the statistical reliability of your test. You will see a nice, predicable pattern when you output the test results for each test item. Be sure to use a focus group of some kind to test the questions - don't use real learners because this is very unfair. Once you see statistical grouping - everyone fails one particular item or item type reassess your multiple choice stem and the choices you've given.
Of course questions need to assess achievement of the learning objectives! While the video touches briefly on this in tip #10, it could definitely stand to elaborate a bit more on this all-important aspect of assessments. A well-written question will stymie an unprepared test-taker because it effectively measures achievement of one or more learning objectives (assuming she doesn't just guess the correct answer - a downside of multiple-choice questions).
So true about the longer, more detailed answers usually being the right ones! I've used that trick before when taking tests.
best video ever
I'm disappointed but not surprised by this video. Yes it captures the basics of writing test questions. But it lacks the first step. Make sure your test and its questions are related to measurable objectives. Each question should relate back to one or more objectives. "Easy" or "hard" isn't the point once you have objectives. The point is did the individual meet or not meet a learning objective - not an outcome - but an objective and yes there is a difference. When we simply create questions based on content tests can be lengthy, empirically troublesome, unjust, unfair and more simply - they may not meet your goal: to measure knowledge and attitudes.
Also in creating multiple choice questions you need to be mindful of creating questions just to create questions. Inexperienced test makers will do this. More is better. Hard is better than easy. There is no real easy or hard. If your questions are hard but not progressively more difficult you will simply exhaust your learner. But if your test questions correctly assess what the learner knew before and after the "treatment" (the transfer of knowledge through e-Learning, lecture, group study, reading, etc.) then the test indeed is easy for learners who have fulfilled the objectives for each learning item. If your learning has no objectives and there is no logical tie between the knowledge they must learn and the test question - the question and the test maker fail - not the learner.
Creating multiple choice questions can be very difficult. But if your learning objectives don't flow from the inception to the end the test will be impossibly difficult and suddenly test takers and those with profound knowledge of skills other than the knowledge at hand will succeed while average learners will fail. Good up front analysis and excellent learning objectives are absolutely essential for the multiple choice test. Other test item types don't require the kind of logic and reasoning that multiple choice does. Also remember that succeeding at multiple choice items doesn't mean your learners have actually learned.
Vary the test items so that all learners have an opportunity to achieve a passing score without excelling at any one type of test item (or its logic). This will help to insure the statistical reliability of your test. You will see a nice, predicable pattern when you output the test results for each test item. Be sure to use a focus group of some kind to test the questions - don't use real learners because this is very unfair. Once you see statistical grouping - everyone fails one particular item or item type reassess your multiple choice stem and the choices you've given.
Of course questions need to assess achievement of the learning objectives!
While the video touches briefly on this in tip #10, it could definitely stand to elaborate a bit more on this all-important aspect of assessments. A well-written question will stymie an unprepared test-taker because it effectively measures achievement of one or more learning objectives (assuming she doesn't just guess the correct answer - a downside of multiple-choice questions).