Steve Holmes researchED Maths and Science Pt1

TL;DR

PhD students judged the mathematical difficulty of various GCSE exam questions and found minimal bias in the judgments.

Transcript

ok so the master that he was a fairly big study well about two thousand items from the current and the reformed gcse's we threw in a few international comparators just sort of bet benchmarking in this instance we had a load of PhD students mass PhD students judging them as Michelle said by getting a lot of judges to look at these items and pull the... Read More

Key Insights

• 🌍 Over 2,000 GCSE exam questions, including international comparators, were analyzed for their mathematical difficulty.
• 🧑‍🎓 PhD students provided judgments on the difficulty of each question, resulting in 43,000 judgments.
• 🧑‍🎓 Statistical models were used to estimate the expected difficulty of each question and compare it to the difficulty experienced by students in the exams.
• 🧚 Minimal biases were found in the judgments, indicating that the questions were generally fair in their difficulty level.
• 🧑‍🎓 Some questions that were judged as easy by the students proved to be tricky for the students, indicating the potential for minor mistakes leading to wrong answers.
• 🧑‍🎓 A few questions judged as very difficult by the PhD students were not as challenging as believed, suggesting that prior knowledge of specific rules made them easy for some students.
• 🧑‍⚖️ Overall, the study found that surface characteristics of the questions did not contribute substantially to biases in judging mathematical difficulty.

Questions & Answers

Q: How were the mathematical difficulty judgments in the study conducted?

PhD students were presented with pairs of exam questions and asked to judge which one was more mathematically difficult. They provided over 43,000 judgments in total.

Q: What was the purpose of comparing the expected difficulty to the difficulty experienced by students?

The researchers wanted to determine if there were biases in judging particular exam board items and understand what factors caused some questions to be perceived as more difficult than expected.

Q: Were any biases found in the judgments of mathematical difficulty?

The study found no substantial biases in judging math questions based on the surface characteristics of the questions. Differences in difficulty were more likely due to the application of knowledge and the way questions caught students off guard.

Q: Did the study analyze the effect of certain features on the difficulty of the questions?

No, the researchers did not analyze the specific features of the questions. They focused on surface features rather than delving into a more cognitive-level analysis of the questions.

Summary & Key Takeaways

• Researchers conducted a study on 2,000 GCSE exam questions, including international comparators, to determine their mathematical difficulty.

• PhD students were asked to judge the difficulty of each question, resulting in 43,000 judgments.

• Statistical models were used to estimate the expected difficulty for each item and compare it to the difficulty experienced by students during the exams.