The Right Data to use for Students' Placement

The Right Data to use for Students’ Placement

Hi Rick-
Our most recent thoughts revolve around the sole use of data to generate a placement. In prior years we have also relied heavily upon teacher recommendations as a factor. The question for me: what type of data is likely to give us the best predictor for high school success? Formative or summative? Our current formulas use 75% of summative-type assessments as the data points by which we generate a placement. I think that may be a bit high, as these summative pieces are mostly "snapshots," rather than a look at growth over time. Any thoughts you can lend or ideas on good reads that could help us make the best decision for kids would be appreciated.

Rick's Response:
Which and how much assessment you use to make such significant decisions really depends on the quality of the data coming from the assessments. Can you claim that the evidence you've gathered is clear, undistorted, and ample? Is it an accurate report of what the student knows and is able to do regarding the standard, objective, or learning outcome? Are you teachers well-trained in this grading method and fully prepared to defend the data they've obtained? This means that the method has been tested over time, too, not just at one sitting. Be concerned if the one data point is from a single performance, such as one final exam score. You want to use clear and consistent evidence over time. I would look for multiple representations of the student's proficiency, not just one.

 

You also want to make sure that everyone is in agreement about the evaluation criteria. If we just say the student "understands the Treaty of Versailles," for example, we don’t know what "understands" means. Can the student just repeat the Wikipedia descriptions? Does she know how the treaty affected German and Allied relations from World War I to the modern day? In retrospect, do some of the treaty's declarations fall short of their goal, or do they still hold up today? Teachers must be able to identify the top 10 to 12 non-negotiable standards and what they mean in every subject.

A collective portrait of a middle school student's readiness for high school courses could include: teachers’ recommendations; analysis of students’ work against subject standards; pre-requisite testing; outside the school testing; maturation scales designed by the high school teachers (capacity to work independently, takes initiative, analyzes tasks well, collaborates with others, is able to extend ideas into other areas, etc.); portfolios of collected work; observations; and central tendency in grades (not averages!). Any one of these by themselves would distort the accuracy of the operative data, and decisions based on that data would be less effective.

Encourage your teachers to eliminate formative data in their decisions, unless the information concerns character and work habits. Summative data is best to use for academic indicators, but we can only use the data if we trust the scores/grades. Do you? Summative assessment can be done over time, such as when we ask a student to write an essay in October, ask him to write another in December and another in March. The versions he writes most recently will reflect the knowledge and skills he has carried forward. The grade on the first essay shows what he could do when immersed in the initial learning experience, but the testimony of his readiness for high school courses demonstrates what he could do at the end of the year. Many high school teachers have to re-teach their students middle school material because the middle school grades were false indicators of what students’ carried forward, or they were tainted by homework completion averages that included extra credit for service projects such as bringing in canned food for the school food drive. Again, do you trust your teachers’ grades to be accurate, undiluted indicators of mastery over time? Are they criterion-referenced, not norm referenced? If so, then you can count quite a bit in your decision.

I hope this is helpful. Without being there to look at specific data, it’s hard for me to know what to advise, but this is what I’d be thinking as I walked into the discussions at your table. Thanks for wrestling with all this. -- Rick