Double-Dipping

One of the things that I'm supposed to be learning about for my new job are certain ways of looking at data. I don't yet have any good data from real scientists, so I'm using leftover data from the math department.

The math department has some low-level classes with common finals. If you have low placement and you want to move on to the real math classes that are required by your major, you need to pass the prep class. Passing the prep class requires passing the common final.

Every semester there are some instructors with dismally low class averages. And not just in the evening section or 8am section that was created the week classes started to respond to unexpected demand from late enrollees. We have some people whose students tend to score poorly during the prime-time sections that filled up during pre-registration.

This semester we only had one outlier section. While the "usual suspects" had lower class averages than the pack, they were close. Some of them were so close that they were not statistically significantly worse than some of the good sections. I spent part of the weekend taking a closer look at the numbers.

This course, like most of our freshman-level courses, requires the use of the WebAssign online homework system. We use the "enhanced" WebAssign, which has options for the students to see a flash video of an example being worked, for students to read the appropriate section of the textbook, or for students to chat with a calculus tutor (probably an underpaid calculus tutor in Asia).

A check of the records shows that the only section that didn't use WebAssign was the outlier section with the alarming low class average. The other worrisome instructors complied with the directions and used WebAssign in their classes. That's the thing about these "new, better freshmen" that the administration says that we have. Even though they are not much better at math than the old freshmen we used to have, they are better at following directions, especially if they believe that following directions will help them get a good grade. Most of them will do the WebAssign and watch the "Watch it Now!" and will "Chat about it!" with the online tutor. They're not very good about coming up with their own strategies for success, but if you give them some concrete actions, many of them will do the work. They are still really bad at identifying the main idea on their own or coming up with their own strategies, but they are pretty good about following specific directions. Especially when it's something with instant positive feedback.

But in the outlier section, the instructor didn't seem to have introduced them to WebAssign. And so they didn't seek it out. They didn't know to Watch it! or Read it! or Chat about it!

Looking deeper at the data, it looked like a homework average of around 80% was the cut-off between students who passed the course and students who failed the course.

So then I took my own WebAssign gradebook from my calculus class and threw it into the data analysis software. This is an entirely different class (calculus vs. algebra) and a completely different format (large lecture vs. small class). I asked the software to calculate the homework average that best separated passing students (C or better) from unsuccessful students (C- or worse). The software calculated 81%.

I've let my successors know about this magic number. I've suggested that they should tell instructors to hassle the students whose WebAssign average is below an 80%. I've told the director of our new online version of one of our courses that the target WebAssign grade for passing should be 80%.

Until my new job finds me some good physical science data to analyze with our software, I'll continue to see what sort of information I can wring out of the math department's data.