SAT Variation by Income: The Test Prep Fig Leaf
I was not at all surprised to see that average SAT scores varied strongly by income bracket. What has surprised me is how quickly everyone has grabbed for the explanation that "its all due to test prep." It strikes me that the test prep explanation is a sham, meant to try to hide the real problem.
First, Alex Tabarrok says that most of the research out there is that test prep explains at most 20% of the variation by income, and probably less. This fits my experience with test prep. I have always felt that 90% of the advantage of test prep was just taking a few practice tests so when the actual test days come, the kids are comfortable they understand how each section of the test works and are not thrown by the types of problems they will face. My feeling is that most of what you can learn in fancy test prep courses is in those books they sell for about $40. We sent our kids to a course that cost a lot more than $40, but frankly I did not do it because I thought they would get any special knowledge they could not get in the book, but because I was outsourcing the effort to get them to do the work. Seriously, I think a parent with $40 and the willingness to make sure their kids actually goes through the book would get most of the benefit.
Which raises the question of whether test prep is correlated to income because of its cost, or whether it is correlated to income because high income folks are more likely to place value on their kids testing well and make them do the prep work. We will come back to this in a minute.
So if its not test prep, what does drive the difference? I don't know, because I have not studied the problem. But I can speak for our family. My kids do well on SAT-type tests because they go to a tough rigorous private school. Let's take one example. When my daughter was a sophomore in high school, she scored a perfect 80 (equivalent of the SAT 800) on the writing and grammar section of the PSAT. Now, my daughter is smart but no Ivy-bound savant. She took no prep course. My daughter aced the PSAT grammar because her freshman teacher drove those kids hard on grammar. I am talking about a pace and workload and set of expectations that kids in our junior high school start talking about and dreading two years before they even get to the class, and this at a school already known for a tough work load.
This teacher is legendarily fabulous, so obviously that is hard to replicate everywhere. But she is fabulous because my kids actually came away excited about Homer and other classics. This is what I pay private-school money for. But what she did in grammar, what got my daughter her perfect score, could be emulated by about any competent teacher...theoretically. But in fact it can't happen because such an approach could never survive in a public school. The work expectations are way too high -- parents and students would revolt. It only works for those who self-select.
Well, it only works today for those who self-select and can afford a private school. Unfortunately, we have an education system where everyone is forced to pay tuition to what is at-best a teach-to-the-mean school. If one wants more, they have to be wealthy enough to pay tuition to a second school. Which is why school choice makes so much sense. Why should only the wealthy have the ability to self-select into more intensive programs? BUt this is a conclusion most the education establishment is desperate for people not to reach. Thus, the hand-waving over test prep.
Of course, there are a million other wealth, genetic, and parental effects that come into this equation. For example, my kids read for fun, probably in large part because my wife and I read for fun. How many kids read 10+ books outside of school each year? They do this not because my kids are awesomer than other kids, but simply because that was the expectation they grew up with, that we spend free time reading books. Other families might spend their free time, say, doing home improvement projects such that their kids all grow up great woodworkers. I am not sure one set of activities is superior to another, but my kids end up testing well. Of course, I am not sure they can use a screwdriver. Seriously, over Christmas break I asked my 20-year-old son to pass me the Phillips head screwdriver and he had no idea which one that was.
I was thinking about the question above of how one separates out parental expectations from all the other effects (like parental DNA and income and quality of schools, etc.) I interview high schoolers for Princeton admissions, so I have come to learn that some public high schools have advanced programs, to allow kids some self-selection into a more rigorous program within the context of public schools (this is usually either an AP program, an honors program, or an IB program). By the way, the existence of these programs at public schools correlates pretty highly with the average income of that school's district.
Here would be an interesting study: Take high schools with some sort of honors program option. We want to look at the income demographics of the kids who chose the honors program vs. those who choose the standard program. We would therefore want to look only at high schools that take all comers into the honors program -- if they have some sort of admissions requirement, then this would screw up our study because we want to test solely for how demographics affect the choice to pursue a more rigorous, college-oriented program. I would love to see the results, but my hypothesis is that test-prep is a proxy for the same thing -- less about income per se and more about parental expectations.
mahtso:
Re school choice: today's AZ Rep had an article about a move in Az to provide parents with vouchers that could be used at private schools (rather than making the parent use a public school). The Rep "reports" that this would effectively kill public education.
The article also notes that Az has one of the nation's most open policies with respect to school choice. [Deleted snark about population demographics.]
March 11, 2014, 10:06 amBrad Warbiany:
Yeah, I think the explanation of "prep courses" doesn't hold much water. Why? Because growing up in a typical upper-middle class home is 18 years of SAT test prep.
As an example, recent studies have suggested that professional households expose their children to ~30M more words during the first four years of life than other socioeconomic categories. It's expected that the effects of this are long-lasting: http://centerforeducation.rice.edu/slc/LS/30MillionWordGap.html
Granted, I'm not sure whether it's cause or effect--since intelligence is highly heritable, and professional households likely have higher median IQ than working-class households, it's unclear whether this effect is due to nature or nurture...
But if anything, I think this does just reveal a flaw in the SAT. It truly is the epitome of a culturally biased test. It favors native English speakers, and furthermore only those households where parents displayed an expansive vocabulary. While I believe that an expansive vocabulary is something that we as a society should value in youth, I don't think it's necessarily important to college success unless a student wants to be an English major. The second-generation Chinese immigrant who doesn't know what "peripatetic" means can still be perfectly successful as an engineer or scientist without that knowledge.
March 11, 2014, 10:07 am3rdMoment:
"It truly is the epitome of a culturally biased test."
I don't know what you mean by this. The purpose is to predict who has the ability to do college level work. Do you think it is easy to make a test that does this better than the SAT? How can you tell if a test is "culturally biased?" And do you really think vocabulary is not predictive of who can read and understand advanced texts?
March 11, 2014, 10:58 amErikTheRed:
Sort of on topic - one thing that drives me freaking insane about education is contained in this sentence: "This teacher is legendarily fabulous, so obviously that is hard to replicate everywhere. But she is fabulous because my kids actually came away excited about Homer and other classics."
In what other modern human endeavor of such importance do we reinvent the wheel tens of thousands of times across the country on an annual basis, and then whine about the astronomical cost and generally disappointing results? Have a few (or even a few dozen) great teachers record lessons, figure out which methods reach individual students best, let them switch virtual instructors as needed, and have tutors available when they get deeply and truly stuck. Let kids learn and certify (test) in what they want, or what their careers and / or wanted universities demand. Let different organizations compete at certification. Get rid of grades - as an employer I want to see levels, RPG-style. How much easier would it be to hire people when you can say "OK, must be Level 6 J2EE programmer with Level 4 Perl scripting skills, Level 4 English grammar and composition, Level 3 algebra, Level 7 resistance to workplace butthurt (probably not the real title), etc."?
If a person has a high school diploma, that tells me pretty close to nothing about their utility as an employee or even as a human being. Sadly, even college degrees (outside of certain hard sciences) have devolved to about the same point. Kids may have accomplished a task (learning something useful), but the education system as it exists right now does a next-to-nonexistant job of communicated exactly what they have accomplished.
March 11, 2014, 11:04 amjon:
I was listening to the *Peace Revolution* podcast and it had a section with a talk by John Taylor Gatto. He mentioned how Jaime Escalante got kids in a poor district to learn and pass AP Calculus tests. In the movie *Stand and Deliver* it makes it look like he was able to get the kids to turn around in a single year and improve, but in reality it took several years to prep the kids and him getting control of the math department so he could self select the good teachers from the bad that would prep the kids for the advanced math class. Since he couldn't control the Junior high school (eventually he did a bit) he had to get the community college to have a summer catch up session. By the time he got the program going full blast he ended up quitting because of all the pressure *not* to perform. Reason.com has a good article on it, just look up Escalante in Wikipedia in the *see also* section.
So, in summary, its mostly parents that matter, but good schools and teachers matter too, but when we use the initiation of force to teach the children the children will always be the ones that lose out.
March 11, 2014, 12:12 pmMingoV:
Way back in 1972 I got the highest verbal score ever in my dinky-dork village public school. I actually knew grammar. Not because any of the English teachers taught grammar, but because my German teacher did. You can't understand German grammar without knowing English grammar.
There have been many studies over the years correlating SAT scores with income, education levels of parents, and high school GPA. My recollection is that the greatest correlation is with parental education, which cross-correlates with income. The SAT score correlation with GPA is positive, but not very strong. Mostly because students bored with classes get so-so grades but can get a high SAT score.
March 11, 2014, 2:39 pmFrank513:
I went to a private school, none of the students took prep courses for the SAT. They grew up in well educated families, studied hard and were held to high standards (no easy A's). 10% of the student body were national merit scholars. A score below 1,400 (out of 1,600) was considered mediocre. These kids went to top tier schools (Princeton, Columbia, Yale, etc.),most graduated summa cum laude and went on to illustrious careers.
My kid wasn't gifted by any means, struggled some with writing and analytical thinking. Went to public school and fortunately had a few superlative instructors. Within 2 years a complete turnaround was well under way. Finished with an excellent SAT score and the distinction of being a national AP scholar. No prep courses, no help from me (so much for the theory that it is the parents) other than encouragement.
March 11, 2014, 7:56 pmFrank513:
Forgot to add this comment...education is anathema to collectivism.
March 11, 2014, 7:58 pmjoshv:
And don't forget that intelligence is inheritable and intelligence is highly correlated with income. Coyote is affluent because he's smart, and his kids are smart because he is.
Now there very well may be components of "smart" that are socialized an not inherent. I came from a low income family and was one of the first of my generation to get a college degree. But my parents were no help. They knew nothing about how to get into college, and which colleges were the best, and neither did any of my friends. My highschool counselor was useless because our school sucked. Because of this I probably made a number of suboptimal choices that adversely affected my education and future earning potential.
March 11, 2014, 7:58 pmobloodyhell:
}}} How many kids read 10+ books outside of school each year?
LOL, only 10+? A YEAR?
Cripes, I recall one time checking close to that many books out of the local public library (I would have been 11, 12, maybe 13, tops, just from where I lived and who I recall talking about it with 12 is my closest recollection) -- I'd read one and a half of them by the next day.
March 11, 2014, 8:10 pmCT_Yankee:
A few years back I mentioned to a coworker that my son in high school had 5 A's and a B+ on his report card. He said his mother must have been proud. I responded that she was furious, and wanted me to head straight to the school and get the misunderstandings that lead to the B+ straightened out. Her son, my step son, was still figuring out how US school was different from his native Russia, and the style was unfamiliar. I made him speak to the teacher to figure it out (not a helicopter parent, really wanted the boy to become his own man). He earned good scholarships and graduated college Magna Cum Laude (and now works for a major tech corp). Certainly he is smart, but not unusually intelligent. His mother simply was not going to accept mediocre or good grades when outstanding or stellar were within reach. She is a teacher herself, and she demands achievement, and the kids love her because they know the difference from "everybody gets a trophy for showing up", and someone gets a few encouraging words for really accomplishing something. Younger kids pressure their parents to be allowed to take lessons from her like their older siblings. She challenges them, and they respond. The expectations are high, and the children strive to meet them. Yes, she teaches at a private school, and has some private students. Public school is all about setting the goals low enough to step over comfortably, but my wife sets them where a student has actually achieved something. The students know it, and they seek her out because good parents do not teach their children to want to be "good enough" or "acceptable" when "awe inspiring" is out there somewhere.
March 11, 2014, 9:20 pmjohnson85:
Most of what makes teachers good is not being a good lecturer though. I don't think it'd be a bad idea to use prerecorded lectures from some of the better lecturers, but that's not the same as replicating what makes them a stellar teacher.
March 12, 2014, 7:17 am