May 6, 2009

An education bubble? Data from the explosion of AP tests

A simple but powerful way to determine whether or not there's a irrational bubble is to look for a lot of people who are participating in a trend who have no business doing so. For instance, a Mexican strawberry-picker making $15,000 a year who gets a $720,000 loan for a home. If these don't-belong-there people make up a larger and larger fraction of all who get loans, that strongly suggests that everyone is trying to get in on a speculative bubble -- and that the gatekeepers of the activity are increasingly debauching their entry standards to accommodate the losers.

One datum that suggests an irrational bubble in education is that a much larger fraction of the population is going to college now, and that not surprisingly the average IQ of college students has declined by about 2/3 s.d. -- admissions boards began to scrape deeper down into the sludgebucket of society.

How about looking even earlier? High school is compulsory, so we can't really use high school enrollment to judge whether there's a bubble or not. But what about the sub-group of high school that ostensibly is there to prepare college-bound students for college? That is up to the choice of students, perhaps being bullied by their parents. There is strong evidence even at this early stage of an irrational bubble.

What got me thinking about this was a recent NYT article on how teachers feel about the Advanced Placement program, which is based on a report from the Thomas B. Fordham Institute. The key item that popped out was the claim that participation in the AP program has exploded in recent years, and that this has made a fair fraction of teachers anxious about whether there are students there who shouldn't be. This sure smells like a bubble.

First, let's make sure that the AP program really is exploding as they say, and then we'll see if there's a rational basis for it or not. To measure participation in the AP program, I simply took the number of AP tests taken and divided it by the high school population size. (The AP data are here, and the high school pop data are here, Table A-1.) The AP data go back to 1988, while the high school pop data end in 2007, so I looked at the period from 1988 to 2007. Here are both the total number of AP tests taken and the per capita rate:

An exponential trend accounts for 99.8% of the year-to-year variation for the total number of tests taken, and 99.2% in the per capita case. So, clearly participation in the AP program has been exploding at least since 1988.

Now, is there a sound basis for this increase -- like, maybe kids these days are just getting exponentially smarter? Without looking at the data, we know this is wrong since the main determinant of doing well in AP classes is IQ, and that is influenced mostly by genes and unpredictable aspects of the environment, which haven't been changing so rapidly from one year to the next.

Turning to data on how well 17 year-olds are doing academically, let's look at some tables from the 2007 version of the Digest of Education Statistics (all under Chapter 2, and then Educational Achievement). Table 112 shows that on the National Assessment of Educational Progress, the average reading score for 17 y.o.s did not change from 1971 to 2004. Table 115 shows that the percent of 17 y.o. students who are at the 300 level or above in reading did not change from 1971 to 2004. Tables 125 and 126 show the same lack of change for math skills tested by the NAEP. Table 135 shows that the average Critical Reading score on the SAT did not change from 1988 onward -- indeed, it was steady back to about 1976, and had been declining before then. There was a modest uptick in Math scores (15 points, or 0.15 s.d.). The Critical Reading or Verbal score is more highly g-loaded than the Math score for the SAT, or is a better measure of IQ, which means the apparent uptick in Math scores may not mean as much as we'd think.

Taken together, these data show that the academic fundamentals of high schoolers has not changed since the 1970s. If there has been no upswing at all in the fundamentals -- let alone an exponential one -- then the explosion of the AP program is accounted for completely by irrational factors. It seems just like the housing bubble -- the size of deserving borrowers didn't explode, so the surge in borrowing must have been due to a bunch of undeserving people pouring into the building, namely low-income people. Here are two graphs showing that this happened in the AP program too:

The first shows the distribution of AP scores, where 5 is greatest. You can check the numbers for yourself in the previous link to the AP data, but there has been no change in the percent of all tests that received a score of 4 or 5 -- there have not been more and more smarties piling into AP classrooms, at least not since 1988. Therefore, everyone who deserved to be there was already there. However, the percent of all tests receiving a score of 1 -- telling the student, "why did you even bother?" -- has doubled from 10% to 21%. Those receiving a 2 shrunk a tiny amount, from about 23% to 21%. But those receiving a 3 declined from about 32% to 24%. This means that, unlike for smarties, more and more dummies have been allowed into the AP program.

This is reflected in the change in the mean and standard deviation of test scores: keeping the smarties fixed while adding a lot more dummies will drag down the mean and increase the heterogeneity or variance. That's analogous to the housing bubble causing a decline in the mean creditworthiness of the population of borrowers, and an increase in their heterogeneity, as both the sound and the unsound begin to rub shoulders in loan offices. And just as lenders increasingly cheapened their standards by not requiring down payments or proof of income, so high school teachers and administrators have allowed increasingly ill-prepared -- stupid -- students into the AP program.

In sum, there is very strong evidence from AP tests for a speculative bubble in education. Most of what I've read on whether or not such a bubble exists has focused on college -- soaring tuition, more and therefore dumber students, and so on. These data, though, show that the mania extends even to high school, not just higher ed. For at least the past five years, there have been many news stories about competitive admission to pre-school, so perhaps someone could dig up some numbers to show an exponential increase there too that can't be rationalized by a change in fundamentals. In any case, it's clear that this bubble is much more general than the college data suggest.

Curiously, the phrase "education bubble" has not appeared at all in the NYT, although it has appeared many times in the blogs that the newspaper hosts. Googling the phrase gets 39,000 hits. Rises and falls in tuition get plenty of coverage, but that doesn't show that the reporters are aware of the irrational bubble -- they just think it's unfair, that college should be cheaper so that more can attend. But just as no one was allowed to say that most low-income borrowers were undeserving of home loans since they were disproportionately black and Hispanic, so we aren't allowed to say that a lot of college students are nowhere near being "college material" -- that would violate the "demotic life and times," as Jacques Barzun has dubbed the zeitgeist from roughly the 1960s until today. We cripple our minds by imbibing political correctness.

The bursting of the education bubble may be decades away -- it sure has been going on for awhile, so its period may be much longer than that of the housing or stock market bubbles. Let's just hope that when it happens, it will turn out that hedge funds and investment banks won't have exposed themselves to all of this silliness, and that we won't be plunged into another multi-year recession.

No comments: