The PISA Rankings and the Role of Schools in Student Performance on Standardized Tests—Posner

International comparisons are tricky, as Becker points out, but the PISA (Programme for International Student Assessment), which tests 15-year-olds for proficiency in reading, math, and science by well designed standard tests conducted in thousands of schools all over the world, is a careful and responsible program, the results of which deserve to be taken seriously. The latest results (which are for 2009) reveal among other things that although the United States spends more money per student on secondary school education than any other country except Switzerland and Austria, Americans’ performance on the PISA tests is mediocre. In the latest tests Americans ranked 17 in reading, 24 in science, and 30 in math. 15-year-old kids in East Asian nations (including Australia and New Zealand), along with Finland, Switzerland, the Netherlands, Belgium, and Canada, outperform the United States in all three subjects. Since 2000, when the PISA tests were first given, the United States has fallen in rank in reading and science, and is unchanged in math.

The rankings tend to be interpreted as measures of the quality of a nation’s pre-collegiate school system (primary and secondary education, since primary education influences performance in secondary schools). But this may be a mistake. Schooling is only one, though doubtless an important, input into performance on the PISA tests. Another is IQ. There have been some efforts to compare IQ across countries, notably by Richard Lynn and Tatu Vanhanen; see their 2006 book IQ and Global Inequality. Their results cannot be regarded as definitive, given significant limitations in the data, but they are suggestive. The authors find that the East Asian countries, which generally rank highest on the PISA tests (including reading—not just math and sciene), have the highest average IQs; the average IQ of Americans is lower because of our large black and Hispanic populations, which have lower average IQs than whites and Asians.

IQ is understood to reflect both genetic endowment and environmental factors, particularly factors operative very early in a child’s life, including prenatal care, maternal health, the educational level of the parents, family stability, and poverty (all these are correlated, and could of course reflect low IQs of parents as well as causing low IQs in their children). The case for very early intervention in children’s development, powerfully urged by the distinguished University of Chicago economist James Heckman, can be understood as an effort to lift IQs in the black and Hispanic communities and by doing so improve the educational performance of black and Hispanic children, including performance on the PISA tests. It is true that Heckman emphasizes noncognitive skills that facilitate learning, but these skills could also increase performance on IQ tests, indicating a positive effect on IQ.

The 2009 PISA test scores reveal that in American schools in which only a small percentage (no more than 10 percent) of the students receive free lunches or reduced-cost lunches, which are benefits provided to students from poor families, the PISA reading test scores are the highest in the world. But in the many American schools in which 75 percent or more of the students are from poor families, the scores are the second lowest among the 34 countries of the OECD; and the OECD includes such countries as Mexico, Turkey, Portugal, and Slovakia.

If IQ is playing a significant role in America’s mediocre showing on the PISA tests, improvements in secondary school education are unlikely to have dramatic effects. The white and Asian kids in American schools are already doing fine, for the most part; the black and Hispanic kids may not do much better until their early childhood environment is improved to the point at which black and Hispanic IQs are raised significantly.

Analysis of the PISA results has revealed some other interesting facts. One is that higher teacher salaries dominate small class size as a factor in high PISA scores. This is a reassuring finding because it suggests that secondary school education can be improved at no net increase in cost, since higher teacher salaries are offset by larger classes—if class size is raised in proportion to increases in teacher salaries, there is no net increase in the school’s cost, and there should actually be a reduction in cost in the long term because a reduction in the number of classrooms reduces the size and therefore cost of a school even if each classroom is larger. Another reassuring finding, in light of all the agitation over charter schools and voucher systems, is that private schools on average do not outperform public schools after adjusting for the quality of students upon entrance and that competition for students does not seem to improve average performance either. Of course these are generalizations across many countries and America’s individualistic culture may not fit them.

This observation is especially pertinent to another finding in the PISA report, which is that poor kids do better in a school that has mostly middle-class kids. Our education system, both public and private, tends as a practical matter to be segregated according to family income and social class. This is a reflection of economic inequality, which is great in the United States and growing.

Becker points out that despite the imperfections of its educational system, America remains preeminent in innovation. This is important but it appears to be due in part to the nation’s attractiveness to immigrants. Many of our innovators are foreign born; increasingly they are Asian. The United States, as a result of generous immigration policies in the nineteenth and early twentieth centuries, also has the largest Jewish population in the world—larger than Israel’s, and a higher percentage of American Jews are Ashkenazi—that is, descendants of European Jewish immigrants. This is significant because Ashkenazi Jews have a significantly higher average IQ than other Americans, including (though the margin is small) East Asians, and, as important, a very strong cultural orientation toward high achievement in business, science, and intellectual fields generally. From the standpoint of innovation, a wide distribution of IQs is more important than the average IQ, because most innovations will come from persons with an above-average IQ, and in scientific and other technical fields from persons with a way-above-average IQ. Moreover, because of the bell shape (normal distribution) of IQ across persons, a higher average IQ translates into a much longer upper tail—the part of the distribution that contains the highest IQs.

If as I am speculating (and I emphasize that it is speculation), IQ is a major factor in school performance, we should hesitate to place too much weight on variance in educational investments, methods, etc. in explaining differences in that performance, relative to genetic and cultural factors and also (and importantly) to economic inequality.

Comments

[Christian Brockman]

I know some attempt has been made to isolate the effects that first and second generation immigrants have on our educational system. See http://www.realclearpolitics.com/articles/2010/12/28/who_owns_the_future_108368.html (making an attempt at this without having access to the underlying data). While both Prof. Becker and Judge Posner address PISA through the lens of standardized testing, does anyone know if anyone else who has access to the actual data has tried to make an apples to apples international comparison of schools with substantial first and second generation immigrant populations (and likewise, those without such populations). The initial attempts at this seem to say that the US is actually performing quite well, but I would love to read more rigorous analysis if anyone had done some.

[Christian Brockman]

Sorry, I meant to link to http://super-economy.blogspot.com/2010/12/amazing-truth-about-pisa-scores-usa.html

Pat Buchanan attempts to use PISA to make a racial (and a racist) argument about our schools. The link in this post attempts to avoid the racial proxy (while not always succeeding) to figure out the relationship between first and second generation immigrants and PISA results.

[RK]

Estimates for the number of Jews in the US are varied, but all the recent ones I've seen count more Jews in Israel than in the US.

[GreeceRUS]

“One [finding in the PISA scores] is that higher teacher salaries dominate small class size as a factor in high PISA scores.”

--------------------------------

Mr. Posner,
Thanks for pointing out the obsession with class size. As if whether one gets to ask one question per day in a 20 student class is really that much different than asking one question per week in a 300 student amphitheater.

One could learn a lot more by e.g. viewing the Faynman Lectures on Physics on an $30 DVD (including intellectual property rights) than having personalized instruction in a one student per class setting taught by the average instructor earning 80K/year.

Most teacher based instruction is obsolete in terms of efficiency, when learning per dollar spent is taken into account. Web or other electronic media based instruction from a top teacher, at a really trivial cost per student, beats a 10-20 student class taught by a mediocre teacher, any day. However, the school environment is 40 years behind the times, as it is primarily driven by the interest of teacher’s unions whose primary goal (as with every union) is to extract the most pay for the least amount of work. But since even that goal is pursued myopically, in the longer term, even that goal is defeated as the entire industry dominated by the union becomes obsolete and fades away (but that is another story..).

Teaching is both an instinctive occupation as well as the first profession most people are exposed to. Hence there is a huge supply of people wanting to be teachers. But few do actually make good teachers. Overall, instruction dominated by online or other electronic media based teaching from top teachers supplemented by scant instruction in a class setting reserved for group activities, would outperform standard education and at a small fraction of the cost. However, we are not free to choose this more modern and efficient model. Our money is sequestered to feed the public education dinosaur regardless of what we choose for our own children.

[Jack]

I wonder about IQs. Mine spiked over 30% after taking 4-years years of standardized tests (actually the "CFA" exams). For many questions on the IQ, I remember thinking: this is "how" to answer the question based on something learned at some point in my life. It didn't feel genetic at all, it felt like a speed test using a collection of learned mini-skills. And of course I took it seriously this most recent time, and not sure how serious I was when I was in grade school.

[Observer]

Standardized IQ tests are a poor measure of innate intelligence because they do not control for self-discipline and motivation on the part of the kids who take the tests.

[J.A.]

@GreeceRUS

Your comment lacks any support whatsoever. Is there any evidence that shows the alleged superior efficiency of your proposed modern system?

There are evidences that countries where the teaching profession is more valued perform better in test such as PISA, e.g., South Korea. There are many factors that influence learning and performance, but haven't been able to point out any in your lengthy and void post.

[Richard]

This was all stated much more explicitly in The Bell Curve. There's no reason on earth to think that after some humans left Africa 50-100,000 years ago racial groups happened to diverge on practically every visible physical trait but evolution did no further work on the mind.

[Out]

The family. We were a strange little band of characters trudging through life sharing diseases and toothpaste, coveting one another's desserts, hiding shampoo, borrowing money, locking each other out of our rooms, inflicting pain and kissing to heal it in the same instant, loving, laughing, defending, and trying to figure out the common thread that bound us all together. ~Erma Bombeck

[wireless headphones]

I must say, you've got one of the best blogs Ive seen in a long time. What I wouldn't give to be able to create a blog that's as interesting as this. I guess Ill just have to keep reading yours and hope that one day I can write on a subject with as much knowledge as you've got on this one!

[Jack]

"The white and Asian kids in American schools are already doing fine, for the most part; the black and Hispanic kids may not do much better until their early childhood environment is improved to the point at which black and Hispanic IQs are raised significantly."

.......... By looking at the forest as "average" we miss a lot. For example while "on average" we spend more per student than many other nations. But! in most of our states and cities school funding is closely related to property taxes - thus we've the problem of inequitable and inadequate funding in the areas of low income and low property values. In San Antonio the difference from one of TX over 1,000 districts was well over 100%.

Nearly every state has had lawsuits over this issue with whoever speaks for "the state" fighting for years to maintain the "right" to prejudicially short fund the low income areas. None of this is new and is a major reason why the Feds got into the school funding biz, though the Fed component is only 8%.

Today, TX (representing nearly 10% or our population) MO, NY and others have, finally, lost all of their appeals and are operating under court mandates to improve the equity and adequacy of K-12 funding.

Posner peers into the woods and notes "white and Asians" are "already doing fine; for the most part". He could as well peer into the suburbs and find that "blacks" and Hispanics in those areas are doing pretty well too. He could then peer into the areas of substandard funding with worn out schools and aging textbooks and find "whites, blacks" and Hispanics not faring well in both urban and rural areas.

One "edge" and perhaps at times a drawback the UK has by comparison is that of having one school district which is funded equitably from throughout the nation. They have long done benchmark testing as well. But what they do with the results is send a SWAT team out to a lagging school to find out what is going wrong. In essence, and contrary to some of the racists posting here, they assume the incoming students will hew close to the average and if a class or school is lagging that something is wrong with the process.

Posner is right to note MOST "whites" and Asians are doing well, and IF we truly wanted to improve our average we'd have to commit to providing, at least, equal resources in the low income areas, if not more as the educational challenges there are often tougher.

How did it work out in Texas several years ago when the courts mandated a more equitable? Instead of gracefully evening up the funding with state funds the bitter legislature made a great show of taking funding from areas where homes were over $300,000 and sending that to the low income areas --- leading to the divisive term of "Robin Hood".

Below is a site where you can track how your state is doing at beating back the forces of equitable and adequate funding for ALL of our kids:

http://www.schoolfunding.info/states/state_by_state.php3

[Terry Bennett]

First, hats off to Judge Posner for waltzing effortlessly right down the middle of this minefield of political correctness.

IQ is generally taken to be a measure (however crude) of some base, mostly immutable quality of the individual, i.e., a gift or talent rather than a learned skill, perhaps akin to processor speed in a microchip. Americans, independent bunch that we are, admire talent. The legend of Bill Gates is enhanced specifically because he dropped out of Harvard; curiously, in many other cultures that decision meets with disdain.

Consider the nexus represented by the Philippines (where my native wife and I have a second home). This country, perhaps the most inextricable blend of East and West, is part of the economic Third World, and simultaneously among the intellectual elite. Their literacy rate, even in English, is higher than ours, on $2k a year per capita. (For context, when Marcos took power they were the #2 economy in Asia, behind Japan; when he left power, they were dead last, behind Bangladesh.) While they struggle economically, their social value system puts great stock in education. In fact, they almost do not recognize IQ. I constantly hear them express sentiments such as, "He's not smart; he didn't go to school", or its inverse, as if education and intellect are the same thing. This outlook seems to prevail in other Eastern countries as well. My 6-year-old Filipino nephew gets on the bus at 7:30 a.m., returns after 5, and has homework - but, he can already communicate with me in English, somewhat. Even the poorest of families spend every peso they can gather on private schools.

As an American, I think school was the biggest waste of time of my entire life. That may be because I'm lazy, or because I'm white and apparently gifted enough that I didn't need much help in learning how to read and do arithmetic. For the short term, we can continue paying poor people to do our math and science for us, either through outsourcing or H1B visas, and our collective business administration acumen will probably be enough to sustain our standard of living. Perhaps we are starting to see the poor harvest of our attitudes in the difficulty of restructuring our current economy to find enough jobs for the unskilled. We could ameliorate this by restricting unskilled immigration, in both its legal and illegal forms, but this would simultaneously require scaling back our welfare state to make our resident unskilled individuals poor enough that they become willing to pick lettuce and sweep floors. Of course, if we do that it may motivate them to get more education...

[rose]

Good post. More related discount information up to 50%-80% at http://www.rosettastoneca.com/ , http://www.toysgiftsrus.com

[Jack]

Terry: Interesting comparison. Korea too, many decades ago had literacy rates higher than the US with a fair percentage trying to learn English which I'm sure served them well during their rapid growth phase.

I doubt that we can limit immigration of "unskilled" while living next door to an impoverished nation now wracked by the further corruption from the industry of supply our drug needs.

IF we could, one effect (given normal rates of unemployment) would be that of an increase in pay for both the ag workers, as well as the QUITE skilled who've been doing most of the work in our housing and commercial construction industry for crappy wages and no benefits.

But! so far even the most ardent "border closers" and fence builders, speak of "guest workers" who'd be expected to live here like lone slaves w/o families and quietly return to their impoverished home land once they'd given us the benefits of their labors.

The discussion brings up the issue of food being cheap for Americans, and why we'd expect our lettuce, tomatoes, fruits etc to be picked for $80/day under tough conditions. Something to think about; are we always dependent on a class of slave labor? And surely in an age of robotics if wages for pickers were to rise, the real American way would seem that of investing in more productive machinery, something that is not spurred by $7 labor.

A recent book compared Germany with the US with one feature being that of Germany finding jobs for those of lower education as their average education is no higher than our own. Not sure exactly what they do but it seems apprentice programs and voc ed is a part of it.

Not easy problems!

[jflycn]

This is an awful post. The IQ differences among groups are very very small, and they are incomparable to score differences. The author didn't know anything he's talking about.

[Taobao English]

Fantastic night shot with great colors and reflections. Happy new year!

[Kris]

All PISA results are broken down by countries. Public attention concentrates on just one outcome: achievement mean values by countries

[Coach Purses]

Too many people live too much in the past. The past must be a springboard, not a sofa.

[Byron M. Roth]

In my book The Perils of Diversity: Immigration and Human Nature, I cite a study by Erling E Boe and Sujie Shin, "Is the United States Really Losing the International Horse Race in Academic Achievement?” (Phi Delta Kappen, 86,9, May 2005, 688-695, online at: http://www.lowellamrine.com/wbqust/documents/BOE.pdf. Quoting the authors:

“As seen in Figure 4, achievement scores of white students in the U.S. were consistently higher than those of students in the Western G5 nations, even though these students were predominantly white. By comparison, the scores of U.S. black and Hispanic students were very low and well below those of the other nations. This is compelling evidence that the low scores of these two groups of minority students were major factors in reducing the comparative standing of the U.S. in international surveys of achievement. If these minority students were to perform at the same level of white students, the U.S. would lead all the other G7 nations (including Japan) in reading and would lead the Western G5 nations in mathematics and science, though it would still trail Japan in these subjects.”

[Tom V]

Christian Brockman:

What's a proxy for what again? First-generation Chinese Americans are doing fine academically, whereas tenth-generation African Americans are not.

[Dan]

In the 10,000 Year Explosion, Cochran and Harpending argue that those who evolved over the last 10,000 years in agricultural societies (Europeans and NE Asians) have higher IQs while those who evolved in hunter-gatherer societies have lower IQs.

[JL]

"This is an awful post. The IQ differences among groups are very very small, and they are incomparable to score differences. The author didn't know anything he's talking about."

You're the one who doesn't know what they are talking about. On IQ tests, white Americans outscore (on average) black Americans by about 1 SD, and Hispanics by about .7 SD, whereas Asian Americans outscore whites by about .3 SD. In the PISA 2009 mathematics test, for example, the US scores were as follows:

515 white
423 black
453 Hispanic
524 Asian

The SD in the PISA tests is about 100, so we can see that the white-black gap is .92 SD, the white-Hispanic gap is .62 SD, and the Asian-white gap is about 0.1 SD. Therefore, the PISA racial gaps are similar but somewhat smaller than those in IQ tests.

[JL]

"The case for very early intervention in children’s development, powerfully urged by the distinguished University of Chicago economist James Heckman, can be understood as an effort to lift IQs in the black and Hispanic communities and by doing so improve the educational performance of black and Hispanic children, including performance on the PISA tests. It is true that Heckman emphasizes noncognitive skills that facilitate learning, but these skills could also increase performance on IQ tests, indicating a positive effect on IQ."

The reason why Heckman emphasizes non-cognitive skills is that he knows that it is very very difficult to produce permanent IQ gains. There have been many experiments and costly programs that were aimed at increasing the IQs of poor minority children, but their results are very meager.

IQ and school achievement are indeed very closely related. I recommend these two papers:

http://www.elsevier.com/authored_subject_sections/S05/S05_357/top/intel.pdf

http://www-classic.uni-graz.at/pslgcwww/rindermann/publikationen/07EJPall.pdf

[BO_Bill]

2009 Reading Score Data

American schools, despite Arne Duncan’s claims, are performing well. US-educated Asians scored 541 in contrast with the Japanese, who scored 520. US-educated whites scored 525 in contrast with Germans, who scored 497. US-educated Hispanics scored 466 in contrast with Mexicans, who scored 425. No data was apparently collected in sub-Saharan countries.

[Mike H.]

“the black and Hispanic kids may not do much better until their early childhood environment is improved to the point at which black and Hispanic IQs are raised significantly.”

Unfortunately as IQ is likely only 20% environment at best(I highly doubt that myself), there can be no chance of a “significant” IQ raise short of genetic manipulation, which is impossible at the present. The Minnesota Transracial Adoption Study proved this beyond the shadow of a doubt. Black, mulatto and white children all raised by rich, intelligent, white parents.

The IQ deviation was still the same, Whites - Mulattos - Blacks.
http://en.wikipedia.org/wiki/Minnesota_Transracial_Adoption_Study

The black children had an average IQ of 89 which means their IQ only rose from the average of 85(please note - this is the average of AMERICAN blacks, who are roughly 15-20% european on average) by four points when their environment was controlled.

The adopted white children were at 106, which is a three point rise from the average white IQ of 103. The non-adopted white children had an average IQ of 115.

Putting it bluntly, you could raise these children in mansions surrounded by private tutors and it wouldn’t make a difference. To quote my favorite comedian Ron White, “You can’t fix stupid.”