In my latest Globe and Mail piece, I summarized a study by Sa Bui, Steven G. Craig, and Scott Imberman on the effectiveness of gifted education. The authors look at students in a large urban American school district who were evaluated for gifted programming in grade five. They ask: Who does better on the grade 6 and 7 standardized tests, the students who just made it into the gifted program, or the ones who fell just below the gifted threshold?
The authors have an impressive amount of data: standardized grade 5 test scores for 5,500 students either side of the gifted cut-off point before the gifted programming begins, and the same students’ grade 6 test scores, one year later. They have similar information for 2,600 grade 7 students.
To test the effectiveness of gifted education, they measure how far each student was away from the gifted cut-off. The authors then estimate grade 6 and grade 7 standardized test scores as a function of distance from the gifted-eligibility threshold and some other controls. A "regression discontinuity" analysis is used to figure out if those who make it into the gifted program experience a jump in educational outcomes.
It's easier to explain with a picture than with words:
For reading and language – the green line and the red line – there is no jump in the test results at the gifted threshold. There's a kink, but not a shift. From this, the authors conclude that “students exposed to gifted-talent curriculum for the entirety of 6th grade plus half of 7th grade exhibit no significant improvement in achievement.” This is despite the fact that the students in the gifted-talented program have more educational resources coming their way – they are in classes with higher performing peers, are more likely to be placed in advanced classes, and more likely to attend a gifted-talented magnet school.
The lack of improvement in reading and language can be explained in a number of ways. The less able "gifted" students might feel discouraged by being in the bottom of the class and thus put less effort into school. The standardized test scores shown in the figure above might be measuring innate ability rather than what is taught in school. Reading and language scores may be more influenced by home environment than what is taught in the classroom.
But what is really striking is the suggestion that math results actually *fall* for those identified as gifted.
There are three possible explanations for the decline in math results. The first is explained here: about 5% of the time, random variation will generate a result that is statistically significant at the 95% confidence level.
The second possible explanation is that gifted programs teach math badly. This is not implausible.
Jump Math is a new approach to math education developed by John Mighton. Mounting evidence finds that using the Jump Math approach leads to large improvements in students' achievement levels and mathematics capabilities. The key to the Jump Math approach is starting with simple problems and building intuition and confidence through repetition; breaking everything up into simple micro-steps. To the extent that gifted math education is even further from the Jump Math approach than regular math education – less repetition, fewer simple, intuition- and confidence-building problems – it may be leading to even worse outcomes.
A third explanation of that downward jump in the blue line – the fall in math results – is that the students at the bottom of the gifted group had lower levels of innate mathematical ability than students at the top of the non-gifted group. Bui, Craig and Imberman find that, once they add in controls for lagged achievement levels, the fall in mathematics achievement levels is no longer statistically significant, which is compatible with the differential-ability story.
It's easy to think of reasons why students who are talented at math might be less likely to be identified as gifted than students who are talented in reading or language. Math scores may receive less weight than reading/language scores when students are identified as gifted. There may be students – and this is a huge issue in Canadian school districts with large immigrant populations – who are gifted in math, but speak English as a second language, so cannot be identified as gifted.
(As I understand it, in my school district there is actually a rule that prevents students from being evaluated for gifted status until they have been in English-language education for three years. This means, for example, that the best math student in the local high school might not be able to get a space in the gifted class. Unfortunately I cannot find this written down anywhere, so I may be mistaken.)
I could go on, here is my take-away: There are great wads of resources thrown at gifted education, and little evidence of positive results for border-line gifted students. [Update: "great wads" might be an exaggeration.] My own interpretation: gifted programs aren't producing results for two reasons. The first is that border-line gifted students get discouraged. The second is that students who are talented in specific subject areas, e.g. math, are not being identified as gifted.
There's a simple solution – more flexible gifted programming. Allow students to be identified as math-gifted or language-gifted or arts-gifted or music-gifted, and develop specific programming for those needs. The best mathematician in the school might be lousy at languages, and vice versa. More flexible gifted programming would allow, for example, new Canadians to access gifted math programs while, at the same time, receiving remedial English instruction.
And forget about classes like gifted physics 11 and gifted calculus 12. Few students take these courses because it's generally harder to get high grades in a gifted class, and good grade 12 grades pay off in terms of university admission and scholarship offers. Plus there is so little difference between the ability distribution of students in Gifted Physics 12 and regular Physics 12 that it is hard to imagine that there are significant advantages to differentiating between the two groups. It's a waste of resources.
But change will not be easy. As my mentor Julian Le Grand used to say: Hell hath no fury like the middle class in defence of its privileges.
Worthwhile Canadian Initiative 
Alice – this is why the method of screening is so important. The ever authoritative NY Times reports
http://topics.nytimes.com/top/reference/timestopics/subjects/g/gifted_students/index.html?scp=1&sq=gifted%20girls&st=cse
that in NY, girls are more likely to be identified as gifted than boys. As you say, that is less likely to happen with admission based on IQ testing, but then there’s all of these other issues with IQ tests, e.g. issue of racial bias etc. There’s a strange paradox here – two comments up Mary is making sarcastic comments about the usefulness of test scores, while at the same time the whole premise of gifted programs is that ability can be identified using test scores!
Another question is whether or not the test adjusts for maturity – i.e. is the gifted cut-off for a Jan child the same as the gifted cut-off for a Dec child? Are parents allowed to submit results of private tests, i.e. test the child until the right answer is obtained? (sort of like law school admissions 😉 )
Mary – this whole debate is becoming very reminiscent of debates about health care. Annual mammograms starting at age 40? It has to be a good idea, because it will save lives, won’t it? Well, not necessarily. The cost per life saved is high, especially once one factors in the health risk of exposure to radiation for women in their 40s. There might be other interventions, e.g. intensive breast-feeding support programs, that would do more to reduce breast cancer risks.
Asking “does the program potentially have benefits for some people?” gets us no where – the program would have to be pretty useless not to have potential benefits for some people.
The much more difficult, but much more important question is: “does this program have greater benefits than alternative interventions?”
Honestly, now that this post has been picked up by Arnold Kling and the Daily Beast, I’m wondering if – actually I’m not wondering, I know that – I’m just incredibly politically naive, and the real debate is about providing good quality universal public education, and some people are using this debate as an excuse not to do that.
I went through the gifted program, and while I absolutely loved it (in regular classes I was ostracized and isolated – Gifted allowed me to develop socially), I think I can see where results like this come from.
My instinct would be to blame the teachers, not the students. The majority of primary and secondary teachers have degrees in softer fields like English and History (particularly among the teachers with MA’s, who are more likely to get assigned to teach the gifted kids). In contrast, because degrees in the sciences and math are in greater demand within the private sector, relatively few teachers have such backgrounds (there may be a gender story there as well, men are more likely to hold the latter type of degree, and rather unlikely to become teachers).
What I am suggesting is that the ability of the education system to nurture above average language skills is greater than its ability to nurture above-average math skills. As far as I know, the homeroom model prevails prior to high school. So gifted kids tend to be taught by English majors struggling to administer the gifted math curriculum. The general curriculum is simpler, but I’ll wager that it is better taught because it is closer to the core competences of those that teach it.
My own experiences with gifted are a natural experiment that inform this argument somewhat. I started gifted in grade 5, and was taught by a hippie art teacher who essentially avoided teaching math. While future teachers were somewhat better, we were probably behind the general cohort, due to having done very little math in grades 5 and 6. When I went on to high school that gap remained – people that joined the Gifted program in high school were generally better at math than those who had been in the Gifted program (and I don’t think this had much to do with innate abilities – we actually did better in tests like the Pascal or the Cayley). Moreover, people like to be special. In Gifted I think there was a tendency for people to develop the skills they were already relatively better at (perhaps experiencing diminishing returns), in order to stand out. Being at the bottom of the class in Gifted math may have pushed some students (myself included) to focus more attention to subjects like English and history.
Secondly, and this should be a rather obvious point, but we pick up language skills from our surroundings much more than we do math skills. People do not usually discuss math, but rather learn it in private. In that sense, language skills may be a positive externality of Gifted classes.
Although I was the one contributing to public good production, rather than one of the people benefiting from it, I wouldn’t give up my Gifted experiences for the world. Gifted was a nerd paradise, that enabled me to socialize in ways that I couldn’t otherwise (dungeons and dragons really was the first place in which I was able to come out of my shell), and to take on leadership roles that would otherwise have been barred to me (because we had very high internal social cohesion, we played a leading role in organizing events and forming clubs within the school).
“that in NY, girls are more likely to be identified as gifted than boys. As you say, that is less likely to happen with admission based on IQ testing, but then there’s all of these other issues with IQ tests, e.g. issue of racial bias etc. There’s a strange paradox here – two comments up Mary is making sarcastic comments about the usefulness of test scores, while at the same time the whole premise of gifted programs is that ability can be identified using test scores!”
Test scores are not the only way Gifted students are identified. In the TDSB (at least in the early 90s) teacher recommendation was more common than test scores. Although I was admitted on the basis of a standardized test (my pre-Gifted homeroom teacher had me sit at the “stupid” table*, and recommended against my admission), a number of my peers were not. This is (as most of my earlier comment was) entirely anecdotal, but I think the transition to standardized tests did have an impact on the makeup of the Gifted program (namely with more Asians and boys, fewer white students and girls).
*Did other people have this experience – of primary students sitting at tables that roughly corresponded to their perceived academic ability?
hosertohoosier: *Did other people have this experience – of primary students sitting at tables that roughly corresponded to their perceived academic ability?
One of my elementary school teachers even gave the groups animal names – I think the slowest table was called the donkeys. It might have been snails. But there was no doubt who was who!
“In the TDSB (at least in the early 90s) teacher recommendation was more common than test scores.” though this contradicts Tom West “The most common practice in Ontario school boards is to identify a student as gifted if they are assessed at above the 98th percentile for FSIQ (some use GAI) on the Wechsler Intelligence Scale for Children IV (WISC-IV) – testing administered by a psychologist.”
Frances, I have noticed your views on education are dramatically shaped by your situation in Ottawa. Ottawa isn’t typical of Ontario.
I also think your views on educational competition are looking for a result where there isn’t one. In my town we had five public high schools, two Catholic ones and zero appreciable difference between them other than the fact that the public one was closer for most students.
My school experience in a small urban centre in Central Ontario was quite different. We had no separate “gifted” classes. I had a “gifted” identification elementary school, it got me access to a few special activities in lieu of general ones, mostly so I wouldn’t be so much of a pain in regular classes. Really, it didn’t amount to much at all and didn’t do me any good.
Prior to funding reform Toronto and Ottawa were the only board who weren’t dependent on provincial funding allowances, local property taxes were enough and they could do what they wanted. Other boards were more restricted in what they could offer.
My son tested gifted in San Francisco and it was important in getting him out of rote education and into challenging education. That plus honors plus advanced placement got him ready for Carnegie Mellon. I really dislike this trashing of ‘middle class parents’. Here in the States it can be HARD to get a good education in the public schools. Parents fighting the school system to get their kids a good education is a necessary and useful part of the provision of quality public education. The alternative is private for more than 20K / year. My son raised the bar in every class he was in – he contributed to the public school environment – and if he had been stuck in less academic – less gifted – classes he would not have been able to do that.
I had to go battle the school system to get him into appropriate schools and classes at least four times. I used his qualification as gifted every time.
I have to concur with hosertohoosier on a lot of his/her points.
One further thing – I am wary of adults identifying kids who are gifted as also socially adapted. This is from an adults perspective, where gifted students probably relate better. I know that in a regular public school stream, my life was hell. It was much better in a full time gifted program, where I was with other socially awkward nerds. But, we probably seemed pretty ok to adults because we were articulate and confident when given the opportunity to stretch ourselves – which is not something that helps in the schoolyard.
One comment I would make is that, to the extent gifted programs are helpful to achievement, could one not make an argument that there are returns to society, that benefit everyone in the end? I would make that argument about all sorts of social/academic programs that don’t directly benefit me, but that I don’t have any problem with supporting.
Finally, to Frances, I don’t understand why attacking the benefits accorded those in the gifted programs helps solve the (real) problems you identify with a school system that has bad teachers and inadequate educational programs. The solution isn’t to punish the groups who can escape that, but to deal with that problem.
Whitfit – yes, well, given the enthusiasm of the National Review and similar for this post I do wonder if perhaps I put the case a little too strongly.
But:
Read lark’s post above. Some parents are willing to fight for an excellent education for their children. If you take those parents/their children out of the mainstream classroom, then there is less pressure on schools to fix inadequate programs.
Not every gifted program is a safe sanctuary for nerdy kids – division 81 at _______ Secondary wasn’t.
Gifted programs cannot be justified by saying “these programs have benefits.” To justify these programs, it is necessary to show that they provide greater benefits than other uses of funds.
Actually, the most damning critique of my argument so far has been right up the beginning – something along the lines of “if schools were delivering programs optimally, wouldn’t you expect to see a continuous relationship between ability and achievement?”
lark: I’m happy for you that your son is successful. I hope he is happy.
I think you might be confusing cause and effect. Was your son successful because of the gifted programs, or was it his innate ability and your dedication as a parent? What if you were poorer and lived in an area where the the school offered no gifted programs? I suspect you and he would have found a way regardless.
France-
You’re right that the data seem to indicate that funds are not allocated optimally, but that doesn’t provide support for any particular reform. Or am I wrong?
Frances:
It is true that there is a discontinuity, but I think an earlier poster had it right when mentioning that the appropriate comparison is a control group, and it would be interesting to see what other kinks develop in different methods of dividing and teaching students.
And I agree with you that there needs to be more evidence based approaches to allocating resources and designing education programs.
Your other point, about separating the children of the most engaged parents is an interesting one. However, this cuts both ways – might some of those parents not take their kids right out of the public system, making it even easier to underfund and kick around? Look at NYC public schools. Maybe that argument requires also that there is no parallel private system, like health care is done in Canada. Without outlawing private schools, and eliminating the programs that relatively more privileged kids take advantage of (though I would also note that from my experience in these programs, many of the kids that I was in school with were from working class immigrant families) what you might end up with is privileged kids going to an increasing number of private schools, and less privileged kids being stuck in a public system that is dysfunctional, like in NYC or other US cities where they have a hard time keeping public schools appropriately funded. I would also point out that in many of these programs, it is not a different set of resources – it is the same science teacher, math teacher etc… teaching the gifted class, just with a segregated class. Maybe there are some teachers who thrive on that variety? Maybe they do a better job overall because they have an opportunity to experience a variety of groups? It seems to me that the special programs are an easy target in a complex school system.
Maybe there is no evidence for gifted or enrichment programs, but I also think that evidence based policy (of the kind that is scientific and self critical and is not just about justifying preconceived notions) is so far from the minds of politicians, the public, and school administrators that, although I would welcome it, I would caution against using one piece of evidence – the discontinuity in achievement in one specific instance – to justify a reform that it is unclear is needed or an improvement.
I wonder why we never see articles asking if varsity sports programs are a waste of money? After all, they are just as elitist and exclusionary since they only cater to the elite athletes and leave everyone else in the dust.
For those of you who think that it does no harm to advanced/precocious/gifted/whatever-you-call-them-so-as-not-to-hurt-your-own-feelings children to place them in a regular class, I can tell that you’ve never met a truly gifted child. You might know some bright children and you might even know some children who are a little advanced for their grade, but you don’t know a child who is completing work 3-6 grade levels above their classmates. For a child at 6 who can perform computations with whole numbers, with negative numbers (and for that matter even understand the concept of negative numbers), with fractions and with decimals- do you honestly think it’s ok for that child to be subjected to nothing more than 2+4=6 for days on end? For that same child to be subjected to Dick and Jane type readers when they’ve already read Harry Potter level books? When they’ve already questioned the existence of a god (at age 5) or Santa Claus (at age 4)? Mindnumbing boredom, so much so that the child at 5 hated school with a passion and felt lonely and said repeatedly that he didn’t fit in? Really? In most other circumstances such treatment would be considered abusive, but in the school system it’s the norm.
Why do we ask our academically gifted children to sit and wait (and even tutor their peers) while we provide every opportunity for our gifted athletes to advance and learn and train more? Are we humans really that threatened by the smart people among us that we would rather them have miserable childhoods and learn to hate school in order to keep ourselves from feeling inadequate? Do we really hate intellectuals so much? From too many of the posts on this site, it certainly sounds so. Disappointing- I really hoped the rest of the world was different in this regard, and I am slowly learning that y’all are not.
Alice wrote:
In Calgary the GATE (gifted and talented program) is overwhelmingly dominated by boys. In my daughter’s class of 18 there are 3 girls. That suggests to me that this isn’t just about a middle class privilege protection, at least here.
That’s strange — I went through the GATE program a decade ago and I recall a pretty decent mix of boys and girls. That being said, most of my peers were from upper-middle class or professional families, and I wasn’t, so I sure got some schooling about class values while I was there.
People who label gifted individuals as nerds are naive.
Giftedness in its true sense should be regarded as a disability or an individual with special needs. They may be advanced intellectually, but in other aspects such as social skills, or even in the psychological aspect – they may find themselves behind in their development.
Starting the gifted program at grade 4 in the public school system is often too late. The child has already lost interest in the school system. In an accepting environment, with fellow gifted childen – where a gifted child’s talents are identified early on and further developed – these individuals are true performers – in math, literature, the arts, in sports, in science etc.
An interesting study would be to see if the gap does exist between gifted kids who have been identified in earlier years (4-8 yrs old) and then addressed, vs gifted children who waited until 4th grade (9 yrs old) to be assessed and catered to.
It is not always “a gift” to be “gifted.