Taking the PISA
I know that the car boot death mayhem on Eastenders and the laughably blood-rich
Hollyoaks is the reality of life in London and Chester. MPs know that Benefits
Street is the reality of poor people's lives because they assume everyone's a
thief. They base this insight on themselves having a higher rate of convicted
criminals than the country as a whole and an impressively inclusive honour role
for those caught innocently, they say, fiddling their expenses. So it’s
empirical, scientific, intellectual and justified nonsense, based on soap
operas.
The insulting Michael Gove believes that schools don't teach the poetry of Sassoon, Brooke and Owen, nor do we read the diaries of the soldiers, nor did I ever teach “All Quiet on the Western Front” for a German point of view. Trips to The Battlefields? Look at the First World War as a “world war” including the world-changing three years of Russian anti Austro-Hungarian imperialism? Nah, I just shoved on the telly and said that Blackadder is the absolute truth.
Gove admires the leadership of, “great patriotic heroes,” generals
struggling in an industrial war. The insulting Michael Gove believes that schools don't teach the poetry of Sassoon, Brooke and Owen, nor do we read the diaries of the soldiers, nor did I ever teach “All Quiet on the Western Front” for a German point of view. Trips to The Battlefields? Look at the First World War as a “world war” including the world-changing three years of Russian anti Austro-Hungarian imperialism? Nah, I just shoved on the telly and said that Blackadder is the absolute truth.
Ignoring the millions of dead already sacrificed, he doesn’t understand satirical TV either. Looking at WW1 and the pity of war makes us, “left wing undergraduate academic cynic(s)” and in his education fiefdom Gove will determine how his history is taught.
Gove has now claimed hero status. Faced with leftie teachers, he felt like “Horatio at the Bridge,” a reference to the pro empire Macauley’s poem where Horatio, holds off 3,000 enemies, and is remembered as the lone warrior against the hordes.( Radio 4 January 2014) Will a statue do, Michael?
Moving on:
Imagine two goalposts so far apart that you have to build a wider stadium to contain them. You have the ball at your feet such a tiny distance from this gaping gap that it is quite impossible not to score . Step forward the Labour spokesman on education, the public schoolboy from the telly,Tristram Hunt. A mere tap and he would score vital election points. With his unnecessary, bureaucratic anti Teacher Licensing Scheme, in one clumsy move, Hunt missed the goal, fouled the spectators and sent the ball into his own goal.
Labour have got nothing to say on education: no leadership, no policy, no charisma and no ears. Because if you can't score against Gove with virtually the entire education crowd roaring you on you deserve historical anonymity. Mind you, Labour seems to be doing better at the “Mine is Bigger than Yours” game on immigration, benefits and public spending cuts.
Last month, the results of the OEDC’s Pisa tests were announced. Labour and Tory politicians played the blame game, The Daily Mail howled whilst teachers worried and wondered. The BBC’s usually thoughtful Reeta Chakrabarti presented her analysis which pretty much went: I talked to four Korean boys and they found the test easy and then I spoke with two English kids and they said it was hard. Therefore we are thick.
While all sorts of press people inaccurately recorded the UK
“plummeting” we stayed in the same mid 20s position. We can do better.
Headteachers did not necessarily take the tests seriously and
administered them grudgingly. I heard
from a head who gave no preparation time, did the tests on a Friday afternoon
with no preparation with a disgruntled student group. Germany, Switzerland and
Spain all reacted to previous poor league standing by prioritizing the content
and conduct of the tests - what Gove calls “Gaming” when he doesn’t like the
results. Let’s hope a Reading position of 19th satisfies the Germans.
We have no way of knowing which schools took the tests. About 4,000 15
year olds in each of the 65 reporting countries sat the tests. I know only of
two secondary modern schools who took part. Secondary moderns are for those
kids in places like Kent who have failed a selection test at age 11. Those who
pass go to grammar schools and are at the wealthier end of our society, “those
who are eligible for free school meals are less likely to secure a place in a
grammar school.” (The Sutton Trust 2014) Could it be that the selection of schools to
be tested is skewed academically according to desired outcomes? We know that public (private) schools take
bright kids on scholarships, to bolster their results as their paying intake
involves wealth not brains. These kids are barred from Pisa tests but in the
high tabled Finland, without private schools, testing their population might be
more representative of the country as a whole. Pity Kyrgyzstan who were bottom
in the 2011 tables, tested during a prolonged strike by their impoverished
teachers.
The tests themselves, the results and their interpretation are riddled with unacceptable weaknesses.
Each Pisa test is not taken by all the 15 year olds tested. By Pisa’s own admission, it would be impracticable to do the full 4 ½ hour test with all kids. Some countries have a Reading Result when no one took the reading test. The Rasch statisical model used extrapolates from the actual answers how each student would have done in questions they were not asked.
Not surprisingly this model is questioned - Professor Spiegelhalter of Cambridge University: expressed “considerable uncertainty” about the model. Svend Kreiner of Copenhagen University added. “I don’t think it’s reliable at all.”
Defending his work Pisa guru, Andreas Schlecher says, “The model is an approximation of reality.” “Large variations in single country ranking positions is likely,” and “people shouldn’t pay too much attention to the precise country rankings.”
There is much criticism of the model concurring that Pisa’s administrators work out what the scores would have been if all the students in all the countries answered the same questions.
If the questions asked in one particular country were switched with another,
league standings vary bizarrely, “with the UK finishing anywhere between 14th
and 30th, Japan between 8th and 40th and
Denmark between 5th and 37th .”(TES Dec 2013)
In the 2006 tests, half those tested were not asked any questions on reading and half were not tested on Maths, although full rankings were issued for both subjects. Kreiner says “this in itself is ridiculous.”
8 of the 28 reading questions were deleted in some countries because the OECD itself considered them ”dodgy,” but only in some countries. Little wonder that Dr Morrison of Belfast University deems the model, “valueless,” “utterly wrong,” and stated, “it will never work.”
As worrying is the attempt to ask the same questions in different
languages. An article on Cloning, “A Copying Machine for Human beings,” became incomprehensible translated word by
word into Norwegian.” I wonder how particular metaphors and other linguistic
nuances translate accurately in and out of the various languages of the Pisa test.
Kreiner, a student of Georg Rasch the creator of Pisa’s statistical
method condemned the tests in 2011, “I am not actually able to find two items
in Pisa’s test that function in exactly the same way in different
countries…Therefore you cannot use this model.”
Gove has called Pisa delivery man Schleicher, the most revolutionary
post war educationalist. What a laugh if all the world’s education politicians
get us to teach Pisa style, teach to the test until every country scores brilliantly,
whilst their kids can't shake hands or smile with feeling, appreciate art,
drama and music or kick a ball correctly.
You may recall the chilling image of the 5 year old Koreans trying to clamber
onto the bus taking them to their extra evening schools, which 80% of them do.
The very dangerous Goveite Schools Minister Elizabeth Truss, bound as Gove’successor,
as I desperately pun her name, has said that individual schools could take the
tests “to compare themselves against the world’s best education systems.” (Nov
2013)The government-friendly NFER is offering trials of this. Soon, we will all
be doing these tests as yet another flawed accountability measure, and yet
another bloody test.
With a nod of thanks to those in America making 1851 blog hits, I thought I’d mention USA performance – like ours it’s stagnated with position 24, 28 and 36 in Reading, Science and Maths respectively. Like everywhere, the middle class schools in wealthy areas did better than average and many of my UK doubts apply to the USA. However, despite such poor Pisa performance America has more millionaires and the highest GDP in the world, scores highly on creativity – the odd film, book and musical item, entrepreneurship and industrial patents granted and, of course, has more Miss Universe winners than any other country.
Back to the tests.
If you were born in 1983 you were the first year guinea pigs, for tests at the end of KS1 (aged 7), KS2 (11), KS3 (14) new GCSEs (at 16) and new A Levels at 18. The school was judged by your success in these tests so you were taught to these tests, practised for these tests and judged by them. And, before my 30 year old daughter revisits her, “I was so bored,” rants, the teachers were too. So if we are going to be judged by Pisa testing I will personally guarantee we move above other countries
We get the Department for Education to choose a fair
cohort of schools who take the tests seriously, prepare the kids for the sort
of approach required, set mocks, give each kid a banana and decent air
conditioning and tell them the tests are really important.
With scientific integrity I tested Diane Carey on
the easily available Sample Pisa Reading questions. At first she struggled,
then, when she saw model answers she soared to total success. And as a Maths
teacher she said that an hour’s preparation of her 15 year olds would lead to a
20% increase in scores. It’s like a cryptic crossword: easier once you know the
style.
There is nothing inherently wrong with tests: like an MOT car test they
can show you what needs improving. However, Goodhart’s Law is instructive:
“Once a measure becomes a target it ceases to be a good measure,” and the
danger is that Gove finds true love with Pisa: diagnosis, judgement and
imposition on flimsy evidence.
Politically, the tests are a ball to be kicked in the direction your team faces. We should kick our way, teach the curriculum that’s best for our students, whilst doing a little prep for the international tests by which they will be judged.
But if you really want, I can train sheep to answer your questions.
