Assessment – Vouching for Portfolios
This post is the result of a relatively extended twitter conversation between myself, @teachingofsci and @informed_edu. The initial impetus for the discussion was the announcement by the Education Secretary, Michael Gove, that schools should be allowed to push their brightest students to skip GCSEs entirely and move onto A-Levels at 14.
The consensus was, that rather than push students into harder sets of exams – it would be better for students to deepen and broaden their areas of strength. This could be done by giving students opportunities to engage with global and community issues (perhaps using sites like Interrobang or We Are What We Do) rather than simply push them to go down the same road faster – incidentally, surely encouraging such broader participation would dovetail nicely with the promotion of a ‘Big Society’?
The main road block to opportunities for such exploratory learning is the pervasive mindset that everything that is taught and learned must be assessed, and that the majority of that assessment (in order to be valid and unbiased) must take place through public exams. In addition, the current scheme of modular exams poses a particular road block, as rather than giving students opportunities to more deeply explore issues, the emphasis is on a cycle of learn, prep, and test.
My suggestion to help with this was to set up computer based testing that was taken when ready, rather than at a set time of a public examination. This suggestion would also deal with one of the often quoted issues with computerized testing – the need for schools to have hundreds of dedicated machines that are available for the examination period, but unused the rest of the time. On an examine when ready basis, a school could have one suite of examination computers, and students would book their testing time when ready to take the tests for a particular module of study. In some ways, it is surprising that the government has not considered this possibility for testing, as this scheme has been used for testing Literacy, Numeracy and ICT skills for those seeking Qualified Teacher Status in England and Wales.
@informed_edu‘s suggestion then was to use portfolios to complement examination testing in order to assess a broader range of skills and competencies than those that can be assessed in an exam. He expands on his ideas for the scheme in this blog post. To summarize briefly, not only do exams limit the range of competencies that can be assessed, the ultimate assessment of student skills comes down to the students’ ability to perform in exams, rather than the particular skills and competencies that are meant to be being assessed. In thinking about how this plays out in the real world, he considers the chain of trust and vouching that takes place as people progress in their careers. To your first employer, the grades you took for an examination may be of interest, and perhaps to a second employer – by the time you get to your third employer, however, the chances are that the vouchsafing of your competencies and skills by your previous employers will be of far more interest to your new employer than any examination results you accrued. For students then, instead of examining their abilities, teachers would vouch for their students skills and competencies, as demonstrated through a student’s portfolio of work. Teachers in turn can also be vouched for by colleagues, professional development institutions, mentors and the like.
@teachingofsci pointed out a potential difficulty, where some teachers would then be faced with in excess of one hundred GCSE student portfolios to assess (a portfolio being far more work to assess than the current coursework requirement of Science GCSEs). My response to this is that firstly, the portfolios should be published online, and secondly, the assessment of these portfolios should be crowd sourced. Good portfolios will be vouched for by many professional educators, in addition to the other students. The value of a ‘vouching’ can be determined through several means: firstly through the social metrics of the network – ‘vouching’ for people who have a very long social distance from you will carry far more weight than vouching for people that are close to you (this should prevent abuse by closed vouching loops); secondly, the quality of the individual’s work (as assessed by other people’s vouches) could increase the weight given to that person’s vouches.
The transparency and openness of the system is important, not just to ensure that everyone is able to contribute to the network, but also so that any abuses of the system readily become apparent and can be rectified in order to produce the ideal outcomes. In many ways, this system replicates many features of Cory Doctorow‘s Whuffie in Down and Out in the Magic Kingdom, as an attempt to give a metric to vouch for people that is non-exhaustible, while also building an online portfolio that will allow a student to be appraised based on their best work, not on what they were able to squeeze onto a page in some time between 45 minutes and 3 hours.
Finally, some concerns that could arise:
How could you prevent a student from being penalized just for being bad at social networking? I would say that this is where the professionalism of teachers would have to play a role – part of the system could ensure that portfolios that have generally been overlooked are flagged and promoted for review, in order to ensure that everyone has the opportunity to receive vouches that they deserve.
How is a teacher meant to keep track of all his/her students’ portfolios and offer formative feedback in order to help them improve? This could wind up being a massive burden – by requiring portfolios to be available online, the opportunities for formative feedback from a wide range of sources are much higher (for example, mentors from outside teaching – non-profit volunteers, business people, university faculty, could be engaged to act as mentors for portfolios – that’s the ‘Big Society’, right?) In addition, by shifting part of the burden of assessment away from exams more time would be freed up from revision classes, grading mock exams and the like to be available for ongoing formative assessment of student portfolios.
I think it is important to return to the point that started this whole discussion – the best way to challenge and stretch bright students is not to make them go the same route they would go anyway, only faster. While many bright students will enjoy the challenge of more complex material, we do them a great disservice if all that they ever learn is more content, and not a broader range of skills and competencies to help them to use that knowledge to great effect within society.
Many thanks to @teachingofsci and @informed_edu for the lively twitter discussion that prompted this blog post – I welcome comment and criticism below.
Consequentialism and Maximization
Consequentialism requires the maximization of some singular good – pleasure or preferences are frequently cited as being the singular good.
Ben Mackay asked me to consider two people A and B. A is very rich and gives, for example, 500,000 pounds to charity. B has an average income and gives 1000 pounds to charity. In addition, it is assumed that A is rich enough that the remaining disposable income he still has available after his generous donation is sufficient for him to have a millionaire’s lifestyle.
Ben argues that because the consequences of A’s actions will produce more happiness (or some other good), then A is a morally better person than B. (The underlying assumption here and in the foregoing is that giving more money will produce more good).
What we seek, however, is the maximization of the good in question – not simply the increase of it. When A donates his money, while he donates significantly more than B, the amount that he could have donated but did not, is far larger than the amount that B could have donated, but did not. As such, A fell far shorter of maximizing the good than B did.
Now, it may be argued that without the incentive to have his millionaire’s lifestyle, the rich man, A, will not continue to earn as much, and hence will produce no additional good over the average person, B, if he gives nearly all his earning. This may be the case, but there are counter-examples (some of which are detailed in Peter Singer’s ‘The Life You Can Save‘). Even if this is practically the case, for A to act in this way is to act in a way that explicitly fails to maximize the good, and hence his actions are even less moral than B’s.
There is a corrolary question of whether holding people to this high moral standard publicly will actually produce the results that we want – and the answer is probably not – but that does not mean that the actual moral value of their actions is any different, rather that in the face of practical concerns we should moderate the actual blame and praise we apportion in order to maximize the good.
Twitter Haikus until January 2011
Haikus I have inflicted on Twitter over the last 5 months – with concomitant hashtags.
Beauty beheld, what
Say you? Beautiful by your
Response, in your aye.
#aesthetics #subjectivism #relativity
Constant relative
Velocity, light on train
Snow days are slow days
Rain like acid on frosty
Crispness; enter sludge
Fiery orb a-
Glow, cloud striped, bridge sillhouette
Sunrise over hills
Anti-hydrogen
Gripped in magnets, super cool
Future fuel today!
Buffeted trees shake
Dappled shit brown concrete hut
Rolling through drizzled
Trainyards, Newark night, lights fail
Gentle calm darkness
Reddish brown leaf stream
Whizzes by speeding train, cone
Of autumn rustles.
A Not So Intelligent Design
Intelligent Design has failed to get much press lately as far as I have seen – but the pressure to have it edged into school curricula (particularly in Texas, where it can begin a domino effect due to the size of that state’s textbook market) has not ceased.
As I see it, Intelligent Design can be appraised in one of two ways: it is either an earnest attempt to present an alternative scientific theory to evolution that happens to involve something akin to the Christian God, or it is a cynical attempt to keep teaching of the scientific theory of evolution out of the classroom by any means necessary, as a precursor to backsliding all the way to New Earth Creationism (in order to maintain a literal reading of the bible).
If it is earnest – and I find it hard to have so little faith in humanity as to suspect every proponent of Intelligent Design of being a cynical, two-faced liar – I believe there are some significant problems with the basis of the theory as both a scientific position and as a credible theological position. The specific biological points have been rendered time and again, and far better than I would do justice to them (the NCSE have a quick primer, and there are many others available) – however, there are matters of physical science that impact as well.
Firstly, Intelligent Design proponents are carefully agnostic in their proclamations about the exact nature of the intelligent designer who is responsible for some of the irreducible complexity of life. This either means that there is some other intelligent life form who acts as the designer, or that it is some God-like entity. In the case of the intelligent life-form, it is conceivable that they might physically be able to travel to our planet and carry out their design work – but what we then find lacking is an explanation of how those life forms came to exist (as presumably they would also possess some level of irreducible complexity also). What is an additional problem for these life forms, however, is the difficulties of interplanetary or interstellar travel (as one assumes they are either no longer here, or came here from somewhere else).
In the case of a God-like entity, the physics becomes more puzzling. How exactly do we explain the intercession of a being who is not part of the universe in terms of physics? I am sure someone wishing to claim for the intercession of God would claim that it was simply a miracle, and hence not explicable by physical laws – so, in essence, what the Intelligent Design proponent is asking for is that not only should the teaching of Biology as it is understood and agreed upon by working scientists be altered, but also the teaching of Physics. Physical laws are no longer immutable and universally applicable. They are instead changeable on the whim of a creator, regardless of evidential basis to the contrary. To undermine our most successful scientific theories for the sake of finding a mechanism to support a theory in Biology seems somewhat absurd. If an evolutionary biologist demanded that the laws of physics be suspended in order to account for their theory, the scientific community would rightly condemn their work.
For the earnest proponent, however, I feel that the theological point is most telling. Assuming that we are talking of a designer who is in fact God, we are asked to imagine an all powerful, all knowing being who creates an immensely complex physical universe. Not only is this universe immensely complex, its complexity derives from what appears to be a set of physical laws, constants and fundamental particles (if Grand Unified Theories of Physics pan out, then this universe is ultimately the expression of an even more limited set of interacting fundamental entities). To think that the God who created this mind numbing complexity from a highly limited set of raw ingredients would need to come back 11 billion years later (as the first bacteria began to emerge on Earth) and tweak the system to add a flagellum for motion, is to limit the power and foresight of a glorious omnipotent God. It simply makes no sense to think that God could create the universe through simple interacting parts, and fail to set up the universe in such a way as to ensure the evolution of all organisms from the simple rules he set in place.
So, we either have God as the creator of the ultimate Rube Goldberg machine – with the universe unfolding from his initial conditions, including all of physics and the evolution of all living things – or he is an ineffective tinker, setting off the universe and having to continually return to it to set it right, and to fix his broken design.
Finally, if it is a cynical ploy to ultimately put New Earth Creationism into the science classroom, then take up the virtue of honesty, and say it straight out, rather than hiding behind a lie.
Inequality over Time
In this article Tim Harford puts forward an interesting reflection on the value of money over time. Roughly summarized, because of the much higher base level of goods and services (e.g. central heating, cell phones) available today, $7 today could to many people be worth more than $7 one hundred years ago, in spite of what inflation indices tell us.
This led me to think about the size of gaps between rich and poor. If it is preferable to be given $7 extra for goods and services now than 100 years ago, then it is more of a detriment to be $7 worse off than someone else now than 100 hundred years ago. This means that today the gap between rich and poor is actually far larger than it was 100 years ago. The main assumption for this of course is that you can use all that additional money to buy additional goods and services – I think this is a reasonable assumption, as there are goods and services enough to spend almost any amount of money on.
There are two ways to analyze these scenarios. Firstly, a utilitarian conception – in this case, as Tim Harford suggests, the greater purchasing power of $7 today due to the sheer availability of goods and services greatly increases the general happiness. Secondly, we can try to think about it in terms of a preferred distribution of wealth/opportunities – in this case, the situation preferred by a neutral observer (as per Rawls’ idea of a just society) would be one where it was more equally distributed, and due to the lesser disparity due to the lower purchasing power, the situation 100 years ago would be preferable. While we may be happier now, we may live in a less just world. It is perhaps our perceptions of injustice in this way that prevent us from using rose tinted spectacles to look at the present.
Alternate Present
My entry for the New Scientist Flash Fiction Competition 2010:
Ptolemy’s Eagle
“We do these things, not because they are easy, but because they are hard.” The words echoed in his head as Edwin sweated. The quintessence drill stuttered to a halt as defects in the drill bit propagated and it sheared. He wound down the power and tapped a code on the wire to the maintenance crew.
Holding his hand against the rail, he felt the increasing thrum as the elevator moved closer.
“H-i/stop/B-u-z-z/stop”, Neil tapped out on Edwin’s helmet. Edwin grabbed Neil’s speaking tube and plugged it into his. A controlled blast of air cleared the interlock, swirling a colorless spiral in the quintessential mist.
“We’re nearly through.”
“Seven years of drilling.” Neil’s voice muffled through the tube.
As they talked, the maintenance crew maneuvered the drill platform back. In its place they brought out a video camera tripod setting its lens on the tunnel face.
“What’s that for?”
“The final breakthrough; Mission Control needs this – taxpayer PR.”
Edwin looked down at Neil’s hand – a diamond tipped hand drill rested clumsily in the gauntlets of his quintessence suit.
“Better stand back, Buzz – you’ll block the shot.”
As Neil hustled towards the tunnel face, their speaking tubes snapped apart. Mike, commander of the drill rig team, grabbed the flailing tube and married it up to an extending tube attached to the camera. Edwin slowly moved back out of shot.
At a signal from Mike – camera rolling – Neil drilled at the tunnel face. The narrow hand drill bored into the face of the mechanically carved tunnel. As the drill bit gained purchase within the quintessence, Neil levered with the drill, impatient for breakthrough. A chip came loose, and a gentle mist of rarified quintessence came through from beyond the celestial sphere.
Edwin watched, unrealizing, as he became a footnote in history.
“It’s one small chip for man, one giant breakthrough for mankind.” Neil pronounced, the first man to break the quintessence barrier.
