Problematising Problem Solving
I first heard of Problem Solving in summer 2016. There was a meeting where the inorganic staff sat down and looked at each others’ exam questions. This was the first time I had written a question, and I wasn’t quite sure what would happen in the meeting.
They looked at my Inorganic NMR question. I got some helpful, practical suggestions about the instruction words I’d used (‘discuss’ is too vague: could you use ‘explain why?’) and the structure of the subsections. Next up was my nanocrystals question. My colleagues were unanimously critical of this question because it didn’t have enough problem solving in it. Again, they suggested useful things: ways of adapting the material to embrace calculations or data interpretation.
While this experience was very collegial, I felt like I’d missed a day of school which they’d all had. Problem Solving? What’s that?
It’s a term which gets used a lot in chemistry education discussions, but I’ve looked and looked and never found a satisfactory definition. I have listened to how other people and tried to develop a view of what it means from context. I have read books and papers. I have reflected on my own intuitions. I have asked other academics.
But I don’t know what Problem Solving is. I am embarrassed about this. I’ve been thinking about it since summer 2016, and it has confused me more and more and more.
Framing the Problem: Accreditation
The RSC accreditation documentation emphasises Problem Solving. Accredited BSc degrees should produce graduates who have
the ability to apply standard methodology to the solution of problems in chemistry.
Master’s graduates should have
the ability to adapt and apply methodology to the solution of unfamiliar types of problems.
These outcomes are to be reached through an assessment diet in which
An appropriate proportion of marks linked to key concepts should be assigned on the basis of formal examination conducted under controlled conditions. Such examinations can be open or closed book.
This post is a reflection on what Problem Solving means in the context of RSC-accredited HE Chemistry exams.
What is Problem Solving?
I have encountered several models of Problem Solving.
Johnstone emphasised the arrangement of data (complete/incomplete), method (familiar/unfamiliar) and outcome (given/open), to give eight categories of PS.
Most chemistry exams reflect categories of problem solving in the 1-3 range. Most research grants reflect problem solving in the 5-8 range. This gap is an interesting thing to think about.
Bodner’s work developed an expert/novice model of problem solvers, which Overton broadened to embrace transitional problem solvers. The focus here is really interesting: it emphasises the solver more than the problem.
This tension between person and problem is also a theme in the conversations I’ve had with other teachers: the distinction between a problem and an exercise looms very large. Exercises are tasks which require the solver to do a thing they already understand; problems are tasks which the solver doesn’t yet know how to solve. 2+3 is an exercise to me today; it was a problem to me before I knew how to add.
This starts to approach the pop vox framing of problem solving as ‘what you do when you don’t know what to do’ and the common sense idea of ‘working something out’.
What Problem Solving isn’t
The most startling thing to note is most definitions of PS do not accommodate the scientific method. The deductive reasoning typified in chemistry exams is in sincere opposition to the inductive reasoning required to make fundamental conceptual breakthroughs.
Similarly, problem solving is not the same thing as critical thinking; a (slightly) uncharitable reading of the RSC guidance is that degrees don’t need students to think until fourth year. To advance a coherent argument; construct a robust narrative; to question whether the evidence provided is relevant; to explore whether the approach used is appropriate - all of these ideas might be more heavily emphasised in a Critical Thinking conception of a chemistry degree.
Problem solving is not recall. To my mind, this is its main strength: at its best, it requires the flexible application of knowledge to situations. It remains extremely easy to ‘dress up’ recall as problem solving when writing an exam paper, though.
Problem solving isn’t a way to train generalist problem solvers. The narrow skill of (say) solving problems about VSEPR structures doesn’t train someone to reverse climate change or negotiate a contract. It might be that advanced problem solving does equip people to transfer skills more readily, but a typical chemistry exam seems like a poor assessment to foster meaningful transferable problem solving strategies.
Finally, assessed problem solving isn’t the same thing as understanding. It is possible to solve problems algorithmically without knowing what’s going on; I did this myself for magnetism in finals.
Criticism of Problem Solving as a Programme Outcome
I have been thinking about problem solving for about five years, and I still don’t feel confident that I know what it is. Perhaps I am over-complicating things, but I wonder if clearer guidance about what ‘problem solving’ means could be useful.
I worry that problem solving is an inappropriate goal as a programme outcome in the specific context of an assessment diet dominated by exams.
If you accept that exercises are not problems, then the scope to assess problem solving relates intimately to teaching. I can teach a student to use Wade’s Rules to predict the structures of main group clusters, but doing so turns cluster prediction problems into exercises for that student. If teaching and assessment are aligned, our assessments must necessarily emphasise exercises over problems.
Perhaps, though, this is taking too narrow a view; it is possible to imagine that completing exercises evidences that the student must have learned to solve such problems along the way. Such a concession would still be a useful thing to state.
My broader concern is that such an assessment regime builds the degree around what is assessable. Students are anxious about how to integrate wavefunctions rather than about addressing global warming; students reps ask for model answers rather than a degree which explores how the discipline might improve humanity’s lot; all-nighters in the library are more about submitting answers that will get ticks and crosses than about reading last week’s JACS publications.
This is not to say that such teaching is absent, but rather to acknowledge that it is not the norm.
Suggestions
The dominance of ‘problem solving’ exam assessment is more cultural than strategic. My impression is that the RSC and QAA regimens would readily incorporate broader conceptions of problem solving if academics chose to shift their focus. I feel that assessments centred on critical thinking would also ruffle few feathers.
I would welcome this, but I also see good reasons to resist such changes. The most persuasive of these seems to be assessment literacy. Unseen exams are a well-understood format in the context of chemistry assessment - essays are not. Students know what sorts of things to do in an exam, but might fail an essay task for reasons unrelated to their grasp of chemistry.
I feel that the core weakness in existing assessment strategies is that the type of problem solving we want our graduates to demonstrate has not been properly articulated. Without some clearer expression of what we intend to achieve, it is difficult to suggest ways to change what exists at the moment.
Broadly, I feel that the core contention is likely to be the trade between coverage and complexity of problem solving. If you want students to write an essay deliberating what carbon capture technologies show most promise, what do you cut? Pericyclics? Soft condensed matter? Bioinorganic chemistry? We feel a really deep emotional attachment to a corpus of knowledge which is too broad for even the existing degree.
Perhaps the cute point to close on is that the over-stuffing of the degree curriculum is a problem I don’t know how to solve; perhaps if we train our graduates differently they’ll be able to fix it.