Convincing colleagues about qualitative research: an analogy with crystal structures
I am confident most ChemEd people have struggled at some stage to talk with colleagues about qualitative research. Some of the more polite responses I’ve had when raising some of the qualitative literature about gendered experiences of degrees:
That’s just their opinion
The sample size is only ten
Our context is different
The discussion then becomes tricky very quickly! Context is often a fair point, speaking to the generalisability of qualitative research. “Opinion” often becomes quite a painful conversation if the word is actually being used to mean “political position” in ways which assume that the status quo is neutral. And sample size is often a tacit attempt to extend standards of rigour in the physical sciences into a different research field, specifically that there is one objective truth of the matter (rather than many subjective truths).
I say this not to criticise colleagues, but to try and point squarely at a problem we share. Finding better ways to improve Chemistry teaching through qualitative research results is something we haven’t figured out how to do yet. And it would be great - for all of us and for all our students - if we could.
This blog attempts to draw an analogy between how synthetic chemists interpret crystal structures and how chemistry educators might interpret qualitative research. My hope is to find a positive way for lab chemists to engage with the agenda of Qualitative Research while still respecting their deep expertise in Chemistry.
Crystal Structures
Single-crystal x-ray diffraction is an incredibly powerful tool for determining structure. The atomic coordinates of each atom can be measured and - through widely-available software - visualised, giving direct information about the geometries and bond lengths within the crystal. This information is valuable to synthetic chemists. A structure of your target proves that you made it, and lets you compare the structure with other systems, situating it firmly within a wider literature.
But there are limits to what a crystal structure can tell you about your work. Getting a single crystal does not establish that your whole reaction formed this product (it could be a tiny by-product, or just the one which crystallised). A solid state structure does not give direct information about behaviour in the gas or solution phases (though it might encourage ideas about what that behaviour might be).
Analogy with Qualitative Research
My central point is that chemists understand that there is great value to crystal structures, but also very particular limits to the claims you can evidence from crystal data. The nature of crystallographic data has useful resonances with the nature of qualitative work because in both cases the merit of the study is its narrow focus and the limits of the study are (typically) also the narrow focus.
So, if you have qualitative work describing how Black students experience Chemistry degrees. There is great value in understanding their experience, but there need not be any attempt to claim that the experience is shared by any other group. Chemists use this argument all the time when they report a crystallographic bond length - telling the community that a C-C bond is a certain length is not an attempt to say that every C-C bond (or every bond) is that length, but rather to explore this particular bond in this particular context.
And, yes, there are limits to how interpretable one qualitative study can be. It is true that a study done in the US might not generalise to the UK context, for example, as it is true that an Fe-O bond in an extended solid might not generalise to one in a molecule. And it is correct that a study in Biology might not map directly onto Chemistry, like the solid structure might not reflect solution phase behaviour.
But, look, Chemists have found ways of using that information anyway. It is interesting to see how your Fe-O bond compares to all sorts of other ones. It is worth seeing whether the MOs you calculate in the crystal’s geometry show you a HOMO which matches the molecule’s reactivity. And, yes, there are huge questions of judgement which professionals make when navigating these arguments. But we do this routinely in Chemistry.
Conclusion
I guess I keep coming back to two things in qualitative work.
First, it is so difficult to have these conversations with people who have learned to be critical within their disciplines and have not been given the chance to develop their thinking about what work can look like in other disciplines. And while I have often felt a very painful rebuffing when trying to contribute to conversations, the dismissiveness with which qualitative work is treated is often curiously free of malice. I am hopeful that people can be convinced, if they find a helpful way to think about things.
Second is the broader equity argument. I don’t think there is going to be one monolithic experience shared by all Black Chemistry students, but if you are depending on a quantitative paradigm to answer that question then it may be decades before we recruit enough Black students to satisfy a p value. I feel there is a danger that the philosophical positivism and the quantitative habits of our discipline are fencing off some of the urgent questions we are facing right now.