A nice essay in the New York Times is slugged (online) as “How You Should Read Coronavirus Studies, or Any Science Paper.” It details a bit of the history of scientific prose and notes the familiar standard background/methods/results/discussion structure of such papers. But it doesn’t actually tell you how to read such a paper. In fact, about the most specific advice given is to find authorities on social media and have them inform you (ick). Which might be useful for really hot button stuff , but will leave you at sea in geology.
So, putting aside the lingo specific to that specialty, the ineptitude of authors’ prose, the gutting of the paper by dismissal of key parts to supplemental materials…just how do you approach a scientific paper? Well, no worries, here is a helpful guide. And a hint: don’t just read these papers front-to-back…
- It is worth your time? Oddly, the abstract got no attention in the NY Times essay, but if you are good on lingo, it is the place to start, for here should be the paper boiled down to brass tacks. Just what was the problem to be solved? What were they looking at? And did the work make progress? If the problem is uninteresting, the approach irrelevant, or the progress seemingly minuscule, move on.
- Worth your time, part 2? OK, maybe the abstract showed some life, or maybe it was too dense, or maybe the journal in question doesn’t use them because their papers are so short. Next up, check out the section labeled "background" or something similar. This should tell you why this work was needed: what is the problem that needs addressing? Now you might have to wade through excess gunk here; much as George Will throws half a dictionary into each of his essays, some scientists will want to prove their erudition through a review of materials tangential to the problem at hand. Don’t be distracted: what is the problem? The background material is often the most accessible part of the paper; if, by the end of the background, you have no idea what this study is after, then there is a decent chance that the authors weren’t sure themselves.
- Is there progress? So the authors are after a problem you care about. Great. Did they get anywhere? Hop straight to the discussion/conclusions. Yes, this may feel like cheating when reading Agatha Christie, but your time is valuable. What progress do the authors claim to have made? Maybe it reinforces existing theory, maybe it shows a hole in existing theory, maybe it raises questions rather than answering them. Occasionally authors will have stumbled onto something important and not recognize it, but if this isn’t your field, odds are you won’t recognize it either.
- Do you care about that progress? If you shrug and say, so what, then note what you've seen and move on. Or maybe this is a data point you mark down in some notes, but for now you don’t need anything more. But if this is really something exciting, then it is time to start peeling the paper apart. The very first thing you can do is look at the figures. Typically there is a figure or two that is a money figure, that shows the data in some light that makes clear that the exciting result is real. Find that figure and understand it. In some cases you might immediately be suspicious: what the authors call a strong correlation might look like a shotgun blast. Or there might be datapoints excluded from analysis for reasons that seem flimsy. You might decide that this result isn’t one to rely on. Or that it is looking pretty solid…
- I need to be sure. If this isn’t your field, well, good luck from this point further. Be prepared to work harder. For now you will want to look at all those guts between the background and the interpretation (including those evil supplements). Are the methods appropriate? The samples clear and sufficient for these purposes? Are analytical reductions novel or standard? If novel, are they clear and well supported? Are there signs of p-hacking, like revising the dataset after analysis was complete, or picking one minor aspect of a much larger dataset? Don’t assume a poor presentation that is impenetrable means that the authors are exceptionally able and writing far over your head: it could be more of a smokescreen than unusually turgid science prose.
- This is the Holy Grail I’ve been seeking. You found the literature that is asking the questions you need answered in a productive way, and you want to get as much context around this as you can. As a rule, papers will cite other papers on the same topic. You will want to find any other researchers addressing this and see what they say. Going backward usually isn’t too hard: those references in the background section point to papers and, just as importantly, names of researchers. You can go to the websites of those researchers not affiliated with this paper and see if they keep their list of publications up to date (most don't). If so, you might find some other views that are relevant. As detailed elsewhere, Science Citation Index is a very useful tool. Google Scholar has been improving with time and might serve your needs; look to see what other papers cite the one you found so important. If you can find a couple other papers directly related to this topic of interest, odds are that you will see a fair bit of the spectrum of thought in the scientific literature on your topic.
For some very high profile literature, there might be a “News and Views” type piece in Science or Nature about the work that might provide some background from other authorities (not very likely for our topics...but maybe). That can be helpful (if the paper in question is in Science or Nature, the broader view piece might well be linked directly from the article webpage).
Will this get you to scientific nirvana? Well, maybe part way. All too often the problem with a scientific paper is what isn’t said more than what is. Is there an implicit assumption that should be reexamined? Is there an analysis that should have been done or documented? Was some other data collected that didn’t fit for some reason? Often the bigger advances are in challenging what has been implicitly assumed; being able to recognize such things is very hard even for experienced scientists (after all, there is a lot that is implicit in a sense because science is building on older science).
But for most purposes, following that script will more or less get you where you want to get to with a minimum inconvenience.
Return to GEOL 4717/5717 Field Seminar Homepage
Please send
mail to cjones@colorado.edu if you encounter any problems or have
suggestions.
C.
H. Jones | CIRES | Dept.
of Geological Sciences | Univ.
of Colorado at Boulder
Last modified at
January 18, 2025 2:26 PM