The goal of a research paper is to extend the community's understanding about a topic by a tiny fraction. The papers themselves are written for experts in the paper's area. As a result, they are challenging for non-experts to read and deeply understand. This is unfortunate because the papers contain often information that is useful to a wide variety of students and practitioners.
The following sections provide templatized guidance to help experts and non-experts read research papers and critically analyze them. We start with how to read papers. We then provide guidance on how to synthesize your thoughts about papers to write summaries of them.
Reading research papers
To deeply understand research papers, you should read them in multiple passes, each time with increasing focus on specific elements. Think of reading papers like peeling onions. Since papers are written at the limits of our understanding, statements contained in them may be incorrect. Do not accept assertions in papers at face value, especially if they are not cited or backed up via evaluations.
First Pass: Obtain a 30,000ft-view understanding of what the paper is about.
Second Pass: Understand the problem the paper attempts to solve and the approach the paper takes to solve it.
Third Pass: Understand how the authors evaluate their approach.
Fourth pass: Understand how this paper improves on previous work in this area.
As you become more advanced in your reading of research papers, try to identify the research question that each paper attempts to answer. Once you've identified the research question, you can now decide whether the paper's approach and evaluation satisfactorily answers this question. You might find that it answers the research question for some setting and not others. Does your conclusion about whether the paper satisfactorily answers the research question match with what the authors claim as contributions?
In addition to the above guidance, you might find the links below to be helpful:
Writing your critical summaries
Once you have read a given paper, you are now ready to syntheize your thoughts and write a summary of it. I often find that writing a summary helps me more deeply understand a paper than reading alone.
The questions below will help you synthesize your thoughts. Please answer them explicitly in the summaries that you hand in. Your summaries should be about 1-page long.
Please remember that research is hard! It's easy to criticize research papers for what they don't do. Instead, when writing your summaries, focus on what is interesting and novel about the paper and what you learned from reading it. Try to think about what you might tell the authors to help them improve the paper even more.
- What problem does the paper address?
- What is the the papers' approach and how does it ameliorate the problem?
- What did you learn from reading this paper?
- Brainstorm a few ways to improve the paper. For example, could the work be applied to a different domain? Is there an important experiment that the authors should run that isn't in the paper?
- Answer any "extra questions" that appear in the “Notes” section for the corresponding lecture in Schedule page.
We are looking for thorough high-level summaries showing that you read more than the abstract. We should be able to read your summary and then give a reasonable approximation of the paper, even if we did not read the paper itself. We do give points for clarify, so a quick proofread can go a long way.
Example Summaries
Here are two excellent summaries of the Pivot Tracing paper
Here are two excellent summaries from Spring'22: