Three-pass approach
1. The first pass gives you a general idea about the paper.
2. The second pass lets you grasp the paper’s content, but not its details.
3. The third pass helps you understand the paper in depth.
The first pass
This pass should take about five to ten minutes and consists of the following steps:
-
Carefully read the title, abstract, and introduction
-
Read the section and sub-section headings, but ignore everything else
-
Read the conclusions
-
Glance over the references, mentally ticking off the ones you’ve already read
At the end of the first pass, you should be able to answer the five Cs:
-
Category: What type of paper is this? A measurement paper? An analysis of an existing system? A description of a research prototype?
-
Context: Which other papers is it related to? Which theoretical bases were used to analyze the problem?
-
Correctness: Do the assumptions appear to be valid?
-
Contributions: What are the paper’s main contributions?
-
Clarity: Is the paper well written?
The second pass
In the second pass, read the paper with greater care, but ignore details such as proofs.
-
Look carefully at the figures, diagrams and other illustrations in the paper. Pay special attention to graphs. Are the axes properly labeled? Are results shown with error bars, so that conclusions are statistically significant? Common mistakes like these will separate rushed, shoddy work from the truly excellent.
-
Remember to mark relevant unread references for further reading (this is a good way to learn more about the background of the paper).
The second pass should take up to an hour. After this pass, you should be able to grasp the content of the paper.
The third pass
The key to the third pass is to attempt to virtually re-implement the paper: that is, making the same assumptions as the authors, re-create the work. You should identify and challenge every assumption in every statement.
This pass can take about four or five hours for beginners, and about an hour for an experienced reader. At the end of this pass, you should be able to reconstruct the entire structure of the paper from memory, as well as be able to identify its strong and weak points.
Doing a literature survey
Here is how you can use the three-pass approach to help.
-
First, use an academic search engine such as Google Scholar or CiteSeer and some well-chosen keywords to find three to five recent papers in the area. Do one pass on each paper to get a sense of the work, then read their related work sections. If you can find such a survey, you are done.
-
find shared citations and repeated author names in the bibliography. Download the key papers and set them aside. Then go to the websites of the key researchers and see where they’ve published recently.
-
go to the website for these top conferences and look through their recent proceedings.
If you are reading a paper to do a review, you should also read Timothy Roscoe’s paper on “Writing reviews for systems conferences” [2]. If you’re planning to write a technical paper, you should refer both to Henning Schulzrinne’s comprehensive web site [3] and George Whitesides’s excellent overview of the process [4]. Finally, Simon Peyton Jones has a website that covers the entire spectrum of research skills [1].
[1] S. Keshav, “How to Read a Paper”, https://www.cs.jhu.edu/~jason/advice/how-to-read-a-paper.html
[2] J. Eisner, “How to Read a Technical Paper”, <https://web.stanford.edu/class/ee384m/Handouts/HowtoReadPaper.pdf
[3] S. Peyton Jones, “Research Skills,” http://research.microsoft.com/ simonpj/Papers/givinga-talk/giving-a-talk.html.
[4] T. Roscoe, “Writing Reviews for Systems Conferences,” http://people.inf.ethz.ch/troscoe/pubs/reviewwriting.pdf.
[5] H. Schulzrinne, “Writing Technical Articles,” http://www.cs.columbia.edu/hgs/etc/writingstyle.html.
[6] G.M. Whitesides, “Whitesides’ Group: Writing a Paper,” http://www.che.iitm.ac.in/misc/dd/writepaper.pdf.
