---

LBSC 878 Paper Critiques

Spring 2005

Being able to critique a research paper is an important skill for a researcher. Often you will be asked to review papers for inclusion in conferences and journals where you will be expected to provide a review of the paper as it stands, and suggestions for improvement. You will also want to be able to look at your own papers with a similarly critical eye before submitting them. This is therefore an important skill that once learned can stand you in good stead for the remainder of your career.

Early in the semester, each student will be expected to lead the class in the critique of a research paper that we all have read. In this document we provide some guidelines for what we expect. These guidelines are meant only to help you, and are not supposed to represent the only way that things can be done -- if you come up with a novel way of tackling this problem then we would welcome that, although you might want to check with one of the instructors before you make changes that are two radical!

Format

Discussion

Group discussions are a great way to share ideas, identify and address problems and get opinions from those who are perhaps reluctant to speak otherwise. They create an informal environment in which opinions can be shared and participants can feel relaxed enough to voice their opinions.

In your role as discussion leader it is important that you ensure the group feels at ease and that the other students all have an opportunity to speak. Remember all participants should have read the paper -- so everyone should have something to say! However, you should have read the paper the most thoroughly out of anyone in the class, you will be the "expert" on this paper. As discussion leader it is your responsibility to guide the discussion,but not to dominate or monopolize the time available.

Some Additional Dos:

• The discussion should begin with you providing a brief synopsis of the paper -- not a summary of all methods and results. Don't spend all yout time on this if your goal is truly to have a discussion!
• Then list your questions. Those without a definite answer will best promote discussion.
• Try to encourage the class to think beyond the few papers they have read for this class -- there is a lot more research literature out there, does it all say the same thing?
• If your paper contains a figure that could help facilitate discussion, then include it on a slide and have it on display as the related point is being discussed.

• Try not to spend a lot of time on the small details of the paper -- focus on the bigger picture-- what does it all mean?
• Having pre-prepared questions is helpful but don't let them tie the discussion -- there is a balance between ensuring the issues you want to address get addressed and ensuring that a fruitful discussion ensues. Use the questions as guidelines to reign in discussion if necessary, but not too much!
• Try not to talk for too long, remember, this approach is different from a lecture!

Lecture

Rather than a discussion you may want to choose a lecture format for your critique. This format requires you to teach the class about the paper as if they did not know the content. You should be prepared to talk for around 30 minutes on the paper, answering questions from other students as you go. Your lecture should not only describe the content of the paper but how it is situated in the context of the field:

It is important to be prepared for your lecture, to create clear slides and to think about the arguments. A lecture will require you to structure the arguments, present insights, and inform the other students about the content of the paper. From this perspective it is more challenging than leading a discussion. However, with a lecture you can perhaps be more certain that the points you are interested in will get addressed. Please remember that there will likely be some discussion of the paper even if you choose this option, but unlike the Discussion format, the discussion will occupy less time than your own oral contribution.

Some Additional Dos:

• Plan your lecture and prepare in advance
• Know what you want to say before you arrive in the class -- make cue cards to help you remember key points if you are not confident that you can talk directly from your slides.
• Lectures should have an introduction, a main body of information, a summary, and some questions for discussion

Some Additional Don'ts:

• Do not think about what you are going to say for the first time as you say it. You will make mistakes, and the focus of your lecture will be lost.
• Do not assume that other students have read the paper thoroughly -- how much they concentrated when reading it will depend on their time and level of interest

Some Questions to Consider

Overview

• What's the paper about?
• What are the contributions?
• What is the main line of argument in the paper?
• What is the paradigm?

Significance and originality

• Were the ideas novel or original at the time?
• Are the contributions significant?
• What are the advantages and limitations of the approach?

Soundness

• Are the ideas that are presented technically correct?

Empirical evaluation

• Are the ideas evaluated using a study design that is appropriate for questions that the authors wished to answer?
• If some assumptions are made, are they realistic?

Related work

• Is the paper properly situated with respect to related work?

Readability

• Is the paper well structured?
• Is the paper well written?

You may also want to consider issues such as: If you were reviewing this paper for publication, what would you recommend? Given the arguments and findings, would you have written the paper differently?

It is important that you are selective in choosing issues for discussion or those to focus your lecture on. There is no sense in discussing something that has a definite answer and there is no sense lecturing about something that is trivially simple (or intractably difficult!). Also, BE SKEPTICAL, don't just accept it because it’s written by a well-known author in a well-respected journal – question all arguments, methodologies and findings.

Grading

You will receive public and private feedback from the instructors. Grading is a cummulative impression -- you will not receive a numerical grade for individual in-class activities. Rather, what you will get is our comments and impressions of what went well and what might have gone better.