Reviewing

picture - Reviewing

Gathering research, getting rid of rubbish and summarizing the best of what remains captures the essence of the science of systematic review. Nevertheless, although the need to synthesize research evidence has been recognized for well over two centuries, it was not until the 20th century that researchers began to develop explicit methods for this form of research.” (Grant & Booth, 2009, p. 92)

As a PhD candidate, you are expected to place your work in the context of current research. Reviewing scholarly literature, i.e. analysing and making a synthesis of various results, points of view or approaches, is critical to every work of research, but how this is done depends on the discipline and the requirements of the individual research project.

On this page will you learn about

  • aims and methods of reviewing literature
  • relating your own project to previous research
  • critical appraisal of research literature
  • different approaches to reviewing literature

Aims and methods of reviewing literature

It is recommended to conduct the literature review in the beginning of the research project, even though you might come back to it during the writing process. The final literature review (or the review parts) of your dissertation will be the result of a number of steps:

  1. Identifying keywords relevant to your research question or topic of research
  2. Literature search
  3. Evaluation, organization and analysis of results
  4. Reviewing literature (and writing using literature)

However, depending on the individual project and discipline, it may be necessary to repeat these steps during your PhD. The research question and the literature search and review are mutually interrelated:

It is recommended to gain an overview of your field of research from the start of your work. Regardless of whether you are writing a literature review as a separate part of your dissertation or whether you are reviewing literature throughout your writing, relating your work to previous research is a crucial requirement in all scholarly literature.

Reviewing literature helps you to

  • gain a general idea of your field of research
  • place your own work within the field of research
  • identify main developments and gaps in research
  • justify the relevance of your research project
  • redefine the scope of investigation and adjust your research question
  • explain your focus and justify delimitations
  • discuss different methods and theoretical approaches used regarding your topic
  • deepen and widen your search
  • assess evidence of the research literature in your field

The ‘literature review’ is […] your opportunity to engage in a written dialogue with researchers in your area while at the same time showing that you have engaged with, understood and responded to the relevant body of knowledge underpinning your research. (Ridley, 2012, p. 3)

Relating your research to previous research

Your PhD cannot be done in isolation from previous research. Your inclusion of existing literature will show how well you know your field of research, as well as demonstrating that your research will fill gaps in current knowledge. Read more about how to relate to previous research.

Critical appraisal of research literature

While you are searching, reading and evaluating, your search results will, on the one hand, focus your search and limit the amount of text you will have to read, while on the other hand, they will constantly lead you to new references and new aspects of the research. Evaluation is first and foremost a question of discipline-based judgements. Your evaluation can include

  • reading titles and abstracts
  • using bibliometric measures
  • using inclusion and exclusion criteria
  • reading selected literature in depth
  • saving search strategies to repeat searches
  • writing using literature
  • quality assurance tools


Inclusion and exclusion criteria

In order to define a framework for your search and to delimit the focus of your literature review, it can be useful to draft an overview of the inclusion and exclusion criteria for your literature review.

Inclusion and exclusion criteria: an example from a literature review in teacher education (Evans, Fraser & Cotter, 2014, p. 535)

The following criteria were used for the inclusion or exclusion of studies in this systematic review:

    1. The study was an evaluation of a program designed to reduce bullying in an elementary school or middle school setting; reducing bullying did not have to be the primary focus of the intervention, but could be one of multiple aims or a secondary aim. High school programs were excluded because children’s normative beliefs about aggression become stable in elementary school and beliefs about aggression are correlated with observed aggressive behavior (Huesmann & Guerra, 1997), such as bullying. In addition, reported rates of bullying are highest among Grade 6 students (Robers, Zhang, & Truman, 2012). Therefore, we focused on vulnerable age groups in which intervention programs might disrupt risk cascades.
    2. Bullying perpetration and/or victimization were required to be measured using self-report questionnaires, peer ratings, teacher ratings, or observational methods. Studies that did not include a measure of bullying were excluded.
    3. Programs designed to decrease aggression or increase social–emotional skills that were also implemented to decrease bullying and used a bullying measure to gauge program effectiveness were included.
    4. The study was published from June, 2009 through April, 2013. Dissertations written before June, 2009 but not published on ProQuest until after June, 2009 were included.
    5. At a minimum, the study used a control and intervention group design.
    6. Program effectiveness was measured by comparing students who received the intervention (i.e., treatment or intervention group) with students who did not receive the intervention (i.e., control group).
    7. Only studies published in English were included.
    8. An intervention was specified and treatment integrity was reported. That is, the treatment was defined and delivered according to established criteria. For example, a study examining the effect of an anti-bullying policy was excluded because no intervention was specified and outcomes could not be associated with an identifiable change strategy (Greytak, Kosciw, & Boesen, 2013).

 

Different approaches to reviewing literature

Methods of reviewing literature can vary considerably, depending on the discipline, topic and individual research project. The research process in medicine and the sciences is usually based on a predefined research question, a systematic search prior to the actual research and standardized forms of theses. In the humanities and social sciences, however, searching, reviewing literature and writing are most often intertwined parts of the research process. But stand-alone literature reviews also vary widely. A literature review may be

  • a separate chapter or integrated in the analytical section of the dissertation
  • a systematic or a narrative review
  • an exploratory or a summary review
  • a critical or a descriptive review

In order to learn more about typical methods and standards of reviewing literature in your field of research, it is recommended to study some dissertations and find examples of best practice.

Find out more about different types of literature reviews.

References

Arksey, H., & O’Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32. https://doi.org/10.1080/1364557032000119616

Boote, D., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15. https://doi.org/10.3102/2F0013189X034006003

Booth, A., Sutton, A., & Papaioannou, D. (2016). Systematic approaches to a successful literature review (2nd ed.). Los Angeles, CA: Sage.

Evans, C. B. R., Fraser, M. W., & Cotter, K. L. (2014). The effectiveness of school-based bullying prevention programs: A systematic review. Aggression and Violent Behavior, 19(5), 532-544. https://doi.org/10.1016/j.avb.2014.07.004.

Foster, M., & Jewell, S. (2017). Assembling the pieces of a systematic review : a guide for librarians. Lanham, Maryland: Rowman & Littlefield.

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Greenhalgh, T., Thorne, S., & Malterud, K. (2018). Time to challenge the spurious hierarchy of systematic over narrative reviews? European Journal of Clinical Investigation, 48(6). https://doi.org/10.1111/eci.12931

Krumsvik, R. J., Øfstegaard, M. & Jones, L. Ø. (2016). Retningslinjer og vurderingskriterier for artikkelbasert ph.d. Uniped, 39(01), 78-93. https://doi.org/10.18261/issn.1893-8981-2016-01-07

Moher, D., Stewart, L., & Shekelle, P. (2015). All in the family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more. Systematic Reviews, 4(1), 183. https://doi.org/10.1186/s13643-015-0163-7

Ridley, D. (2012). The literature review. A step-by-step guide for students (2nd ed.). London: Sage.

A s k -u s