Chapter 8: Conducting Literature Reviews

Having explored various contribution types and detailed explanations of empirical and constructive research approaches, along with guidance for starting your first HCI project, we now turn to another fundamental component of rigorous research — conducting comprehensive literature reviews. This critical process forms the foundation for identifying meaningful research gaps, building theoretical frameworks, and positioning your work within the broader HCI landscape. Whether you’re validating a new interaction technique or developing novel evaluation methods, systematic literature review methods will help you establish credibility while avoiding redundant efforts.

8.1 Selecting a Research Topic

Before diving into literature review techniques, it’s important to first understand how to select a research topic worth investigating. According to Prof. Tan Chuan Hoo from the National University of Singapore, there are several effective approaches to choosing a research topic, rather than relying on inspiration alone or forcing a topic to fit existing data.

Picking a good research topic is one of the most challenging aspects of research. However, due to this difficulty, many researchers fall into problematic approaches. Some wait passively for inspiration or " eureka moments " rather than actively searching for topics. Others work backwards by trying to force a topic to fit existing data or methods they already have. And some select topics based purely on convenience rather than research merit. These approaches typically lead to poorly motivated research and may frustrate advisors who expect more thoughtful topic selection.

Roberts recommend a more systematic approach to topic selection ( Roberts, 2010 ) . The first step involves deep immersion in the relevant literature while engaging in meaningful discussions with faculty and other scholars in your field. Writing about potential topics helps crystallize and organize your understanding. Most students will consider between three to five potential topics before finally settling on one, and it’s completely normal to scrap a topic and start over at least once during the process.

When searching for potential topics, researchers should draw from diverse sources. Personal/professional interests serve as a natural starting point — consider what excites and energizes you, and what career goals could be enhanced by studying a particular topic. Faculty members, professional colleagues, and fellow students can provide valuable suggestions, though it’s important to maintain independent judgment while considering their advice. Professional journals in your field and digital libraries provide essential resources, and creating targeted keyword lists can help initiate focused searches.

Previously written dissertations and PhD defense discussions often suggest promising research directions. Keeping track of current theories in your field helps identify new developments or existing theories being questioned. Conferences and seminars provide opportunities for direct dialogue with presenters and authors about researchable topics. Outside agencies, professional organizations, leading scholars in your interest areas, and your current workplace can all generate valuable research ideas.

When evaluating potential topics, several key criteria warrant careful consideration. The topic must sustain your interest over an extended period, as completing a paper/dissertation often takes longer than anticipated. It needs to be manageable in scope — attempting Nobel Prize-level research is best reserved for postgraduate work. The topic should have clear potential to make an original and significant contribution to knowledge, typically by addressing an identified gap in the existing literature.

Practical considerations carry equal weight in topic selection. The research must be feasible within your available time frame and budget constraints, with reliable access to necessary data. The topic should not be over-researched, as this limits opportunities for original contribution. Finally, it must gain acceptance from your advisor and committee members, whose support proves crucial throughout the research journey.

8.2 Introducing Literature Review

Once you’ve selected a research topic through careful consideration of the criteria above, the next crucial step is conducting a thorough literature review. It plays a crucial role throughout the entire HCI research process, warranting dedicated attention as a fundamental research skill. From a knowledge creation perspective, it serves multiple critical functions:

1. At the problem identification stage, it helps researchers spot genuine knowledge gaps and research opportunities, ensuring their work addresses real needs rather than reinventing existing solutions.
2. During research design, it provides theoretical foundations and methodological insights from related work, helping researchers build upon proven approaches while avoiding known pitfalls.
3. For knowledge synthesis, it enables researchers to position their contributions within the broader landscape of HCI research, demonstrating how new findings advance or challenge existing understanding.
4. In the evaluation phase, it provides frameworks and benchmarks against which new contributions can be assessed, ensuring research meets the field’s standards for rigor and significance.

8.2.1 Basics: Search Directions and Strategies

Having established the core activities of literature review and their purposes, let’s explore specific search strategies that can help execute these activities effectively.

Backward search methodology , sometimes called " snowballing, " involves systematically reviewing reference lists of key papers. This approach is particularly effective during the initial exploration phase, helping identify seminal works and theoretical foundations in your field. When examining references, focus particularly on papers that appear repeatedly across multiple sources — these often represent fundamental contributions to the field. Webster and Watson ( 2002 ) recommend this approach as crucial for ensuring comprehensive coverage of prior research.

Forward search methodology naturally supports the " following research threads " activity by looking at papers that cite your key references. This approach, facilitated by tools like Google Scholar’s " Cited by " feature, helps track how ideas have evolved and identifies current developments in the field. It’s particularly valuable for understanding the current state of research and identifying potential research gaps. This method can reveal how initial concepts have been applied, critiqued, or extended in different contexts.

The connected papers approach enhances knowledge synthesis by providing a network-based view of literature relationships. Tools like Connected Papers ( connectedpapers.com ) and VOSviewer visualize connections between papers based on bibliometric coupling and co-citation analysis. This approach can reveal clusters of related research that might not be immediately apparent through traditional keyword searches, supporting both systematic reading and synthesis activities.

Venue-specific searching complements these approaches by systematically exploring key publication venues in your field. This method supports all core activities by helping you stay current with the latest research while understanding historical developments. Beyond just searching major journals and conference proceedings, this approach includes examining special issues, workshop proceedings, and technical reports. For HCI researchers, this might mean regularly reviewing proceedings from venues like CHI, UIST, and CSCW, while also monitoring specialized venues relevant to their specific research focus. Many venues maintain digital libraries (like the ACM Digital Library ) that offer advanced search features and paper recommendations.

8.2.2 The Iterative Nature of Literature Review

Literature review is an iterative process that becomes more focused and effective with each cycle. Here’s a systematic approach to conducting literature reviews:

When you have a specific research topic in mind, start by searching for researchers and papers that directly address that topic. Look for high-quality recent papers to understand the current state of research. Be aware that your topic may be discussed across multiple fields using different terminology — consulting with experts can help identify these alternative terms and perspectives. The literature review process often accelerates once you discover seminal papers that clearly define your topic and provide comprehensive overviews — these can serve as valuable maps for exploring both their references (backward search) and papers that cite them (forward search). If available, survey papers are excellent starting points as they synthesize key work in the field and can help identify the most important papers to read in detail.

Once you’ve identified some key papers, develop a systematic search strategy. Create targeted search strings using Boolean operators to search across multiple academic databases. Focus on specific venues and journals that publish relevant work in your area. Keep track of which search terms and combinations yield the most useful results. Tools like Connected Papers can help visualize citation networks and relationships between papers, revealing additional relevant work you may have missed through traditional searches.

Throughout the process, continuously refine your approach:

- Adjust keywords based on relevant papers found

- Narrow or broaden search scope as needed

- Document effective search patterns

- Focus more time on high-quality papers that advance your understanding

- Maintain a structured bibliography with detailed notes on key papers

- Regularly reassess your search strategy and be prepared to pivot based on new insights

- Organize papers by themes and approaches

- Map relationships between different research directions

- Identify potential gaps and contributions

The goal is to progressively refine your understanding while ensuring comprehensive coverage of relevant literature. Each iteration should build upon insights from previous cycles, becoming more focused while remaining open to new directions and perspectives that emerge.

8.2.3 Literature Survey Decision Charts

Pierre Dragicevic ( Dragicevic ) provides two valuable decision charts to help researchers navigate the literature review process. While these charts don’t cover aspects like search tools, bibliography management, or writing literature surveys, they offer clear guidance on key decisions when encountering papers — such as whether to read them in detail and how to incorporate them into your related work section.

The first decision chart outlines a general framework for handling papers encountered during research. The second chart helps assess a paper’s relevance, assuming you have already defined your research problem and potential solution approach. Together, these charts provide a systematic way to efficiently process papers while maintaining focus on your research goals.

8.2.4 Types of Literature Reviews

Literature reviews do not just happen at the start of the research process, but have to be performed iteratively at various stages of the research process. Based on the stage of research you are at, the type of literature review that HCI researchers conduct differs according to the goals of that stage.

8.2.4.1 Viability Literature Review (Scoping)

Purpose and goals: To evaluate if a research idea is viable by assessing whether it has already been done and identifying its unique contribution. Used before committing to a research question, with the goal of quickly mapping the existing literature territory and ensuring the idea’s novelty. Helps researchers validate potential research directions through rapid assessment iterations.

Process and timeframe: Involves scanning titles and abstracts of dozens to hundreds of papers over 1-3 days. When encountering papers with potentially overlapping titles or abstracts, researchers should first quickly scan through the paper to assess if there is actual overlap before investing time in detailed reading. This rapid assessment helps efficiently eliminate non-relevant papers. Only if initial scanning suggests genuine overlap should researchers do an in-depth review. Researchers identify and search key terminology, building foundational knowledge through focused keyword searches. If substantial overlap is found with existing work, researchers should refine or redirect their idea.

Best practices: After reviewing papers, use mindmapping as a tool to classify and organize the research landscape around your specific topic. Focus the mindmap on the immediate area around your research idea rather than trying to map the entire field. The goal is to understand how your work fits into and differs from closely related work, not to create an exhaustive map of all possible connections. Carefully examine papers that appear similar to determine if they truly overlap with your proposed idea. For each important keyword, conduct focused searches to find directly relevant literature. Remember that the entire viability check, including paper review and mindmapping, should not take more than 3 days — if you’re spending longer, you’re likely going too deep. Success is indicated by your ability to clearly articulate how your proposed work relates to and differs from the most closely related existing research, without getting lost in mapping tangential areas.

When to use this approach: At the beginning stages of research before research question is decided. Most frequently used type of literature review, requiring several iterations. Uses flexible inclusion/exclusion criteria and broad overview level of detail. Appropriate when validating potential research directions and establishing the unique positioning of a research idea within the existing literature.

8.2.4.2 Formal Literature Review

Integration with research papers: Conducted at different stages, often when main studies are completed and results/insights developed. Forms the literature review section after the introduction. Groups previous literature to clearly distinguish researcher’s contribution from previous work. Requires deeper analysis of each paper with detailed notes integrated into research question’s story.

Documentation methods: Take detailed notes on papers (usually more than 50), creating narrative that weaves into research question story. Uses spreadsheet approach with database containing different tabs and classifications. Highlights relevant papers and shows clear relations to current work. Requires specific inclusion/exclusion criteria based on research question.

Best practices: Maintain database with classifications, highlighting most relevant papers and their relation to current work. Based on study results, identify related studies, extensions, disputed findings, and nuanced additions. Use step-by-step approach with examples and writing templates. Justify designs and research interests through comprehensive documentation.

Common applications: Used to support research findings and distinguish contributions after key studies are complete. Takes 1+ weeks, requiring detailed analysis with specific criteria. Helps researchers understand what studies they’re extending, disputing, or adding nuances to. Creates formal structure within research paper to support research story.

Example: In this video , PhD student Julia shared her effective system for organizing research papers using a spreadsheet approach. Her spreadsheet contained columns for: 1) the full citation, 2) which section of the literature review the paper belongs to (e.g., theoretical foundations, methodology, related systems — these sections correspond to major themes or topics in her thesis), 3) a one-sentence key message about why it’s important for her work, 4) detailed key points from the paper, and 5) critiques/limitations noted. She would first populate the spreadsheet with citations from papers she had already collected, then work through one section at a time — reading papers, taking focused notes on key points relevant to her work, and writing a summary sentence for each. This systematic approach allowed her to efficiently organize hundreds of papers, maintain clear documentation of important findings, and write her literature review more effectively by focusing on one thematic section at a time with all relevant papers and notes readily accessible.

8.2.4.3 Systematic Literature Review

[This section is based on notes and slides prepared by Sonne Chen]

8.2.4.3.1 Overview and Definition

Systematic review is a scientific research method that comprehensively analyzes the evidence of a particular subject through a systematic and transparent method. In the following we will detail the difference between system review and literature review, the type of system review, data extraction and synthesis methods, and the specific steps of how to conduct system review.

8.2.4.3.2 Comparison with Traditional Literature Review

Literature Review

• R ange: It usually has a wide scope and may include discussion of past work, criticism, and identification of current knowledge gaps.
• Methodology: Non-standardization may not have a clear search strategy or contain/exclude standards.
• P urpose: Provide a comprehensive overview of a topic and identify new research directions.
• limitations: There may be prejudice, it is difficult to copy, and it may be too specific.

Systematic Review

• R ange: The preset review scope is usually narrower and more specific.
• S earch: A comprehensive search of all relevant research has a clear search strategy.
• S tandard: Use clear criteria to include/exclude research.
• Quality assessment: Critically evaluate the quality of research.
• Data extraction: Extract and synthesize research results.
• purpose: Providing a reliable basis for decision-making may involve meta-analysis.

8.2.4.3.3 Core Characteristics

The system review has the following characteristics:

- transparency : The research process and methods are open and transparent to facilitate external review.

- Reliability : Reduce prejudice and improve the reliability of research through a systematic approach.

- Decision basis : Provide scientific basis for policy formulation and practice.

8.2.4.3.4 Data Synthesis Methods

Meta-analysis

Definition

Meta-analysis is a statistical technique that allows researchers to combine the results of multiple studies to assess the overall effect of an intervention or variable. This method is particularly suitable for quantitative research because it depends on the consolidation of statistical data.

Applicable conditions

• Research questions must be answered through quantitative data.
• The selected research must be sufficiently similar in design, measurement and analysis so that the results can be reasonably combined.

Process

• Select research : Select research based on clear inclusion and exclusion criteria.
• Data extraction : Extract the effect amount from each study (such as average difference, correlation coefficient, etc.).
• Statistical synthesis : Use statistical methods (such as weighted average) to merge the effects.
• Assess heterogeneity : Check the variability between the combined studies to determine the consistency of the results.
• Forest map : Forest maps are usually used to visualize the combined effects and the contribution of various studies.

Advantages and Limitations

Advantages:

- Provide quantitative and objective comprehensive results.

- You can detect patterns or trends that may not be obvious in a single study.

Limitations:

- Research is required to be highly consistent in design and measurement.

- May obscure the unique findings or background differences of individual studies.

Narrative Analysis

Definition

Narrative analysis is a qualitative research method that focuses on the detailed description and interpretation of the research results rather than statistical synthesis. This method is suitable for qualitative research or quantitative research that is difficult to combine with statistical methods.

Process

1. Select research : Select relevant literature based on research issues and inclusion/exclusion criteria.
2. Data extraction : Extract key themes, concepts and discoveries.
3. Theme analysis : Identify and compare topics and models in different studies.
4. Comprehensive explanation : Based on the extracted data, provide a comprehensive explanation of the research results.

Advantages and Limitations

Advantages:

- Allow in-depth discussion of the diversity and complexity of research results.

- Can handle research that is not suitable for meta-analysis.

Limitations:

- May be affected by the subjectivity of researchers.

- The results are not as statistically accurate as meta-analysis.

8.2.4.3.5 Types of Systematic Reviews

Systematic Review

Purpose: Identify global research on a specific topic, recommend best practices, and provide information for policies.

Examples:

- Comparative effectiveness of telemonitoring versus usual care for heart failure: a systematic review and meta-analysis ( Yun et al., 2018 )

- Experiences of learning, development, and preparedness for clinical practice among undergraduate paramedicine students, graduate/intern paramedics, and their preceptors: a qualitative systematic review ( Hanna et al., 2021 )

- Determinants of weight loss maintenance: a systematic review ( Varkevisser et al., 2019 )

Scoping Review

P urpose: Identify existing research on a wide range of topics or a series of research issues. Features: The existing evidence may be critically evaluated, but it does not attempt to synthesize the results like a systematic review.

Examples:

- Engaging with nature through technology: A scoping review of HCI research ( Webber et al., 2023 )

- Dying, Death, and the Afterlife in Human-Computer Interaction ( Albers et al., 2023 )

- The use of flipped classrooms in higher education: A scoping review ( Boonprakong et al., 2025 )

8.2.4.3.6 Conducting a Systematic Review

Before starting the system review, it needs to be decided according to the following criteria:

- Scope of review: Whether all evidence needs to be collected.

- Manpower and time: Are there sufficient resources.

- Research question: Whether it is specific and operational.

- Scientific method: Are you willing to follow strict scientific methods/agreements.

- Meta-analysis: Is it possible to involve meta-analysis.

Key Considerations:

- Determine the type of review : Select the appropriate type of review according to the research purpose.

- Follow the guidelines: Refer to relevant research guides, such as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses).

- Expertise: Ensure that the team has expertise in related fields.

- Collaborators: Identify potential collaborators, especially experts in emerging fields.

- Format and requirements: Determine the format requirements of the journal and whether a letter is required.

- Journal selection: Choose a journal suitable for your review.

- Inclusion/exclusion criteria: Clearly define the inclusion and exclusion criteria of the study.

- Data coding: Consider the reliability between data coding and scoring.

- Database update : Ensure that the database used is up-to-date.

- Time efficiency: Set a timetable to ensure that the project is completed on time.

System reviews are a powerful research tool that can help us understand knowledge in specific fields more deeply and provide a basis for practice and policy development. By following the guidelines provided in this article, you can effectively conduct a systematic review and contribute high-quality research to the scientific community.

8.2.4.3.7 Comparison of Literature Review Types

	1. Viability Literature Review	2. Formal Literature Review	3. Systematic Literature Review
Stage	Beginning of study (Before RQ is decided)	After all key studies complete	Applies to certain paper types such as surveys
Purpose	Determine if idea is workable	Support research story, findings from previous results and current results	Survey a large dataset of past studies, analyse and draw conclusions on patterns, and relationships
Research question	Not yet determined	Established	Established
Inclusion criteria	Flexible	Specific to research question	Specific to research question
Exclusion criteria	Flexible	Specific to research question	Specific to research question
Number of papers reviewed	Depends on level of familiarity with topic. Could be dozens to a hundred	Varies, but usually more than 50	Typically over 1,000 (dependent on the submission venue’s requirements)
Level of detail	Broad overview	Detailed analysis	Detailed analysis and synthesis
Narrative	Limited	Detailed and integrated with research question	Comprehensive and integrated with research question
Structure	Not formal	Formal literature review section of research paper	Structured survey or meta-analysis
Timeframe	1-3 days per cycle	1 week or more	Several months to a year or more
Frequency of use	Most frequently used	Used once per paper	Less commonly used, typically for specific contributions or PhD thesis

8.3 Effective Paper Reading

The success of any literature review heavily depends on how effectively researchers can extract, comprehend, and synthesize information from academic papers. While having good search strategies helps identify relevant papers, the ability to efficiently read and process these papers is equally crucial.

Here’s a proven method for reading academic papers systematically and efficiently that has helped many researchers:

Prof. Pete Carr’s method for efficiently reading academic papers provides a structured three-pass approach that helps researchers extract key information while managing their time effectively:

Pass 1 (5-10 minutes): - Read the title, abstract, and introduction thoroughly - Read section and sub-section headings - Read the conclusions - Glance at references, mentally noting ones you’ve read This first pass helps you grasp the “big picture” and decide if you need to read further

Pass 2 (up to 1 hour): - Read the paper more carefully, but skip mathematical proofs or complex methodological details - Note key figures, diagrams, and illustrations - Mark relevant unread references for follow-up This pass helps you grasp the paper’s content without diving too deep into details

Pass 3 (1-5 hours): - Attempt to virtually re-implement the paper - Identify and challenge every assumption - Think about how you would present the ideas - Compare with related work - Note ideas for future work This deep pass helps you truly understand the paper’s contributions and limitations

Additionally, drawing from other academic reading guides, here are complementary practices to enhance your paper reading:

• Critical Reading Practices: - Question the authors’ assumptions and methodology - Evaluate if they are solving the right problem - Consider if their data gathering and interpretation is sound - Make notes in margins about questions and criticisms
• Creative Reading Elements: - Look for potential applications or extensions of ideas - Consider possible improvements - Think about what research you would do next - Identify ways to generalize the findings

This systematic approach helps researchers efficiently process large volumes of literature while ensuring thorough understanding of important papers. The three-pass method can be adjusted based on the paper’s relevance - some papers may only need a first pass, while crucial papers warrant all three passes.

Additional Comprehensive Guides: " How to (seriously) read a scientific paper" by Science Magazine

8.4 Documentation and Organization

While effective search strategies and reading techniques form the foundation of a successful literature review, equally important is how researchers document and organize their findings. Without proper documentation and organization systems, valuable insights can be lost or become difficult to retrieve when needed. This section explores key approaches to managing the wealth of information gathered during the literature review process.

A systematic approach to extracting and organizing information from research papers is essential for effective knowledge management during literature review. The key is to develop a consistent structure that captures both the objective details of the paper and your own analysis and insights. This structure should be comprehensive enough to include all critical information while remaining flexible enough to accommodate different types of papers and research approaches. Below is a practical framework that demonstrates how to organize paper information in a way that facilitates both immediate understanding and future retrieval.

The structure includes sections for basic metadata, core research elements, methodological details, key findings, and your own analysis. This comprehensive yet flexible approach ensures you capture not just what the paper says, but also how it relates to your research and what questions or insights it generates. By maintaining this structured approach across all papers you review, you create a valuable knowledge base that supports both your current research and future work in the field.

Here’s a concrete example of how to structure your note-taking for each research paper:

1. Basic Information

• Full citation (authors, year, venue)
• DOI/URL for easy reference
• Number of citations (to gauge impact)

2. Core Research Elements

• Research Question: “How does screen size affect mobile reading comprehension?”
• Objectives: “To measure reading speed and comprehension across 3 different screen sizes”
• Key Terms: Define important terminology used (e.g., “comprehension score”)

3. Methodology Details

• Study Design: Within-subjects experiment with 24 participants
• Data Collection: Reading speed (WPM), comprehension tests, eye-tracking
• Analysis Methods: ANOVA, post-hoc t-tests
• Key Variables: Independent (screen size), Dependent (WPM, comprehension)

4. Main Findings

• Bullet-point list of key results (e.g., “7” screens showed 15% faster reading than 4” screens”)
• Statistical significance and effect sizes
• Important figures/tables to reference

5. Your Analysis

• How this relates to your research: “Could extend this to AR displays”
• Limitations you’ve identified: “Only tested with English text”
• Questions for follow-up: “What about different text formats?”
• Potential citations to look up from their references

Using this structured framework ensures you capture all essential information systematically and can easily retrieve it later when writing your literature review or research paper. While detailed paper-level documentation is essential, researchers often need a higher-level view of how different papers and concepts relate across the research landscape. Mind mapping complements this documentation by helping researchers visualize and understand broader connections, themes, and patterns that emerge from the literature. Creating visual maps allows researchers to identify clusters of related work, trace how ideas have evolved, and spot gaps in the literature that might not be apparent in linear note-taking.

Tools like XMind or MindMeister provide flexible platforms for creating and updating mind maps as understanding grows (See Figure 8.1 ). For example, when conducting a literature review on " HCI & Games " , a mind map might look like this:

Figure 8.1: Example of a mind map used to organize a literature review on HCI & Games

The mind map above shows how different aspects of AR in education research connect — from technical considerations like display types and tracking methods, to pedagogical approaches and learning outcomes, to evaluation methods. Regular revision of these maps can reveal new patterns in how different subtopics connect to form a broader research landscape, and help plan the narrative structure for writing literature reviews. This provides a valuable way to step back from the details and see how different pieces of research fit together into a cohesive whole.

Beyond mind mapping, effective database management of research papers is crucial for a successful literature review. Reference management software like Zotero or Mendeley can automate citation tracking and PDF organization, but researchers need thoughtful systems to make the most of these tools. For example, when using Zotero, you might create collections for different research themes (e.g., " AR Interaction Techniques " , " Mobile AR Studies ") while using tags to mark papers by methodology (#controlled-study, #field-study) and contribution type (#empirical, #artifact).

While reference managers like Zotero provide robust functionality for organizing papers, some researchers prefer using spreadsheets as an alternative or complementary approach. A spreadsheet can offer more flexibility in customizing views and tracking specific details. Here’s an example spreadsheet structure that some researchers find useful:

• Sheet 1: Citation Analysis

• Basic metadata (title, authors, year, venue)
• Citation count and impact metrics
• Key research contributions
• Research problem addressed
• Relevance to my work (1-5 stars)
• Sheet 1 focuses on capturing the fundamental information and initial assessment of the paper’s relevance and impact

• Sheet 2: Research Design

• Research questions/hypotheses
• Methodology and study design
• Sample characteristics
• Key variables and measures
• Analysis techniques
• Limitations identified
• Sheet 2 dives into the technical details of how the research was conducted

• Sheet 3: Knowledge Synthesis

• Main findings and implications
• Connection to theoretical frameworks
• Relationship to other key papers
• Gaps/opportunities identified
• Future research directions
• My critical analysis notes
• Sheet 3 focuses on higher-level synthesis and integration of the paper’s findings with existing literature

As shown in Sheet 3 of the spreadsheet example above, synthesis is a critical part of the literature review process that goes beyond just organizing information. The " Knowledge Synthesis " sheet specifically focuses on connecting findings across papers, identifying gaps, and developing critical analysis — but how do we actually perform this synthesis effectively?

Here are specific synthesis strategies that researchers can use to develop deeper understanding:

• Create comparison matrices: Make tables comparing key aspects across papers, such as:

• Study methodologies (e.g., lab study vs field study)
• Sample sizes and participant demographics
• Key findings and effect sizes
• Limitations identified

• Track evolving themes: Keep a running document of:

• Recurring concepts that appear across multiple papers
• How definitions of key terms change over time
• Major shifts in research approaches

• Map contradictions: When you find conflicting results, document:

• The specific claims that conflict
• Potential reasons for the contradiction (different methods, contexts, etc.)
• Questions that need investigation to resolve the conflict

• Write bi-weekly synthesis memos (1-2 pages) addressing:

• New patterns you’ve noticed across papers
• Questions raised by recent readings
• How your understanding has evolved
• Potential gaps or opportunities identified

• Develop visual concept maps showing:

• Relationships between key theories and findings
• How different papers build on each other
• Clusters of related work and methodological approaches

These concrete practices help transform individual paper notes into deeper analytical insights that can drive your research forward.

Regular maintenance of these documentation and organization systems ensures that literature review findings remain accessible and useful throughout the research process. While setting up these systems requires initial investment of time and effort, they significantly reduce the cognitive load of managing large amounts of information and make writing literature reviews and research papers more efficient. The key is to develop systems that are comprehensive enough to capture important details while remaining flexible enough to adapt as your research focus evolves.

8.5 Best Practices and Common Pitfalls

8.5.1 Recency Considerations

When conducting your literature review, carefully balance recency with relevance:

• Focus primarily on papers published within the last 10 years to ensure you’re building on current knowledge and methods
• Don’t automatically exclude older papers - some may be foundational to your area
• For rapidly evolving fields like HCI, more recent papers may be especially important as technologies and methodologies advance quickly
• Use citation counts and impact to help evaluate the ongoing relevance of older papers

8.5.2 Handling Seminal Papers

Seminal papers require special attention, regardless of age:

• Include highly-cited foundational papers that established key theories or methods in your field
• Understand how seminal papers have influenced subsequent research
• Look for papers that are consistently referenced across recent work in your area
• Consider both the historical significance and current applicability of classic papers

8.5.3 Quality Assessment

When evaluating papers, it’s crucial to read critically and form your own assessment rather than relying solely on venue reputation or accolades. While publication in prestigious venues like CHI indicates the work has passed peer review, this alone does not guarantee perfection or validate all aspects of the paper. Even papers from top venues can have limitations or questionable assumptions that warrant scrutiny.

Best Paper Awards often reflect papers that were exceptionally well-executed and clearly written rather than those making the most innovative or impactful contributions to the field. These awards frequently go to work that reviewers could easily evaluate and agree upon due to strong methodology and presentation. However, papers introducing more radical or transformative ideas may be more controversial and receive mixed reviews despite their potential significance.

When assessing paper quality, develop your own standards based on scientific merit and contribution. Examine whether the work meaningfully advances knowledge in the field, regardless of where it was published. Question the authors’ assumptions, methodology, and conclusions. Look for clear acknowledgment of limitations and careful reasoning. Consider both the technical execution and the innovative potential of the ideas presented.

Critical reading means engaging deeply with the content rather than accepting arguments based on venue prestige or awards. Take time to understand the authors’ reasoning, identify unstated assumptions, and consider alternative approaches or interpretations. This thoughtful analysis will help you extract maximum value from the literature while developing your own research perspective.

Common Question: How do I transfer RQs to suitable and related keywords when I’m doing the lit review. Any tips or suggestions?

Converting research questions into effective search keywords requires a systematic approach. Here are some key strategies:

1. Break down your RQ into core concepts

• Identify the main topics, variables, or phenomena you’re studying
• Example RQ: “How do AI-powered generative interfaces affect creative workflows?”
• Core concepts: generative AI, creativity support tools, workflow impact

2. Brainstorm alternative terms for each concept

• List synonyms and related terms
• Include both broader and narrower terms
• Example for “generative AI interfaces”:

• AI co-creation tools
• Large language models
• Generative design systems
• AI-assisted creativity

3. Consider relevant methodological terms

• Add terms related to your research approach
• Example: “user study”, “experiment”, “survey”, “qualitative”

4. Use Boolean operators effectively

• Combine terms with AND, OR, NOT
• Example search string: (“generative AI” OR “AI co-creation” OR “large language model ”) AND (”creative workflow ” OR “design process*“) AND (”user experience” OR “creative output” OR “productivity”)

5. Iterate and refine

• Review initial search results
• Note keywords used in relevant papers
• Adjust your terms based on what works

Remember to document your search strategy and keywords for reproducibility and future reference.

References

Roberts, C. M. (2010). The dissertation journey: A practical and comprehensive guide to planning, writing, and defending your dissertation. Corwin Press.

Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. MIS quarterly, xiii-xxiii.

Dragicevic, P. (n.d.). Literature survey decision charts. AVIZ, Inria. https://www.aviz.fr/literature

Yun, J. E., Park, J. E., Park, H. Y., Lee, H. Y., & Park, D. A. (2018). Comparative effectiveness of telemonitoring versus usual care for heart failure: a systematic review and meta-analysis. Journal of cardiac failure, 24(1), 19-28.

Hanna, H., Jordan, Z., Stern, C., & Pearce, J. (2021). Experiences of learning, development, and preparedness for clinical practice among undergraduate paramedicine students, graduate/intern paramedics, and their preceptors: a qualitative systematic review. JBI evidence synthesis, 19(9), 2052-2154.

Varkevisser, R. D. M., Van Stralen, M. M., Kroeze, W., Ket, J. C. F., & Steenhuis, I. H. M. (2019). Determinants of weight loss maintenance: a systematic review. Obesity reviews, 20(2), 171-211.

Webber, S., Kelly, R. M., Wadley, G., & Smith, W. (2023, April). Engaging with nature through technology: A scoping review of HCI research. In Proceedings of the 2023 CHI conference on human factors in computing systems (pp. 1-18).

Albers, R., Sadeghian, S., Laschke, M., & Hassenzahl, M. (2023, April). Dying, Death, and the Afterlife in Human-Computer Interaction. A Scoping Review. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (pp. 1-16).

Boonprakong, N., Tag, B., Goncalves, J., & Dingler, T. (2025, April). How Do HCI Researchers Study Cognitive Biases? A Scoping Review. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (pp. 1-20).