Now that you understand the two main approaches to HCI research — empirical and constructive — and have a good grasp of the overall research workflow, you're ready to embark on your first project. This chapter provides a comprehensive guide broken down into three key stages to help you navigate your first HCI research project.
Starting your first research project can feel overwhelming. You may wonder how to scope your research questions, conduct an effective literature review, or design your studies. You might lose sight of where individual tasks fit into the bigger picture of completing your project and publishing your work. This is completely normal — we've all been there. We'll walk you through each stage: from selecting and refining research questions, to exploring and validating them through preliminary studies, to conducting formal research including prototyping, controlled studies, and writing up findings. Throughout, we'll provide practical tips, examples from successful projects, and help you avoid common pitfalls.
This guide is designed to be a practical reference that you can return to throughout your research journey. We recommend:
|
1. Preparation |
2. Select Research Questions |
3. Explore, Concretize, and Validate your Research Questions |
4. Prototyping and Formal Studies |
|
|
Principle |
Build your background understanding on HCI research |
Identify a researchable topic that is original, narrow enough but important such that it is worth researching on |
Verify that the research questions provide interesting and important preliminary answers. Begin writing and developing a concrete aim |
Begin formal experimentation, analysis and complete paper writing |
|
Milestone |
a. Test background understanding |
a. Literature review b. Literature search c. Literature Mind mapping d. Concretize questions into mini research plan |
a. Choose research route b. Organize thoughts in writing c. Validate ideas through pilots d. Write up the result and discussion section based on pilots |
a. Design controlled experiment (5-step approach) b. Find and follow ‘good’ templates c. Explain ‘Why’ with theories/analysis d. Review writing guide |
Choosing a research topic is a crucial decision that deserves dedicated time and effort. It often requires several months of serious reading and investigation to truly understand the literature and research space. For a more comprehensive explanation of this important process, we highly recommend reading "The Craft of Research" by Booth et al. The approach we present below is more practical and hands-on, designed specifically for beginner HCI researchers taking their first steps.
|
Suggested Reading: " The Craft of Research " |
For more experienced researchers or those with a dedicated research focus, we encourage starting with a grand vision or significant problem that can shape your research direction for years to come. From this bigger vision, you can then identify and select specific subproblems to tackle as initial projects. However, this approach requires significant research maturity and may not be suitable for everyone starting out. The method below provides a path that helps beginners navigate the process of topic selection, while acknowledging that the rich skill of problem-finding develops with research experience.
Before performing a literature review, you can first choose your focus within 2 domains: Technological and Application
|
Technological domain |
Application domain |
|
|
With your technological and application domains defined, e.g. 'smart glasses' and 'text input on smart glasses', you can begin your literature review. We discuss how to conduct a thorough literature review in detail in Chapter 8 .
Narrowing your topic into something investigable
When your topic or scope of the problem is too broad (examples will be given below), and the design space is not constrained, there will be too many factors/variables that can influence the design of a potential solution. Going through them all will be exhaustive and difficult.
If the scope of the problem is narrow enough , fewer but important factors of clearer relevance can be identified. You will know better what factors to analyze. How do you know if your topic is narrow enough? It would depend on your problem, but a good gauge would be to limit your problem into 6 variables or less .
Pay attention to the following criteria to help narrow down the problem:
|
Context |
In what contexts will your technology be used? E.g. If you state ‘Mobility’ as your context, it may be too broad. A better-defined context could be “While commuting, on public transportation”, or “Standing, walking, and sitting” |
|
Interaction e.g. input/output modality, visual/audio/haptic |
Consider the types of interaction you want to focus on. Select which input and output channels to investigate. E.g. Visual output (primary), Auditory output (secondary) Manual and voice input, visual and auditory output. Take a look at Eyeditor paper ( Ghosh et al., 2020 ) |
Keep in mind as you find your focus, that the problem should not lose its originality (i.e. no one has worked on the same problem before) and importance (i.e. people care about solving this problem and it enhances our understanding of the phenomenon at hand).
To paint a clearer picture on how this concept of narrowing one’s topic can be applied, we provide two examples experienced personally by our PhD students Nuwan and Ashwin .
Examples
|
Nuwan started with the idea of working on AR (technological domain) and learning (application domain). After a literature review, the topic gradually narrowed down to “Visual attention on microlearning of the second language on the commute with public transport, comparing mobile phones and smart glasses”. |
|
|
Ashwin was initially interested in improving education (application domain) using mobile and wearable technology (technological domain). After several rounds of exploration and topic narrowing, he landed upon “Presentation effects (visual) in on-the-go (walking situations) for digital blackboard videos learning using smart glasses” . |
|
Common pitfalls as you narrow down on your topic
Narrowing down your topic can be a painful process. We highlight the common issues students face:
|
Common Issue |
Description |
Possible Solutions |
|
1. Paradox of choice |
There may be many possible topics to choose from. It can be difficult to select a specific area, but it has to be done. |
One useful approach is to ask your peers and advisors for help and feedback, then pick your topic without excessive delay. |
|
2. Project Ambition |
Wanting to solve a problem that covers many smaller problems. E.g. When designing notifications for smart-glasses, the researcher may want to cover both work related and non-work related scenarios, yet that is too broad as it would cover hundreds of different situations. Instead, the researcher could cover only short text notifications (with a maximum number of words) for on-the-go commuting scenarios. |
It is generally harder to start with something broad and narrow it down. We recommend starting with something small** (but grounded in the literature), then expand it later if it is too narrow. |
** ‘Small’
here refers to a concept that you can
concretely describe and implement.
Examples of concrete and narrow research topics include the
effects of 3 dimensions of information presentation (which can affect cognitive load and attention),
and
the use of fingers for ink annotations on mobile phones, to help with vocabulary memorization
.
You can always refer back to Chapter 8 on conducting literature reviews as you learn to find the right research question.
Mind-mapping the Literature
Mind-mapping is a great way to visualize and compile all the information you have collected during your literature review (especially if it feels like a lot of information!).
We suggest that you do the following:
To create mind maps, we suggest XMind if you do not have a preference for another software tool. Here is a quick video introduction to XMind .
With the approval of your lab members/advisor, you now have successfully narrowed down to 3-4 research questions. Let’s take it a step further and look at how research questions can be effectively framed, as well as what research route you can follow:
Research Routes
As discussed in previous chapters on empirical and constructive research, there are multiple approaches you can take to address your research problem. We encourage you to review those chapters to understand the different types of research work and determine which approach best suits your goals.
Your research route — your strategy to address the research problem — should align with your research objectives and desired impact. Key considerations include:
The answers to these questions will help guide how you concretize your research questions into actionable research plans.
Organize Your Thoughts in Writing
A trick to concretizing your research questions, and to see which of the 3-4 questions you currently have is best, is to begin writing the abstract and paper outline (problem, motivation, and potential contribution).
Of course, you don’t yet have any results. That is ok — in fact, we want you to speculate possible findings and use them as a guide for you to plan your research. These imagined results are essentially what you are looking for, and you can shape your study accordingly. Take 1-3 days working on the steps below:
Using Your Outline to Plan Next Steps
Once you have a promising outline, use it to strategically plan your next steps. The key principle is to focus first on your biggest unknown or the most critical factor that will determine the success of your project/paper.
The critical factor varies depending on your project/paper type:
By identifying and tackling the most critical unknown first, you:
Only after addressing the critical factor should you move on to other aspects like polishing the implementation, running additional studies, or exploring edge cases.
Set SMART Goals for Your Research Plan
Once you know what to focus on, you need to excute it, yet executing research goals can be challenging due to their inherently vague and open-ended nature. For instance, a task like "conduct literature review" provides little concrete direction. This is where SMART goals become particularly valuable in research planning — they help transform abstract research objectives into concrete, actionable tasks.
When planning your research steps, frame them as SMART goals — Specific, Measurable, Achievable, Results-focused, and Time-bound:
Specific: Define exactly what needs to be done
Measurable: Include concrete metrics and deliverables
Achievable: Set realistic goals given your resources and constraints
Results-focused: Focus on outcomes rather than activities
Time-bound: Set clear deadlines
For example, vague goals like "run some user studies" can become:
Using SMART goals helps you:
Remember to regularly review and adjust your SMART goals based on emerging insights while maintaining clear direction.
Validate Ideas through Pilots
About pilots: We conduct pilot studies to verify that our research questions provide interesting and important preliminary answers. The process is iterative, and we try to design quick and dirty pilots (elaborated below) such that it takes no more than 1-3 days for each study.
Why conduct pilots:
' Quick and dirty ' method: For each pilot study, focus on the main result you want to report in your paper, and get creative with devising a quick and dirty pilot to collect these results. For example, if your study explores a newly designed technique (e.g. a new menu system) that you believe can outperform the state-of-the-art (e.g. marking menu), you do not have to write out all the code to your new technique in order to pilot test it. A quick and dirty example would be to simulate technique usage with finger/pen, and use a timer to check for its performance. While this method may not be able to find subtle differences between your idea and the baseline, that's actually a good thing, as if the effect is subtle, you probably don’t want to waste time investigating it.
Depending on your research question, the Wizard of Oz method may be applicable. It is a process that allows a user to interact with an interface without knowing that the system responses are being generated by a human behind-the-scenes rather than the computer.
Tip: 'Quick and dirty' designs require creativity. Cool ideas may develop as a result of brainstorming with your lab members.
Contribution and feasibility check
It is important that the results of the pilot study support your story, and we do so by verifying the study's feasibility and contribution. This check can be approached differently for 'Be the best' papers and 'Be the first' papers.
|
'Empirical' Description |
Examples |
|
For empirical studies, pilots help verify if your research questions can yield interesting and publishable results. Run quick tests with 2-3 participants to check if your hypothesized effects exist and are meaningful enough to warrant a full study. If pilot results show promising trends that align with your research goals, proceed with the formal experiment. If not, consider revising your research questions or approach. The goal is to avoid investing significant time in studies that may not yield publication-worthy findings. |
For example, consider a study investigating whether AudioXtend improves learning. AudioXtend is a system that augments audiobook experiences with AI-generated glanceable visuals displayed through optical see-through head-mounted displays, designed to enhance information retention during multitasking scenarios ( Tan et al., 2024 ). A pilot might involve having 2-3 participants learn concepts through both pure audio and audio + visual conditions. If pilot data shows that participants remember significantly more information in the AudioXtend condition and it does not affect the performance of the primary task, this suggests the research direction is promising. However, if there's no noticeable difference in learning outcomes, you may need to revise your approach or explore different research questions. |
|
'Be the best' Description |
Examples |
|
Your pilot study should be able to indicate that your new technique can outperform the state-of-the-art technique in certain scenarios. A convincing technique usually leads to pilot results that can show advantages over the state-of-the-art even with just 3-4 people. If no convincing results are observed after pilot testing with 3-4 people, this could be a risky study to proceed with. We recommend that you either move to a different research question, or pivot by changing the scenario or technique (example listed on the right). The bottom line is that pilots for 'be the best' papers need to convince you of a winning case, otherwise, the study may lead to a null result and be difficult to publish. In certain cases however, a failed "be the best" technique can reveal important insight about interaction design. For example, in Ken Hinckley's Springboard paper , his initial idea was that the local marking menu would be better than a fixed widget placed at the corner of the screen. When the experiment results showed otherwise, he turned his study into an analysis paper which informed readers on the underlying causes of that. |
In the context of mobile eyes-free interaction, if you believe that your new menu system can outperform the marking menu , your pilot results should indicate a convincing win in speed and accuracy, i.e. either comparable in speed, but much more accurate, or comparable in accuracy but much faster in speed, or better in both speed and accuracy. Pivoting by changing scenario: If you explored your system but found that it did not significantly outperform the marking menu under the eyes-free mobile scenario, you could explore the same technique under a single handed usage scenario. If pilot results are favourable in the latter, you could craft a new story based on the winning case. You could also pivot by changing the technique . |
|
'Be the first' Description |
Examples |
|
A 'be the first' paper does not necessarily have to outperform the state-of-the-art existing technique (great if it does!), but to know its feasibility, check that the pilot demonstrates some of these characteristics, especially the first: The new technique/topic needs to be useful and desirable in certain usage scenarios, i.e. people like and want it. To check for this, implement a quick and dirty pilot on a few people - If the reaction it elicits includes "this is cool", "this is useful", or "I like it", then you are onto something interesting. If not, you may need to switch your direction. You explore the design space for potential solutions within the problem domain. Since it is a new technique for a novel problem domain you are proposing, reviewers want to know that your solution is well-thought out rather than arbitrary. You can do this by identifying the parameters considered as you were seeking the solution, and conducting a comparative study to demonstrate which configuration of the parameters reveal the best possible results. Once that is established, it is often useful to compare your approach with existing approaches. Compared to 'be the best' papers, the requirement for having a convincing winning case is more relaxed with 'be the first' papers, because usage scenarios can vary with the latter. Justify that in some cases it is more desirable to use your technique, and you will have a reasonable story. |
Ashwin's LSVP paper ( Ram & Zhao, 2021 ) is a 'be the first' paper introducing smart-glasses as a platform for watching videos. He first had to establish that watching videos on-the-go (using smart-glasses) is not only possible but useful in certain cases. Ashwin's LSVP paper compares LSVP with the mobile phone, and demonstrates that LSVP achieves better results in both learning and navigation cases. This is a convincing winning case. |
Statistical significance estimation in pilots
While pilot studies typically have too few participants for statistical significance, there's a useful estimation technique to predict if your full study might yield significant results. This technique, called the data duplication method, involves taking your pilot data (e.g., from 3 participants), duplicating it several times (e.g., 4x to simulate 12 participants), running statistical tests on the duplicated dataset, and checking if this shows significance. If significance emerges from this analysis, it suggests you may achieve significance with a larger sample in your full study.
Consider an example where 3 participants complete tasks with two different interaction techniques. The original data might show a clear trend but with p > 0.05. However, when duplicating this data 4x to simulate 12 participants, if the analysis shows p < 0.05, this suggests promising potential for the full study.
There are important caveats to consider with this approach. The method assumes future participants will show similar variance patterns, and small pilot samples may not be representative of the broader population. It's crucial to triangulate these statistical estimations with qualitative evidence. Consider whether all participants notice and articulate the difference, if their explanations are logically sound, and whether the effect aligns with theoretical expectations.
The strongest evidence that your pilot results are meaningful and worth pursuing in a full study comes from the combination of promising estimated statistical trends, consistent qualitative feedback, and sound theoretical reasoning. When these three elements align, they provide a solid foundation for proceeding with a full-scale study.
Write Up the Result and Discussion Section based on Pilots
Writing up pilot results and discussion sections is a crucial step that serves multiple important purposes. First, it helps organize your thoughts and findings in a structured way, making it easier to identify potential logical gaps or weaknesses in your argument. Second, it allows you to share concrete writing with peers and advisors who can evaluate whether your results are truly promising and your story compelling. Finally, it helps identify what additional data or explanations you'll need to collect in your formal study.
|
Section |
Key Elements |
|
Results |
Present pilot findings organized by research question. Document clear trends and patterns, even without complete statistical analysis. Include both quantitative metrics (e.g., task completion times, error rates) and qualitative observations from participants. Highlight any unexpected or particularly interesting findings. |
|
Discussion |
Analyze how results support or challenge your hypotheses. Use participant quotes and feedback to build your argument. Connect findings to existing literature to demonstrate relevance. Identify any gaps in your evidence or explanation that need addressing in the formal study. |
Aim to write up results and discussion for each promising pilot within 1-2 days while observations are fresh. Share these write-ups with your research team to get early feedback and ensure you're building toward a compelling research contribution. This iterative process helps refine your approach before investing in a full study.
If you’ve received good feedback from the previous steps, and feel confident that you have a viable project at hand, congratulations! You know now that you have good material to work with, though it is the next steps of your formal study that will determine whether you complete a successful cycle, and publish. Despite having a good problem with promising results to work with, students often fail due to weak methodology and poor writing.
In this section we revisit guidelines on how to design controlled experiments in HCI. We also highlight strategies to avoid the common pitfalls of this process, such as by following 'good' paper templates and using theories and analysis to support your results. Finally, we provide you with a comprehensive guide to writing your entire paper. We encourage you to write out parts of your paper even before you implement your formal experiment (to ensure that you do not miss out on important considerations). Let’s begin.
Designing controlled experiments in HCI
Once your pilot results show promise and you've drafted initial write-ups that receive positive feedback from your team, you can begin designing your formal study. This critical phase requires careful thought and methodological rigor to ensure valid, publishable results. For beginners, we recommend referring to Chapter 4 's structured 5-Step Approach to Experiment Design, which provides a systematic framework for conducting controlled HCI experiments. Additionally, Chapter 9 offers detailed guidance on writing up experiment designs — we suggest drafting your experimental protocol following these guidelines and sharing it with your research team for feedback before proceeding. Remember to be thorough and methodical in your planning, as a well-designed experiment forms the foundation for generating meaningful research contributions. Take time to consult with your advisor and lab members throughout this process to refine your approach and catch potential issues early.
Find and Follow 'Good' Templates
It is crucial that you find 'good' template papers in the next 1-3 days to benchmark your design methodology and writing against.
We encourage you to look for 1-3 well-established papers that you admire and that fit your research question and type. Use these papers as templates to structure your paper and design your methodology by.
There are two benefits to finding and following 'good' templates:
Here, we’ve compiled and categorized papers that could serve as a template for you:
|
Suggested Reading: " Great Paper Collection/Templates " |
Explain ‘Why’ with Theories/Analysis
Research is fundamentally the contribution of new knowledge. Support and explain your results with theories and analyses. Explain why something worked or did not in your experiment. Explaining how and why your method may be better/worse than something else that exists or the state-of-the-art.
For example, Keystroke-level Analysis provides one effective way of explaining an interaction technique ( video overview ). You can find this applied in the Springboard KLM analysis paper (page 7 onwards) submitted to CHI.
For more information on supporting theories and analysis, we recommend Chapter 7 of Scott MacKenzie’s book .
Guide to writing
For detailed guidance on writing your paper, please refer to Chapter 9 which provides comprehensive instructions on structuring and writing each section of your research paper.
Summary: Your First Project Journey
Your first HCI research project represents a critical milestone in your academic journey. While it is often the most challenging undertaking as you navigate unfamiliar territory, successfully completing your first project can be transformative for your research career. Many students discover that after overcoming this initial hurdle, subsequent projects become more manageable as they build confidence and develop their research instincts.
This guide provides a structured approach to help you through the key stages — from selecting your research questions to conducting formal studies. However, it's important to recognize that research is inherently complex, with daily decisions and obstacles that require careful navigation. We strongly recommend pursuing your first project under the guidance of an experienced mentor who can provide contextual advice, help you avoid common pitfalls, and support you through the inevitable challenges that arise.
While the path may seem daunting at times, remember that persistence through this process is invaluable. Your first project teaches crucial skills in problem identification, experimental design, academic writing, and research methodology that will serve as the foundation for your future work. The hands-on experience gained cannot be replicated through coursework alone. By following this guide while leveraging mentor support, you'll be well-equipped to tackle this important milestone and emerge as a more capable researcher ready to make meaningful contributions to the field.
Additional resources
Congratulations! You have reached the end of the step-by-step guide and we hope it has equipped you with the necessary knowledge and confidence for beginning your first HCI project.
|
Additional Resources: |
Wang, B., & Grossman, T. (2020, April). BlyncSync: Enabling multimodal smartwatch gestures with synchronous touch and blink. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-14).
Han, T., Wang, S., Wang, S., Fan, X., Liu, J., Tian, F., & Fan, M. (2020, April). Mouillé: Exploring wetness illusion on fingertips to enhance immersive experience in vr. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1-10).
Kianzad, S., Huang, Y., Xiao, R., & MacLean, K. E. (2020, April). Phasking on paper: Accessing a continuum of physically assisted sketching. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-12).
Dai, J., & Moffatt, K. (2020, April). Making space for social sharing: Insights from a community-based social group for people with dementia. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1-13).
Ghosh, D., Foong, P. S., Zhao, S., Liu, C., Janaka, N., & Erusu, V. (2020, April). Eyeditor: Towards on-the-go heads-up text editing using voice and manual input. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13).
Tan, F. F. Y., Xu, P., Ram, A., Suen, W. Z., Zhao, S., Huang, Y., & Hurter, C. (2024, May). Audioxtend: Assisted reality visual accompaniments for audiobook storytelling during everyday routine tasks. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (pp. 1-22).
Ram, A., & Zhao, S. (2021). Lsvp: Towards effective on-the-go video learning using optical head-mounted displays. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5(1), 1-27.