Expert searching requires identifying key language describing a topic, combining it into logical statements that are correct for the syntax of a given database, and then ascertaining that the search is accurate, has the necessary scope, and matches the intent of the researcher(s) who designed the research question. Several tools, if done prior to the search development process, aid in the smooth and effective development of an expert search. This module discusses three practices that, when combined, increase precision, provide quality control, and clarify scope not only to the searcher but also to the project team. Librarians facilitating evidence synthesis can use these three practices in consultation with new teams, in intake forms for consultations, as activities to facilitate team development and cohesion or as a series of activities to move teams toward the development of a search strategy.
Frequently evidence synthesis projects begin when a researcher discovers an article that is directly in their topic of interest, but that points to a gap in the literature that the researcher is interested in exploring. Authors may also discover seminal articles, which are article that are highly influential in the area they are interested in studying and which point the way to other articles on similar topics. Articles such as these, when combined with other articles on similar conceptual tracks but in slightly different frames or lenses, serve as a test group for the quality of a search strategy. (Bramer 2018) If the researcher determines that a given article will be identified by an effective expert search, search designers can use the metadata of the article to identify if the search is calibrated correctly by cross checking to ensure that the citation is included in the resulting set. Cross-checking can be performed by using pre-programmed limiter functions in the database to search for the author or title and date. Boolean strings can also be written that will identify a specific article among a larger search set.
How do I find seminal or exemplar articles?
Using one of these three articles in the database specified, create a search string of at least 200 relevant articles that would elicit a search set of articles on a similar topic, and then confirm that the article listed in this box is within the final search set.
Hint: In order to do the confirmation, look for limiters that may be useful, or combine search strings in the search history using AND and NOT to reveal whether the search resulted in the article that you are interested in.
Fox et al. 2009. "Household peanut consumption as a risk factor for the development of peanut allergy". Journal of Allergy and Clinical Immunology, 123(2) https://doi.org/10.1016/j.jaci.2008.12.014
Calkins et al. 2025. "Signals and noise: How higher music education institutions define popular music". Research Studies in Music Education, https://doi.org/10.1177/1321103X241309291
Zahedi et al. 2025. "The impact of a fairy tale-like story on the food choices of preschool children". Appetite, (206)1, https://doi.org/10.1016/j.appet.2024.107839
Identifying relevant keywords can be a time consuming part of any evidence synthesis search. Due to the complexity of jargon in a variety of disciplines, the inconsistency of vernacular terminology depending upon geographic origin, disciplinary enculturation, other factors unique to each individual author of potential resources, and highly technical descriptors arising from patents, law, government regulatory guidance, it can be difficult to anticipate which terms will result in the most complete search possible. Techniques exist to help with this process, and the use of existing tools can streamline this process.
Snowballing
In the snowballing technique, you use a database such as Scopus or Google Scholar that includes cited references, and you screen all of the articles that cite your exemplar article for keywords and concepts that would be useful for your research project. This involves not only adding relevant articles to the list of potential exemplar articles, but also involves creating a list of terminology that is used to describe your concepts. This may include gathering specific variables or constraints that are common to the articles that you anticipate would be in the resulting search set. Note: In the case of articles published in the past year, they are unlikely to have citing articles due to the recency of their publication. In that case, family tree searching or hand searching are more likely to be useful.
Family Tree searching
Family tree searching is an extension of snowballing. In this case, look at all of the articles cited in your exemplar, and then investigate the papers that those articles cite, and so forth. Make notes of recurring keywords, themes, variables or constraints that emerge as you immerse yourself into the intellectual roots of your exemplar paper.
Hand searching
Search issues of the journal that published your exemplar article, and identify any other articles published in that venue that relate to your research question. Again, create a list of relevant terminology that is used to describe your concepts. Gather any variables or constraints that are common to the literature that are relevant to your research question.
For one of the three articles below, identify keywords by using either Snowballing (searching all works that cited it), Family Tree searching (searching articles that it cited), or Hand Searching (looking at all articles published in the same journal over time). Note relevant keywords as they emerge from the process.
Siirtola. 2019. "The cost of pie charts". https://doi.org/10.1109/IV.2019.00034
Campbell and Hazelrig. 2023. "Tolerancing for an apple pie: A fundamental theory of tolerances". https://doi.org/10.1115/1.4057040
Welters. 2015. "The natural look: American style in the 1970s." https://doi.org/10.2752/175174108X346959
Data dictionaries are documentation created to document the data files created in research projects. They originate from the scientific notebooks that have been kept in the hard sciences since the 1700s, but they reflect the needs of modern research projects to enhance reproducibility. Data dictionaries list all variables collected, the acceptable values that may be input for that data, and codes for missing data. They enable someone outside of that research lab to reuse the data set.
For evidence synthesis, data dictionaries are a helpful tool at multiple stages of the research process. During the creation of the project protocol, they help to clarify the search string terminology, define conventions used in the course of the project and create clear mutual understanding among the research team of the exact meaning of inclusion and exclusion criteria. During the phase of the project that requires all members of the team to work together to process all citations that they collected, it produces higher inter-rater reliability (a measure of how similarly members of a team classify the same articles.) It is a signal of the quality of the methodology, and therefore the quality of the conclusions of the study. It allows for clear communication on project goals, and makes it easier to write up the discussion section of the methodology because it fundamentally outlines the key points.
In a supplementary file for Cumpston et al. 2023. "Synthesis questions are incompletely reported: a survey of systematic reviews", an exemplary data dictionary was made available. Note that not all of the variables being considered have numeric specifications, but when possible, they are articulated from the beginning of the project. It also includes all valid data input syntax for every question.