There are several ways to measure the impact of textbooks. Authors and publishers of traditional textbooks track certain metrics like books sales, royalties, course adoptions, and e-book downloads. Creators of open educational resources often have different goals in mind than commercial publishers and may want to consider alternative types of metrics. For example, the author of an open textbook might be more concerned with the amount of savings to students rather than book sales. Deciding which metrics are important to track should not be a last minute decision. Instead, it is important to consider metrics closer to the beginning of a project for a few reasons. One reason to consider metrics early is because authors and publishers might not be interested in the same metrics. For example, an author is concerned with the success and adoption of their individual textbook, while a publisher is more concerned with the overall sustainability and success of their program. Also, authors are better positioned to collect certain metrics while publishers and libraries are better positioned to collect others.
The Metrics Toolkit was developed for researchers and evaluators to provide guidance for demonstrating and evaluating claims of research impact. While it was primarily designed to address metrics for original research, it includes useful information about different types of altmetrics that could be useful to OER creators and publishers. The Toolkit also identifies which services can provide particular metrics such as Google Analytics, Altmetrics, and others.
Open Syllabus Project is an online database of university course syllabi. The website scrapes university websites for syllabi and lets instructors submit syllabi. While it is a new tool, it may prove to be a convenient way to track and measure course adoption.
36 Indicators of OER Impact by Janet Swatscheno, licensed CC BY 4.0. This graphic is adapted from 56 Indicators of Impact by Holbrook, J Britt, Kelli R. Barr, and Keith Wayne Brown. 2013. Figshare. https://doi.org/10.6084/m9.figshare.707081.v3. Licensed under CC BY 4.0
In the article “We Need Negative Metrics Too”, J. Britt Holbrook, Kelli R. Barr and Keith Wayne Brown identify a number of indicators of impact. Many of the indicators are standard measures of success such as book sales, but the list also takes into account alternatives like policy change and student success. Library publishers should consider the types of quantitative and qualitative data to collect about the resources they publish.
Metrics to consider for open educational resources:
The BCcampus Open Textbook statistics page is a good example of an OER publishing program collecting and sharing metrics about the books they’ve published. It includes information about student savings, known adoptions, and the number of reviews for each book.
