What is text mining?

Text mining is a research practice that involves using computers to discover information in large amounts of unstructured text.

Unstructured text is data not formatted according to an encoding structure like HTML or XML.

Examples of unstructured data used for text mining include journal and news articles, blog posts, and email. 

Researchers use text mining tasks such as:

• sentiment analysis
• entity extraction
• document summarization

By using these methods, researchers can make connections and draw conclusions about the content of large text corpora. 

The image on the right is one example of what you can do with text mining. This pie chart represents the total words spoken by characters in the Jacobean play The Revenger's Tragedy.

Credit: Chart by Pgogy, available via Creative Commons license.

Why do text mining?

Text mining helps researchers detect patterns and connections in large volumes of textual material.

According to researcher Marti Hearst, "In text mining, the goal is to discover heretofore unknown information, something that no one yet knows and so could not have yet written down." Text mining enables researchers to draw conclusions from large volumes of material they would not be able to otherwise read, synthesize, and incorporate into their scholarship.

Researchers in fields ranging from biological sciences to the humanities have begun using text mining to detect patterns and discover unknown information. 

If you have questions about text mining, reach out to Scholarly Communications and Publishing (scp@library.illinois.edu). ​

Except where otherwise indicated, original content in this guide is licensed under a Creative Commons Attribution (CC BY) 4.0 license. You are free to share, adopt, or adapt the materials. We encourage broad adoption of these materials for teaching and other professional development purposes, and invite you to customize them for your own needs.