Skip to main content

University Library, University of Illinois at Urbana-Champaign

Ways of the Web: Filter Bubbles and the Deep Web: Dive Deeper

This guide is a companion to the Ways of the Web Savvy Researcher Workshop.

Introduction

The web is like an iceberg. You can see the tip, but most of it is submerged. Just as in most cases we are unable to see a whole iceberg from the surface of a body of water, the Deep Web is invisible to us when we search with standard search engines. 

Iceburg pieces floating in water

By Individuo [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

One other strategy is to get around your filter bubble by searching in the Deep Web. There are many websites that search engines do not search/index. These are considered to be in the Deep Web. It is estimated that 96-99 percent of information online resides in the Deep Web.

Additional Resources

What's in the Deep Web?

The "Deep Web" refers to everything that a standard search engine does not search. "Web crawlers" follow links on the internet, and index pages that they find. These indexed pages will then show up in our search results. If there are pages that web crawlers can't access, then they are considered to be in the Deep Web.

Deep web content includes dynamic web pages, which are web pages generated based on user input. A database query result list would be an example of a dynamically generated web page. Search engines may be able to get to the homepage of a database, but typically they do not search within a database.

Some also consider pages that have a low ranking (showing up several pages into your results list) to be in the Deep Web.

Content in the Deep Web includes:

  • Subscription databases (on library websites)
  • Intranets (Company or organization's password-protected websites)
  • Bank accounts
  • Social media - depends on the privacy settings
  • Webpages that are "buried" in very large websites, meaning you have to click on a lot of links to get there, such that web crawlers may not reach them - Government websites are an example.
  • Other search engines and online directories (Google may be able to find the homepage, but does not search within it)
  • Isolated pages - webpages that no other pages link to
  • Web pages that have been deliberately hidden from search engines, using special code

For more information on the Deep Web, view Mashable's video:

Deep Web Links

Here is a list of links to Deep Web sources of high quality information. Search outside your filter bubble here! The url's are accompanied by descriptions of what you will find at each website. All of the resources are free, although some, like Google Scholar, include partial records of articles that require a library subscription. (For more information, click on the circled "I" next to Google Scholar.)