Did you know that what most people think of as the Web is actually just one tiny layer of a much bigger world? When you surf the Internet, you probably regularly visit sites like Facebook, Google, YouTube, Amazon, Twitter, Wikipedia, Yahoo!, and more. But beneath it all, there is another fathomless world––the Deep Web. Literally trillions and trillions of pages live on the Deep Web. A good way to think about it is that the Web we all know and use is merely 1% of the entire World Wide Web.
So what is this Deep Web? How did it start? What’s it for? And, how does it relate to identity theft? I was intrigued so I thought I’d share what I learned here.
Let’s begin with the fundamentals of how search engines work. When you do a search with Google, Yahoo!, or Bing, you are actually just scraping the surface of the World Wide Web. Search engines are constantly indexing pages by following the links between sites––literally crawling the Web’s threads like a spider. But search engines can’t capture pages behind private networks or standalone pages that connect to nothing at all. These are what make up a significant part of the Deep Web.
The Deep Web: So What’s Down There?
The web pages that make up the majority of the Deep Web contain valuable information stored in databases. Among the world’s largest are the U.S. National Oceanic and Atmospheric Administration, NASA, the Patent and Trademark Office, and the Securities and Exchange Commission’s EDGAR search system––all of which are public. The next batch of databases has pages kept private by companies that charge a fee to see them, such as the legal documents on LexisNexis and Westlaw or the academic journals on Elsevier.
Another 13% of pages are only found on company or university intranets––restricted access communications networks created using World Wide Web software. Typically these networks allow users access to message boards, personnel files, or industrial control panels that can flip a light switch or shut down a power plant.
“Tor”: The Deepest and Most Illicit Part of the Internet
The truly deepest part of the Internet is “Tor,” a collection of secret websites (ending in .onion) that require special software to access them. People use Tor so that their Web activity can’t be traced. Tor runs on a relay system that bounces signals among different Tor-enabled computers around the world.
The U.S. Naval Research Laboratory originally made Tor to allow anonymous communications. It first debuted as The Onion Routing project in 2002. It can be used for extremely sensitive communications, including political dissent.
But in the last decade, it’s also become a hub for black markets that sell or distribute drugs (Silk Road), stolen credit cards, illegal pornography, pirated media, illegal guns, hit men, and more. Silk Road uses the untraceable currency, Bitcoin. Last fall the FBI cracked down on Silk Road, blocking it and arresting its leader.
It was widely reported last year that the NSA monitors Tor. However, the technology behind Tor still exists so it’s not hard to imagine that someone is out there right now developing a newer, even more secret version. This is a stark reminder that criminals never stop looking for new ways to steal people’s identities and commit fraud in someone else’s name. And they can do it easily and anonymously on the Deep Web.
Big Data and The Deep Web
While the Deep Web stays mostly hidden from public view, it is growing in economic importance. Whatever search engine can accurately and quickly comb the full Web could be useful for Big Data collection––especially for researchers of climate, finance, or government records.
Stanford University, a technology leader, has built a prototype engine called the Hidden Web Exposer, HiWE. Others that are publicly accessible are Infoplease, PubMed, and the University of California’s Infomine.
Protect What Matters Most
The Deep Web fascinates me, but it’s yet another reminder of how vulnerable we all are to identity theft. By enrolling in IdentityForce’s UltraSecure your personal information will be monitored 24/7 and you’ll be notified immediately of any suspicious activity so you can act before any damage is done. If anything does happen, IdentityForce will be with you every step of the way helping you restore your identity.
Image courtesy of Flickr user *n3wjack’s world in pixels.