What is URL filtering? | Web filtering

What is URL filtering?

URL filtering restricts what web content users can access. It does this by blocking certain URLs from loading. Companies implement URL filtering to help prevent employees from using company resources — devices, network bandwidth, etc. — in a way that negatively impacts the company. URL filtering also helps mitigate malware and phishing attacks by blocking malicious webpages. Secure web gateways often include a URL filtering feature.

What is a URL?

A URL, or Uniform Resource Locator, is the string of text that appears in the address bar of a browser. URLs indicate precisely where a user is on the Internet, somewhat like a residential address or GPS location.

URLs are more specific than domain names. A URL can refer to exact webpages or files hosted at a domain, not just the domain itself. For instance, the main Cloudflare website domain name is cloudflare.com, but a specific page on that website would have a URL like: https://www.cloudflare.com/learning/access-management/what-is-access-control/

How does URL filtering work?

URL filtering blocks URLs from loading, or only allows certain URLs to load, on a company network. If a user attempts to reach a blocked URL, they are redirected to a "blocked" page.

URL filtering bases its filtering policies on a database that classifies URLs by topic and by "blocked" or "allowed" status. Typically a company will not develop this database internally, relying instead on the vendor providing the filtering service. However, most vendors enable companies to customize which URLs are blocked or allowed.

URL filtering can block individual URLs or categories of URLs. By blocking individual URLs, companies can block specific webpages that are known to be dangerous or inappropriate. Meanwhile blocking URL categories allows companies to more efficiently restrict the type of content accessed over their networks by blocking large groups of URLs at once instead of having to list hundreds of individual URLs.

Typically the URL filtering vendor will create the categories and fill them out with groups of URLs that are all related to the same topic or are considered objectionable for similar reasons. For instance, all known URLs used for phishing attacks could be tracked in one "phishing" category, and a company could block all these webpages by using that category. Categorization can be an automatic process: some URL filtering services can use machine learning to identify websites that fit a particular category.

URL filtering takes place at the application layer of the Internet (see What is the OSI model?). The web protocols most frequently used at this layer are HTTP, FTP, and SMTP. The URL filter examines requests that use these protocols, and if they are directed at a blocked URL, it filters out the request and directs the device that the request originated from to a block page.

Is URL filtering the same thing as web filtering?

URL filtering is a type of web filtering. The term "web filtering" refers to a number of techniques for controlling the content users within a network can access over the Internet. DNS filtering is another common technology for restricting web content.

What are the differences between URL filtering and DNS filtering?

DNS filtering and URL filtering perform similar functions. The main difference is that URL filtering blocks URLs, while DNS filtering blocks DNS queries. Another way to put it is that URL filtering blocks webpages, while DNS filtering blocks domains.

DNS filtering makes it possible to block a website and all its webpages, no matter their URLs, by blocking the domain name. However, URL filtering provides more granular and detailed filtering by allowing companies to block individual webpages instead of the whole website at once.

Because URL filtering is more granular than DNS filtering, it may also require more maintenance and customization. Additionally, it needs to be implemented separately for each application protocol. By contrast, DNS filtering is protocol-agnostic: once turned on, it applies to all types of web traffic.

How do these differences look in practice? Suppose internal employees at a company regularly visited a certain news website that broke news about their company's industry. However, the news website also published articles about professional sports, and employees were wasting time and company resources by reading this sports section. DNS filtering would block the news website altogether, so that employees could no longer access the site at all, even the news pieces they needed for work purposes; URL filtering would be able to block the sports pages only.

Another use case for URL filtering: Suppose an otherwise harmless website had one webpage among hundreds that had been compromised by an attacker and delivered a malware payload. DNS filtering would block the domain altogether; URL filtering makes it possible to block just that page. (Of course, often it is safest to block the domain altogether, as DNS filtering does.)

How does URL filtering help block malware and phishing attacks?

Several types of cyber attacks require users to load one or more webpages in order to be successful. Some cyber attackers aim to trick users into loading a malicious webpage that initiates a malware download. If these malicious webpages are identified as dangerous, URL filtering can block them, preventing this type of attack.

Other cyber attackers attempt to steal user accounts via phishing attacks: tricking users into giving away their login credentials or active session. Many phishing attacks ask users to load fake websites that appear legitimate, where the users then are prompted to enter their credentials, thus giving them to the attacker. Known phishing websites can be filtered out using URL filtering, thwarting these types of attacks.

Does Cloudflare offer web filtering?

Cloudflare Gateway offers fast and highly effective DNS filtering, along with other technologies to keep internal employees secure. Cloudflare Gateway is part of the Cloudflare for Teams product suite, which provides internal company security.

Learn more about DNS filtering.