URL filtering

URL filtering ensures that the users of your network cannot access web objects that are considered a risk for web security or are not allowed because they contain inappropriate subject matter or for other reasons.

The filtering process uses blocking lists, category information, and reputation scores for the URLs of web objects and blocks or allows access accordingly.

A default process for URL filtering is implemented on Web Gateway after the initial setup. Important configuration items used in this process include:

  • URL Filtering rule set — Default rule set for URL filtering

  • Dynamic Content Classification rule set — Default rule set supporting the URL filtering process

    The rules in this rule set categorize web objects based on the analysis of the Dynamic Content Classifier component when other URL filtering methods yield no results.

  • URL Filter settings — Default settings for the URL Filter module, which handles the retrieval of category information and reputation scores from intelligence systems.

    These settings also include options for configuring the Dynamic Content Classifier.

The default process requires that you maintain the block lists used by the rules in the URL Filtering and Dynamic Content Classification rule sets. You can further modify this process to meet the requirements of your organization.

You can also extend the process in several ways or set up a process of your own.

URL filtering process

The URL filtering process includes several elements, which contribute to it in different ways.

  • Filtering rules — Control the process. There are usually the following types of rules.

    • Blocking rules — Block access to web objects with particular URLs.

      The rules apply if a URL has been entered in a list that is used by these rules or falls into a category that is on a list.

      When categories are used in a rule, the URL filter module is called to handle the retrieval of category information from the Global Threat Intelligence (GTI) service.

    • Whitelisting rules — Exclude web objects from further URL filtering to ensure they can be accessed by the users in your network.

      Whitelisting rules are placed before the blocking rules in an URL filtering rule set. If a whitelisting rule applies, processing of the following URL filtering rules is stopped to ensure that the blocking rule is not executed.

  • Whitelists and blocking lists — These lists are used by whitelisting and blocking rule that exist in the URL filtering process.

    Because a URL filtering rule set is only used for URL filtering, multiple whitelists for several types of objects are not needed in the filtering process, in contrast to, for example, anti-malware filtering.

  • URL Filter module —This module, which is also known as an engine, retrieves information on URL categories and reputation scores from the Global Threat Intelligence™ service that is provided by McAfee. Based on this information, blocking rules block access to URLs.

    Various technologies, such as link crawlers, security forensics, honeypot networks, sophisticated auto-rating tools, and customer logs are used to gather this information. An international, multi-lingual team of McAfee web analysts evaluates the information and enters URLs under particular categories into a database.

    To gather information on the reputation of a URL, its behavior on a worldwide real-time basis is analyzed, for example, where a URL shows up in the web, its domain behavior, and other details.

    You can configure settings for this module, for example, to perform a DNS lookup for URLs and include the corresponding IP address in the search for category information.

Administering the URL filtering process

When administering the URL filtering process, you can use several configuration items that are available by default.

  • URL Filtering rule set — Default rule set for URL filtering

    This rule set includes two nested rule sets, which allow you to run the default URL filtering process on Web Gateway in two different ways.

    • Special URL Filtering Group rule set — Nested rule set for performing URL filtering with regard to particular users, user groups, and IP addresses.

      The rule set includes a blocking rule and whitelisting rules. Further rules ensure a high level of filtering quality.

      For example, one rule requires that the complete body of a web object is scanned for infections, even if only a request for accessing the object in parts was submitted.

    • Default rule set — Nested rule set for performing URL filtering in general, regardless of any particular users, user groups, or IP addresses

      The rule set includes a blocking rule and whitelisting rules. Further rules ensure a high level of filtering quality.

      For example, one rule requires that the complete body of a web object is scanned for infections, even if only a request for accessing the object in parts was submitted.

  • Whitelists and blocking lists — Used to allow and block access to web objects with particular URLs

    • URL Whitelist — Lists URLs. Use this list to exclude requests for access to web objects with particular URLs from further URL filtering.

      This way you ensure that users are not prevented from accessing these objects by URL filtering

      The list is empty by default and you need to fill the entries.

    • URL Blocklist — List user agents. Use this list to exclude requests with particular user agent information in its headers from further anti-malware filtering.

      The list is empty by default and you need to fill the entries.

    • Category Blocklist — List user agents. Use this list to exclude requests with particular user agent information in its headers from further anti-malware filtering.

      The list is empty by default and you need to fill the entries.

  • Blocklists — Used to exclude web objects from further anti-malware filtering

    • URL Host Whitelist — Lists the URLs of hosts. Use this list to exclude requests with particular URLs from further anti-malware filtering.

      The list is empty by default and you need to fill the entries.

    • User Agent Whitelist — List user agents. Use this list to exclude requests with particular user agent information in its headers from further anti-malware filtering.

      The list is empty by default and you need to fill the entries.

  • URL Filter settings — Default settings for the URL Filter module

    An option is selected in these settings that enables the McAfee Gateway Anti-Malware (GAM) engine for scanning web objects.

    You can change these settings, for example, to involve the Avira engine in the scanning process.

    You can also create your own rule set, lists, and settings for URL filtering.

Extending the URL filtering process

A default process for URL filtering is implemented after the initial setup. You can extend this process in several ways.

  • You can implement your own URL filter database.
  • You can use an IFP proxy for URL filtering.