Blacklist and Whitelist

This section describes the concepts of the blacklist and whitelist and the relationship between the blacklist/whitelist and URL categories.

Concept

A URL blacklist is a list of URLs inaccessible to users, and a URL whitelist is a list of URLs accessible to users. The priority of the whitelist is higher than that of the blacklist. The blacklist and whitelist are used to filter simple and fixed websites. Compared with URL categories, blacklists and whitelists are more fine-grained. When a user requests to access a URL, the device matches the extracted URL with the blacklist and whitelist.

If the URL matches the whitelist, the device permits the URL access request. For example, an enterprise allows employees to access only some websites related to work. This requirement can be met by adding the websites related to work to the whitelist.
If the URL matches the blacklist, the device blocks the URL access request. To improve employees' working efficiency and optimize enterprise network bandwidth usage, online behaviors of employees need to be controlled. Employees are not allowed to access entertainment, game, and video websites. This requirement can be met by adding such websites to the blacklist.

Relationship Between the Blacklist/Whitelist and URL Categories

The blacklist and whitelist are often used together with URL categories. For example, an enterprise blocks employees to access pornographic and illegal websites. In addition, the enterprise wants to:

Permit employees to access intranet websites: www.example1.com and www.example2.com.
Prevent employees from accessing external forum websites: www.example3.com and www.example4.com.

You can use predefined URL categories to block access to adult and illegal websites. To separately control the access to a few websites (for example, www.example1.com, www.example2.com, www.example3.com, and www.example4.com), you can add www.example1.com and www.example2.com to the whitelist and www.example3.com and www.example4.com to the blacklist.

In addition, a user-defined URL category can implement the blacklist/whitelist function. Add a URL to a user-defined URL category and set the action to Allow or Block for the category. Therefore, the blacklist/whitelist can be considered as a special user-defined URL category. The action of a blacklist/whitelist cannot be modified, whereas the action of a user-defined URL category can be set to Allow, Alert, or Block. Note the following points when using a whitelist/blacklist or user-defined URL category:

The whitelist/blacklist has the higher matching priority than the user-defined and predefined URL categories. Therefore, it can achieve the control effect expected by the enterprise and reduce the probability of errors.
Generally, if there are few URL filtering profiles on the FW, either blacklist/whitelist or a user-defined URL category can be used. If there are many URL filtering profiles on the FW and the websites to be controlled separately are fixed, a user-defined URL category is recommended. A user-defined URL category needs to be configured once, and each URL filtering profile automatically references the user-defined URL category. You only need to set different actions for the category. However, the whitelist/blacklist needs to be set for each URL filtering profile, which wastes much manpower.

Whitelist for Embedded Web Pages

Generally, a major web page contains the links to other web pages. If only the main web page is added to the whitelist, the embedded web pages of the main web page cannot be accessed. For example, if only www.example.com is configured as a whitelist rule, web pages that are embedded in www.example.com but do not use www.example.com as the domain name are inaccessible. To allow the access to such embedded web pages, you can add them one by one to the whitelist, but this method is complex. To solve this problem, you can enable the whitelist function for embedded web pages. This function matches the referer field (indicating the source web page) in a user's HTTP request with the whitelist for embedded web pages. If the referer field is matched, the user can access the web page. Therefore, if a whitelist for embedded web pages is configured for a web page, users can access the web pages embedded in this web page, simplifying the configuration.

The whitelist for embedded web pages can be implemented in either of the following ways:

Use the manually configured referer-host to match the referer field in the HTTP request. If they are matched, the URL request is allowed. If the referer field in the HTTP request does not match the configured referer-host, you can determine whether to match the referer field with all configured whitelist rules. After the function of matching the referer field with the whitelist is enabled, if the referer field matches a whitelist rule, the URL request is allowed.
After the function of matching the referer field with the whitelist is enabled, the device matches the referer field in the HTTP request with the configured whitelist. If they are matched, the URL request is allowed.

By default, the function of matching the referer field with the whitelist is enabled. You can disable this function.

Only Whitelist

If you want to use only the whitelist to filter URLs, you can enable this function to simplify the configuration. After this function is enabled:

If an HTTP request matches the whitelist, the HTTP request is allowed.
If an HTTP request does not match the whitelist, the HTTP request is blocked.