URL Filtering Mechanism

This section describes how a FW implements URL filtering.

If a user accesses a network resource using HTTP or HTTPS through the FW, the FW applies the filter, as shown in Figure 1.

Figure 1 URL filtering flow

1. A user initiates a URL access request. If the data flow matches a security policy, and the action of the policy is permit, the system implements URL filtering.

2. The FW checks the HTTP packet.

   • If the HTTP packet is abnormal, the FW blocks the request.
   • If the HTTP packet is normal, the FW performs the next detection.

3. The FW matches the URL with the whitelist.

   • If the URL matches the whitelist, the FW permits the URL access request.
   • If the URL does not match the whitelist, the FW proceeds to the next step.

4. The FW matches the URL with the blacklist.

   • If URL information matches the blacklist, the FW blocks the URL access request.
   • If the URL does not match the blacklist, the FW proceeds to the next step.

5. The FW matches the referer field in a received HTTP request with whitelist rules.

   • If the referer field in the HTTP request matches the configured referer-host rule, the request is allowed.
   • If the referer field in the HTTP request does not match the configured referer-host rule, the user can determine whether to match the referer field with all whitelist rules.
     • When the function of matching the referer field with whitelist rules is enabled, the referer field will be matched with all whitelist rules.
       • If a match is found, the HTTP request is allowed.
       • If the no match is found, proceed to the next step.
     • When the function of matching the referer field with whitelist rules is disabled, the referer field will not be matched with all whitelist rules, and the next detection step will be taken.

   By default, the function of matching the referer field in a URL request with whitelist rules is enabled. You can disable this function.

   

   After URL filtering that supports only the whitelist mode is enabled, the data flow that has a matching whitelist rule is permitted, and the data flow that does not have a matching whitelist rule is blocked.

6. The FW matches the URL with user-defined categories.

   • If the URL matches a user-defined category, the FW processes the request based on the action for this user-defined category.

     

     URLs added to predefined categories are still considered user-defined URLs.

   • If the URL does not match user-defined categories, the FW proceeds to the next step.

7. The FW matches URLs with the external dynamic malicious URL list.

   • If a URL matches the external dynamic malicious URL list, the FW blocks the request.

   • If a URL does not match the external dynamic malicious URL list, the FW proceeds to the next step.

8. The FW matches the URL information in the request with malicious URLs or low-reputation URLs.

   • If the URL matches a malicious URL or low-reputation URL, the FW blocks the URL request.
   • If the URL does not match any malicious URL or low-reputation URL, the FW proceeds to the next step.

9. The FW matches the URL with predefined categories in the cache.

   • If a predefined category is matched, the FW processes the request based on the action for this predefined category.

   • If no category is matched, the FW starts remote query.

     • If the remote query server is available, the FW performs remote query.

     • If the remote query server is unavailable, the FW takes the default action on the request.

10. Enable remote query.

    1. If the remote server does not respond within a specific period, the FW will take the action configured for query timeout.

    2. If the URL belongs to a defined category on the remote query server, the FW takes the action defined for the predefined category.

       

       If the URL does not belong to any category, the FW will take the control action for Other categories.

If a session contains multiple URLs, the FW performs URL filtering on each URL and blocks the entire session as long as any one of the URLs is blocked.