Report Aggregation Mechanism

This section describes the report data aggregation mechanism.

Reports are aggregated based on log data over a period of time. Due to the restrictions of disk read and write performance, reports are not aggregated in real time. Report aggregation is performed as follows:

Original logs generated in the current 5 minutes are aggregated at an interval of 5 minutes to form a 5-minute data table.
The generated 5-minute data tables are aggregated at an interval of 20 minutes to form a 20-minute data table.
The generated 20-minute data tables are aggregated at an interval of 1 hour to form an hourly data table.
The generated hourly data tables are aggregated at an interval of 24 hours to form a daily data table.

Report data comes from various data tables generated during log aggregation. Therefore, when users select a time range on the web UI to view reports, the data in these data tables is queried. The data tables queried in various time ranges are different. Details are as follows:

Time Range	Data Source
Last hour	5-minute data table
Last 24 hours	20-minute data table
Last 3 days	Hourly data table
Last week	Hourly data table
Last month	Daily data table

To save the report data for a longer period of time, data tables of various types are aged and deleted gradually according to the time sequence. The rules are as follows:

Data Table	Deletion Time
5-minute data table	Deleted 8 hours after generation. That is, data in a 5-minute data table will be deleted 8 hours after the table is generated.
20-minute data table	Deleted 192 hours after generation.
Hourly data table	Deleted 31 days after generation.

Report data is of a large volume and constantly changing. To save the report data for a longer period of time, the system clears the data according to the preceding rules. Therefore, users may successfully obtain accurate data at a specific time point but fails to do so some time later. For example, users may want to query the traffic report from 12:00 to 13:00 of a day one month ago. However, users can obtain only the daily data table but not the more detailed hourly data table.
If the device is powered off at 00:00 every day, the system cannot generate daily data tables, affecting the query result.