Alternate Copy and Backup Method
for Large Scrutinizer Databases
Some
users with large amounts of data being stored by Scrutinizer may find that
using MySQL Administrator for backups is extremely slow and CPU intensive. MySQL Administrator takes a very
conservative (safe) approach to doing the backup. This is reasonable for a general purpose
backup tool with no application knowledge of what is being backed up. For Scrutinizer the additional overhead can
degrade performance and is not necessary for a successful backup.
What
follows is a description of the recommended method to implement backups for
large networks..
Backing up the data
The
history and configuration data for Scrutinizer is stored in \Scrutinizer\mysql\data. Users
are not recommended to backup \Scrutinizer\mysql\data
directly. The recommended solution is to
copy the data directory to a backup location and optionally run the backup on
that directory.
Copying the data:
Use
the windows
command xcopy to hotcopy
the MySQL data files to the backup directory.
C:\>xcopy /D /E /I /Y /Z
"\SCRUTINIZER\mysql\data" “\SCRUTINIZER_backup”
The above command will
only copy files that have changed. The
initial copy will need to duplicate the entire DB. Subsequent runs will only
copy files that have been modified since the previous run.
This technique relies on
the fact that after a known period of time the historical tables in Scrutinizer
are static (read only). If the tables
aren't being updated, it is safe to just copy the data tables. Even when tables are being updated MySQL
MyISAM tables are very robust and seldom corrupt. The following table shows how long each table
is written to before it becomes read only:
Table Type |
duration |
1m |
1 hour |
5m |
5 hours |
30m |
30 hours |
2h |
5 days |
12h |
30 days |
1d |
60 days |
1w |
420 days (60 weeks) |
Up through the 30m tables, they become read only in 30 hours or less. Above 30m tables, a little knowledge about
when tables are written to can help to ensure that as much current data is
saved as possible.
·
2h are written
to starting at the top of even hours
·
12h are
written to starting after 12 and midnight
·
1d are written
to starting after midnight
·
1w are written
after midnight on Saturday
By picking a time when
little is going on (e.g. 1pm Sunday or 1am Monday) it is possible
to safely backup of your data without needing MySQL Administrator.
Caveats
Incrementally copying
files with xcopy will not remove tables from the
backup directory that have been dropped by Scrutinizer. To get a list of files to
delete from the backup directory simply reverse the xcopy
command and add the /L to only print results.
xcopy /D /E /I /Y /Z /L "\SCRUTINIZER_backup" "\SCRUTINIZER\mysql\data"