MySQL Backup and Restore
by Mark Nielsen
Copyright 2021
This should cover mysqldump, xtrabackup, binary,
snapshots via LVM or KVM, mariabackup but not anything to do with the Cloud. All the backup
sections will include there sample restore commands.
In case you are wondering. Oracle bought MySQL for years and did very little with it for years.
They might have done that because it competed with Oracle database, but now Oracle
has seemed to be doing a lot with it. This gave Percona and MariaDB time to rise
and make improvements and become popular as they are offshoots of MySQL.
Nobody uses Oracle MySQL except Oracle diehards. Percona was the default for many years
and will likely be a strong player for many years to come. MariaDB seems to be getting
very popular lately and one of the cofounders of MySQL leads it. SkySQL from MariaDB
is a database service which shows GREAT promise.
People often forget about restores. Here are important points:
- No one ever asks about restores, so who are cares about backups? The requirements of
the restore determine how you perform which backup.
- How long will the restore take?
- Did the backup include all replication info for a proper restore?
- Is the data 100% accurate for the restore? Did you compare Master to Slave data?
- Do you have proper procedure for restore? This should include proper procedure
for backups.
- Is diskspace for backups enough and being monitored? If no, your restores will
suffer.
- Do you keep backups back in time according to requirements? If no then you can't
restore data for the requirements.
In general, here is a breakdown of the different backups.
- ALL: If made right all can be compressed and encrypted. All can be done non-locking
or sub second downtime. I recommend to always backup on a slave. I always recommend
to compare data from Master to the Slave, like using Percona pt-table-checksum.
- mysqldump, xtrabackup, and mariabackup and maybe snapshots can do incremental
backups.
- MySQLDump: The dump is text. If there is an error in the dump you can
edit it directly. Easy to compress. But takes a long time for the backup to run
and restore. Can be impractical for large databases. You cannot edit data in other
backups like you can with text backups. One byte error on binary backups ruin
the backups. Always good to do text backups for emergencies.
- xtrabackup and mariabackup: One is for Percona and the other for MariaDB.
It creates a binary backup for quick restore. Can include binlog of master or slave
if you backup from a master to slave. Can error out easier than other backups methods
but usually never causes a problem and is the most widely used default.
I highly recommend using a Slave. This is used for larger databases.
Can still be time consuming for really large systems.
- Binary: Its a hard backup. You do not need any software. But you should shut down
the server. Its better to use snapshots. Snapshots are the same as Binary backups
for the most part. This is rarely ever used.
- Snapshot: Very similar to binary backups. However the snapshot is an ENTIRE filesystem.
If you take a snapshot and then do a binary backup of mysql only it is effectively
the same as a binary backup. The big advantage here is there is little downtime.
The amount of downtime is basically the time it is to setup the snapshot which is
normally quick. Thus, always use on a Slave dedicated for backups or one you take
out of your load balancer. It will use a lot of IO if you are making a copy of the
snapshot to a backup device. This is used for VERY big databases. If you do your
snapshotting in a VM environment, you might be able to restore a server very fast.
- Links
- Best Practices
- Binlog backups and Time Travel
- mysqldump
- xtrabackup and mariabackup
- binary
- snapshots via LVM or KVM
Links
- https://www.percona.com/software/mysql-database/percona-xtrabackup
- https://mariadb.com/kb/en/backup-and-restore-overview/
- https://mariadb.com/kb/en/full-backup-and-restore-with-mariabackup/
- https://www.percona.com/blog/2006/08/21/using-lvm-for-mysql-backup-and-replication-setup/
- https://www.tecmint.com/take-snapshot-of-logical-volume-and-restore-in-lvm/
- https://linuxize.com/post/how-to-back-up-and-restore-mysql-databases-with-mysqldump/
- https://www.virtkick.com/docs/how-to-perform-a-live-backup-on-your-kvm-virtual-machines.html
- https://www.linux-kvm.org/images/b/b6/2011-forum-LiveBackup.pdf
- https://www.tecmint.com/install-logrotate-to-manage-log-rotation-in-linux/
Best Practices
Backups
- Backups should be done ONLY in a prod environment for prod systems.
- A restore policy, requirements, and procedures should be documented and approved.
- You should be backing up binlogs so you can do time travel.
- Backups should be done in a certain amount of time and track how long the backups
take and if they use a lot of resources to do them.
- Backup all data, so if using xtrabackups or mariadbbackup, Mark you copy all non-innodb
tables.
- You should be backing up other things like configurations, error logs, etc.
- Misc stuff should put into rotation like logrotate. Set logrotate for slow, general,
and other logs. Except binlogs which should be set with "expire_log_days" in mysql
generally set to 30 days, but some companies set it shorter if they have high data
throughput. Backup errors logs with backups. If they are rotated to 5 days they will
eventually be deleted.
Restore
- Restores should be tested once every 6 months or more.
- Its preferred a restore is automated and test frequently then.
- If a restore happens outside of PROD, data should be scrubbed
for any sensitive information according to requirements. It can be replaced
with fake data which several products can make for you.
- A restore policy, requirements, and procedures should be documented and approved.
- Restoration should be able to occur within specified time.
- If backups are getting longer, it will take restorations, longer, maybe. Be aware of this.
- If restoring binlogs, make sure you do it correctly and test it. Otherwise if done
wrong you can assume your data is corrupt.
Binlog backups and Time Travel
In MySQL, you should do the following with binlogs:
- Rotate binlogs so that old binlogs get removed. Easiest way is to
"set expire_logs_days=30" in Mysql.
- Backup binlogs of any master except the current one. Trick, flush the logs
and then backup all logs but the current one. Also, rsync or only backup new logs
created.
- Do the same to the backup MySQL server for its binlogs or any server where
you need to save binlogs according to its replication position.
- If you use GTID, you can probably backup binlogs from any server. This needs to
be verified.
- You should have a policy, requirements, and procedure for backing up and purging
binlogs.
- Binlogs with a full backup can be used to time travel for restoration purposes.
I am not going to get into this except to say, stop mysql replcation, do show slave
status and show master status, take backup, then start mysql. You can start mysql replication
after the mysqldump starts. The mysqldump will be single-transaction.
mysql -u root -e "stop slave io_thread"
sleep 5
# NOTE: You should really so a check that the io_thread and sql_thread are the same.
mysql -u root -e "stop slave sql_thread"
d=`date +%y%m%d_%H%M%S`
h=/backups/mysql/mysql3/$d
f=/backups/mysql/mysql3/$d/mysql1_$d
mkdir -p $h
mysql -u root -e "show slave status\G" > $h/show_slave_status.info
mysql -u root -e "show master status\G" > $h/show_master_status.info
options=" --all-databases --single-transaction --master-data"
mysqldump -u root $optionsA -S /tmp/mysql3.sock | gzip > $f.gz
Both XtraBackup and MariaBackup both create binary copies of the database, but only
the innodb files. Copy over the "mysql" directory also. There are tools to do complete
backups but the innodb files tend to be the most important and the information in the
"mysql" database and be restored -- accounts, etc.
Both are very similar.
MariaBackup
# This is mariabdb, so we have to use mariadb-backup
apt-get install mariadb-backup
# We can use xtrrabackup or innobackupex
# If xtrabackup, you should also backup the mysql database.
mkdir -p /backups/mysql/mysql1
d=`date +%y%m%d_%H%M%S`
mariabackup --defaults-file=/data/mysql/mysql1/my.cnf \
--datadir=/data/mysql/mysql1/data \
--user=root --compress --backup \
--slave-info \
--target-dir=/backups/mysql/mysql1/mysql1_$d \
--socket=/tmp/mysql1.sock \
--compress
exclude="--exclude=gtid*,innodb*,transaction*"
S=/data/mysql/mysql1/data/mysql
D=/backups/mysql/mysql1/mysql1_$d/MYSQL_copy
mkdir -p /backups/mysql/mysql1/mysql1_$d/MYSQL_copy
rsync -av $exlude $S $D
XtraBackup for Percona MySQL
# We can use xtrrabackup or innobackupex
# If xtrabackup, you should also backup the mysql database.
mkdir -p /backups/mysql/mysql1
d=`date +%y%m%d_%H%M%S`
xtrabackup --defaults-file=/data/mysql/mysql1/my.cnf \
--datadir=/data/mysql/mysql1/data \
--user=root --compress --backup \
--slave-info \
--target-dir=/backups/mysql/mysql1/mysql1_$d \
--socket=/tmp/mysql1.sock \
--compress
exclude="--exclude=gtid*,innodb*,transaction*"
S=/data/mysql/mysql1/data/mysql
D=/backups/mysql/mysql1/mysql1_$d/MYSQL_copy
mkdir -p /backups/mysql/mysql1/mysql1_$d/MYSQL_copy
rsync -av $exlude $S $D
Simple binary backup.
d=`date +%y%m%d_%H%M%S`
h=/backups/mysql/mysql3/$d
mkdir -p $h
# kill mysql
# This won't work on my server because I don't use service.
service mysql stop
# I do this
# pid = `cat /tmp/mysql3/sock`
# kill $pid
# sleep 10
rsync -av /data/mysql/mysql3/* /backups/mysql/mysql3/$d/
service start mysql
# For my server.
# mysqld --defaults-file=/data/mysql/mysql3/my.cnf &
# sleep 5