欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  数据库

阅读xtrabackup代码的一点笔记

程序员文章站 2024-02-14 18:45:28
...
xtrabackup binary最重要的两个过程是backup和prepare,对应的函数分别是xtrabackup_backup_func()和xtrabackup_prepare_func(),这里做一些阅读代码时的笔记。

xtrabackup backup的线程模型:

1. 一个log拷贝线程;

2. n个ibd文件拷贝线程;

3. 一个io监控线程;

4. 通过suspend_start/suspend_end文件来标注是否启动终止线程;

typedef struct {

datafiles_iter_t *it;

uint num;

uint *count;

os_ib_mutex_t count_mutex;

os_thread_id_t id;

}data_thread_ctxt_t;

数据线程上下文切换工作目录;

116 /** Set if InnoDB must operate in read-only mode. We don't do any

117 recovery and open all tables in RO mode instead of RW mode. We don't

118 sync the max trx id to disk either. */

xb_set_innodb_read_only() 将innodb设成只读模式

srv_backup_mode=TRUE; 将innodb设成backup模式;

设置innodb的一系列参数

innodb_init_param()

xb_normalize_init_values(void)

修改srv_unix_file_flush_method

根据bp大小,调整srv_max_n_threads参数

1017 /*********************************************************************//**

1018 Initializes the synchronization primitives, memory system, and the thread

1019 local storage. */

srv_general_init()

ut_crc32_init()

xb_filters_init()

2567 /************************************************************************

2568 Initializes the I/O and tablespace cache subsystems. */

xb_fil_io_init(void)

838 /******************************************************//**

839 Initializes the log. */

log_init(void)

585 /*********************************************************************//**

586 Creates the lock system at database start. */

lock_sys_create()

open_or_create_log_file

创建xtrabackup_extra_lsndir/extrabackup_traget_dir

表空间memory cache

fil_system_t* f_system = fil_system;

recv_find_max_checkpoint(&max_cp_group, &max_cp_field)

log_group_read_checkpoint_info(max_cp_group, max_cp_field)

checkpoint_lsn_start/checkpoint_no_start

确认一致的checkpoint状态;

创建XB_LOG_FILENAME文件,写入文件头信息;

创建io_watching_thread;

从checkpoint位置开始copy log文件;

xtrabackup_copy_logfile(checkpoint_lsn_start, FALSE)

log_copying/log_copying_stop

创建日志copy线程

os_thread_create(log_copying_thread, NULL, &log_copying_thread_id);

2591 /****************************************************************************

2592 Populates the tablespace memory cache by scanning for and opening data files.

2593 @returns DB_SUCCESS or error code.*/

xb_load_tablespaces()

挂起,等待XB_FN_SUSPENDED_AT_START文件被删除

xtrabackup_suspend

xb_page_bitmap_init()

根据xtrabackup_parallel设置,创建data_copy_thread_func线程

等待所有data_copy_thread_func线程退出

挂起,等待XB_FN_SUSPENDED_AT_END文件被删除

xtrabackup_suspend

读取最新的checkpoint, 记录在metadata的to_lsn字段;

通过设置log_copying=FALSE && set log_copying_stop,停止log_copying_thread;

创建一个文件XB_FN_LOG_COPIED,通知外部脚本,log_copying_thread已经结束;

写metadata;prepare_func

1. 切换到xtrabackup_real_target_dir

2. 读取XTRABACKUP_METADATA_FILENAME,获取原信息;

metadata_type

3. xtrabackup_init_temp_log()

4. innodb_init_param()

2670 /************************************************************************

2671 Initialize the tablespace memory cache and populate it by scanning for and

2672 opening data files.

2673 @returns DB_SUCCESS or error code.*/

xb_data_files_init()

应用增量到全量

xtrabackup_apply_deltas()

重设innodb初始化参数

innodb_init_param()

innodb_init()遍历文件

mtr_start -> mtr_commit

trx_sys_print_mysql_binlog_offset()

将binlog位置信息输出到 xtrabackup_binlog_pos_innodb文件中

xtrabackup_close_temp_log(TRUE)

输出记录metadata_log

backup

$./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --backup --target_dir=/u01/xianlin.lh/backup_dir/

./xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)

xtrabackup: uses posix_fadvise().

xtrabackup: cd to /u01/my3928/data

xtrabackup: using the following InnoDB configuration:

xtrabackup: innodb_data_home_dir = /u01/my3928/data

xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend

xtrabackup: innodb_log_group_home_dir = /u01/my3928/data

xtrabackup: innodb_log_files_in_group = 4

xtrabackup: innodb_log_file_size = 1073741824

2014-05-05 17:29:35 2ac06bc4a2c0 InnoDB: Using Linux native AIO

xtrabackup: using O_DIRECT

>> log scanned up to (1451746590)

[01] Copying /u01/my3928/data/ibdata1 to /u01/xianlin.lh/backup_dir/ibdata1

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

[01] ...done

[01] Copying /u01/my3928/data/ibdata2 to /u01/xianlin.lh/backup_dir/ibdata2

[01] ...done

[01] Copying ./test/t2.ibd to /u01/xianlin.lh/backup_dir/test/t2.ibd

[01] ...done

[01] Copying ./test/t4.ibd to /u01/xianlin.lh/backup_dir/test/t4.ibd

[01] ...done

[01] Copying ./test/t3.ibd to /u01/xianlin.lh/backup_dir/test/t3.ibd

[01] ...done

[01] Copying ./test/sbtest1.ibd to /u01/xianlin.lh/backup_dir/test/sbtest1.ibd

>> log scanned up to (1451746590)

>> log scanned up to (1451746590)

[01] ...done

[01] Copying ./test/t1.ibd to /u01/xianlin.lh/backup_dir/test/t1.ibd

[01] ...done

[01] Copying ./mysql/innodb_index_stats.ibd to /u01/xianlin.lh/backup_dir/mysql/innodb_index_stats.ibd

[01] ...done

[01] Copying ./mysql/slave_worker_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_worker_info.ibd

[01] ...done

[01] Copying ./mysql/innodb_table_stats.ibd to /u01/xianlin.lh/backup_dir/mysql/innodb_table_stats.ibd

[01] ...done

[01] Copying ./mysql/slave_relay_log_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_relay_log_info.ibd

[01] ...done

[01] Copying ./mysql/slave_master_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_master_info.ibd

[01] ...done

>> log scanned up to (1451746590)

xtrabackup: The latest check point (for incremental): '1451746590'

xtrabackup: Stopping log copying thread.

.>> log scanned up to (1451746590)

xtrabackup: Transaction log of lsn (1451746590) to (1451746590) was copied.

创建备份的流程

http://www.percona.com/doc/percona-xtrabackup/2.1/xtrabackup_bin/creating_a_backup.html

1 backup_type = full-backuped

2 from_lsn = 0

3 to_lsn = 1451746590

4 last_lsn = 1451746590

5 compact = 0prepare

$./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --prepare --target-dir=/u01/xianlin.lh/backup_dir/

./xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)

xtrabackup: cd to /u01/xianlin.lh/backup_dir/

xtrabackup: This target seems to be not prepared yet.

xtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(1451746590)

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup: innodb_data_home_dir = ./

xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend

xtrabackup: innodb_log_group_home_dir = ./

xtrabackup: innodb_log_files_in_group = 1

xtrabackup: innodb_log_file_size = 2097152

2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO

xtrabackup: using the following InnoDB configuration for recovery:

xtrabackup: innodb_data_home_dir = ./

xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend

xtrabackup: innodb_log_group_home_dir = ./

xtrabackup: innodb_log_files_in_group = 1

xtrabackup: innodb_log_file_size = 2097152

2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO

xtrabackup: Starting InnoDB instance for recovery.

xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)

InnoDB: The InnoDB memory heap is disabled

InnoDB: Mutexes and rw_locks use GCC atomic builtins

InnoDB: Compressed tables use zlib 1.2.3

InnoDB: Using Linux native AIO

InnoDB: Using CPU crc32 instructions

InnoDB: Initializing buffer pool, size = 100.0M

InnoDB: Completed initialization of buffer pool

InnoDB: Highest supported file format is Barracuda.

InnoDB: The log sequence numbers 552524932 and 552524932 in ibdata files do not match the log sequence number 1451746590 in the ib_logfiles!

InnoDB: Database was not shutdown normally!

InnoDB: Starting crash recovery.

InnoDB: Reading tablespace information from the .ibd files...

InnoDB: Restoring possible half-written data pages

InnoDB: from the doublewrite buffer...

InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023

InnoDB: 128 rollback segment(s) are active.

InnoDB: Waiting for purge to start

2014-05-05 18:38:33 2b7f49056700 InnoDB: Warning: table 'test/sbtest1'

InnoDB: in InnoDB data dictionary has unknown flags 50.

InnoDB: 5.6.15 started; log sequence number 1451746590

[notice (again)]

If you use binary log and don't use any hack of group commit,

the binary log position seems to be:

InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023

xtrabackup: starting shutdown with innodb_fast_shutdown = 1

InnoDB: FTS optimize thread exiting.

InnoDB: Starting shutdown...

InnoDB: Shutdown completed; log sequence number 1451747281

prepare的流程:

http://www.percona.com/doc/percona-xtrabackup/2.1/xtrabackup_bin/preparing_the_backup.html

1 backup_type = full-prepared

2 from_lsn = 0

3 to_lsn = 1451746590

4 last_lsn = 1451746590

5 compact = 0