Row-based replication, MySQL 5.6 upgrades and temporal data_MySQL
Whither your rollback plan?
MySQL 5.6 upgrades are in full swing these days and knowing how to safely upgrade from MySQL 5.5 to 5.6 is important. When upgrading a replication environment, it’s important that you can build a migration plan that safely allows for your upgrade with minimal risk — rollback is often a very important component to this.
For many people this means upgrading slaves first and then the master. The strategy of an older master replicating to a newer slave is well known and has been supported in MySQL replication for a very long time. To be specific: you can have a MySQL 5.6 slave of a 5.5 master and this should work fine until you upgrade your master and/or promote one of the slaves to be the master.
However, there are those of us who like to live on the edge and do unsupported things. Suppose that when you cut over to that MySQL 5.6 master your application completely breaks. What would your rollback plan be? In such a case, leaving a 5.5 slave of the new 5.6 master (or perhaps a dual-master setup with 5.5 and 5.6) would be useful to allow you to rollback to but still have the data written on the 5.6 master.
What might break?
With Statement-based replication (SBR), you are generally ok with this type of setup, provided you aren’t doing any MySQL 5.6 syntax-specific things until you don’t have any more 5.5 slaves. However, with Row-based replication (RBR), things are a bit trickier, particularly when column formats change.
Now, one nice new feature of MySQL 5.6 is the improvement of thestorage requirements forDATETIME fields as well as the addition of fractional second support for TIME, DATETIME, and TIMESTAMP. This is great, but unfortunately this is a new column format that 5.5 clearly would not understand. Does this put our 5.6 to 5.5 replication in jeopardy? The answer is, if we’re careful, NO.
Quite simply, MySQL 5.6 supports both old and new types and mysql_upgrade does not make such a conversion on existing tables. Only NEW tables or REBUILT tables in 5.6 will use the new format. Any tables from 5.5 with a simple mysql_upgrade to 5.6 will still be using the old types. For more information on how to find columns in 5.6 that are using the old format, seeIke Walker’s excellent blog post on the topic. (Thanks Ike!)
An imperfect test
To test this out, I created a simple experiment. I have a master and slave using RBR, both on 5.5, and I setuppt-heartbeatto update the master. I realized that pt-heartbeat actually uses a varchar for the timestamp field — I suspect this makes multiple database support easier. However, since pt-heartbeat’s update uses a NOW() to populate that field, I can convert it to a DATETIME:
[root@master ~]# pt-heartbeat --update --database percona --create-tableCREATE TABLE `heartbeat` (`ts` varchar(26) NOT NULL,`server_id` int(10) unsigned NOT NULL,`file` varchar(255) DEFAULT NULL,`position` bigint(20) unsigned DEFAULT NULL,`relay_master_log_file` varchar(255) DEFAULT NULL,`exec_master_log_pos` bigint(20) unsigned DEFAULT NULL,PRIMARY KEY (`server_id`)) ENGINE=InnoDB DEFAULT CHARSET=latin1master mysql> alter table heartbeat drop column ts, add column ts DATETIME;slave mysql> select * from heartbeat/G *************************** 1. row *************************** server_id: 1file: master-bin.000002position: 5107583 relay_master_log_file: NULL exec_master_log_pos: NULLts: 2014-05-02 17:03:59 1 row in set (0.00 sec) CREATE TABLE `heartbeat` ( `server_id` int(10) unsigned NOT NULL, `file` varchar(255) DEFAULT NULL, `position` bigint(20) unsigned DEFAULT NULL, `relay_master_log_file` varchar(255) DEFAULT NULL, `exec_master_log_pos` bigint(20) unsigned DEFAULT NULL, `ts` datetime DEFAULT NULL, PRIMARY KEY (`server_id`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1
|
[root@master~]# pt-heartbeat --update --database percona --create-table CREATETABLE`heartbeat`( `ts`varchar(26)NOTNULL, `server_id`int(10)unsignedNOTNULL, `file`varchar(255)DEFAULTNULL, `position`bigint(20)unsignedDEFAULTNULL, `relay_master_log_file`varchar(255)DEFAULTNULL, `exec_master_log_pos`bigint(20)unsignedDEFAULTNULL, PRIMARYKEY(`server_id`) )ENGINE=InnoDBDEFAULTCHARSET=latin1 mastermysql>altertableheartbeatdropcolumnts,addcolumntsDATETIME; slavemysql>select*fromheartbeat/G ***************************1.row*************************** server_id:1 file:master-bin.000002 position:5107583 relay_master_log_file:NULL exec_master_log_pos:NULL ts:2014-05-0217:03:59 1rowinset(0.00sec) CREATETABLE`heartbeat`( `server_id`int(10)unsignedNOTNULL, `file`varchar(255)DEFAULTNULL, `position`bigint(20)unsignedDEFAULTNULL, `relay_master_log_file`varchar(255)DEFAULTNULL, `exec_master_log_pos`bigint(20)unsignedDEFAULTNULL, `ts`datetimeDEFAULTNULL, PRIMARYKEY(`server_id`) )ENGINE=InnoDBDEFAULTCHARSET=latin1 |
So my heartbeat table now has a 5.5 DATETIME, pt-heartbeat is working properly, and the heartbeat is replicating to the slave. Now I will upgrade my master to MySQL 5.6:
[root@master ~]# rpm -e Percona-Server-devel-55-5.5.36-rel34.2.el6.x86_64 Percona-Server-shared-55-5.5.36-rel34.2.el6.x86_64 Percona-Server-client-55-5.5.36-rel34.2.el6.x86_64 Percona-Server-server-55-5.5.36-rel34.2.el6.x86_64 --nodeps[root@master ~]# yum install Percona-Server-server-56.x86_64==============================================================================================================Package ArchVersion RepositorySize==============================================================================================================Installing:Percona-Server-server-56x86_645.6.16-rel64.2.el6Percona 19 MInstalling for dependencies:Percona-Server-client-56x86_645.6.16-rel64.2.el6Percona6.8 MPercona-Server-shared-56x86_645.6.16-rel64.2.el6Percona712 kTransaction Summary==============================================================================================================Install 3 Package(s)...[root@master ~]# service mysql startStarting MySQL (Percona Server)....... SUCCESS![root@master ~]# mysqlWelcome to the MySQL monitor.Commands end with ; or /g.Your MySQL connection id is 1Server version: 5.6.16-64.2-56-log Percona Server (GPL), Release 64.2, Revision 569[root@master ~]# mysql_upgradeLooking for 'mysql' as: mysqlLooking for 'mysqlcheck' as: mysqlcheckRunning 'mysqlcheck with default connection argumentsRunning 'mysqlcheck with default connection argumentsmysql.columns_priv OKmysql.db OKmysql.eventOKmysql.func OKmysql.general_logOKmysql.help_categoryOKmysql.help_keyword OKmysql.help_relationOKmysql.help_topic OKmysql.host OKmysql.ndb_binlog_index OKmysql.plugin OKmysql.proc OKmysql.procs_priv OKmysql.proxies_priv OKmysql.serversOKmysql.slow_log OKmysql.tables_privOKmysql.time_zoneOKmysql.time_zone_leap_secondOKmysql.time_zone_name OKmysql.time_zone_transition OKmysql.time_zone_transition_typeOKmysql.user OKRunning 'mysql_fix_privilege_tables'...Running 'mysqlcheck with default connection argumentsRunning 'mysqlcheck with default connection argumentspercona.heartbeatOKOK
|
[root@master~]# rpm -e Percona-Server-devel-55-5.5.36-rel34.2.el6.x86_64 Percona-Server-shared-55-5.5.36-rel34.2.el6.x86_64 Percona-Server-client-55-5.5.36-rel34.2.el6.x86_64 Percona-Server-server-55-5.5.36-rel34.2.el6.x86_64 --nodeps [root@master~]# yum install Percona-Server-server-56.x86_64 ============================================================================================================== Package Arch Version Repository Size ============================================================================================================== Installing:Percona-Server-server-56 x86_64 5.6.16-rel64.2.el6 Percona 19M Installingfordependencies: Percona-Server-client-56 x86_64 5.6.16-rel64.2.el6 Percona 6.8M Percona-Server-shared-56 x86_64 5.6.16-rel64.2.el6 Percona 712k TransactionSummary ============================================================================================================== Install 3Package(s) ...[root@master~]# service mysql start StartingMySQL(PerconaServer).......SUCCESS! [root@master~]# mysql WelcometotheMySQLmonitor. Commandsendwith;or/g. YourMySQLconnectionidis1 Serverversion:5.6.16-64.2-56-logPerconaServer(GPL),Release64.2,Revision569 [root@master~]# mysql_upgrade Lookingfor'mysql'as:mysql Lookingfor'mysqlcheck'as:mysqlcheck Running'mysqlcheckwithdefaultconnectionarguments Running'mysqlcheckwithdefaultconnectionarguments mysql.columns_priv OK mysql.db OK mysql.event OK mysql.func OK mysql.general_log OK mysql.help_category OK mysql.help_keyword OK mysql.help_relation OK mysql.help_topic OK mysql.host OK mysql.ndb_binlog_index OK mysql.plugin OK mysql.proc OK mysql.procs_priv OK mysql.proxies_priv OK mysql.servers OK mysql.slow_log OK mysql.tables_priv OK mysql.time_zone OK mysql.time_zone_leap_second OK mysql.time_zone_name OK mysql.time_zone_transition OK mysql.time_zone_transition_type OK mysql.user OK Running'mysql_fix_privilege_tables'... Running'mysqlcheckwithdefaultconnectionarguments Running'mysqlcheckwithdefaultconnectionarguments percona.heartbeat OK OK |
I can now verify that Ike’s INFORMATION_SCHEMA queries correctly detect the ‘heartbeat.ts’ column as the old format:
master mysql> select t.table_schema,t.engine,t.table_name,c.column_name,c.column_typefrom information_schema.tables tinner join information_schema.columns c on c.table_schema = t.table_schema and c.table_name = t.table_nameleft outer join information_schema.innodb_sys_tables ist on ist.name = concat(t.table_schema,'/',t.table_name)left outer join information_schema.innodb_sys_columns isc on isc.table_id = ist.table_id and isc.name = c.column_namewhere c.column_type in ('time','timestamp','datetime')and t.table_schema not in ('mysql','information_schema','performance_schema')and t.table_type = 'base table'and (t.engine != 'innodb' or (t.engine = 'innodb' and isc.mtype = 6))order by t.table_schema,t.table_name,c.column_name; +--------------+--------+------------+-------------+-------------+ | table_schema | engine | table_name | column_name | column_type | +--------------+--------+------------+-------------+-------------+ | percona| InnoDB | heartbeat| ts| datetime| +--------------+--------+------------+-------------+-------------+ 1 row in set (0.04 sec)
|
mastermysql>selectt.table_schema,t.engine,t.table_name,c.column_name,c.column_type frominformation_schema.tablest innerjoininformation_schema.columnsconc.table_schema=t.table_schemaandc.table_name=t.table_name leftouterjoininformation_schema.innodb_sys_tablesistonist.name=concat(t.table_schema,'/',t.table_name) leftouterjoininformation_schema.innodb_sys_columnsisconisc.table_id=ist.table_idandisc.name=c.column_name wherec.column_typein('time','timestamp','datetime') andt.table_schemanotin('mysql','information_schema','performance_schema') andt.table_type='base table' and(t.engine!='innodb'or(t.engine='innodb'andisc.mtype=6)) orderbyt.table_schema,t.table_name,c.column_name; +--------------+--------+------------+-------------+-------------+ |table_schema|engine|table_name|column_name|column_type| +--------------+--------+------------+-------------+-------------+ |percona |InnoDB|heartbeat |ts |datetime | +--------------+--------+------------+-------------+-------------+ 1rowinset(0.04sec) |
To make replication work from MySQL 5.6 to 5.5, I also had to add a few backwards compatibility options on the master:
log_bin_use_v1_row_events = ONbinlog_checksum = NONE
|
log_bin_use_v1_row_events=ON binlog_checksum=NONE |
Once I fixed that up, I can verify my slave is still working after this and receiving heartbeats. Clearly the new formats are not a show-stopper for backwards replication compatibility.
slave mysql> show slave status/G*************************** 1. row *************************** Slave_IO_State: Waiting for master to send eventMaster_Host: 192.168.70.2Master_User: replMaster_Port: 3306Connect_Retry: 60Master_Log_File: master-bin.000005Read_Master_Log_Pos: 120 Relay_Log_File: slave-relay-bin.000002Relay_Log_Pos: 267Relay_Master_Log_File: master-bin.000005 Slave_IO_Running: YesSlave_SQL_Running: Yesmaster mysql> select * from heartbeat;+-----------+-------------------+----------+-----------------------+---------------------+---------------------+| server_id | file| position | relay_master_log_file | exec_master_log_pos | ts|+-----------+-------------------+----------+-----------------------+---------------------+---------------------+| 1 | master-bin.000002 |5115935 | NULL|NULL | 2014-05-02 17:04:23 |+-----------+-------------------+----------+-----------------------+---------------------+---------------------+1 row in set (0.01 sec)slave mysql> select * from heartbeat;+-----------+-------------------+----------+-----------------------+---------------------+---------------------+| server_id | file| position | relay_master_log_file | exec_master_log_pos | ts|+-----------+-------------------+----------+-----------------------+---------------------+---------------------+| 1 | master-bin.000002 |5115935 | NULL|NULL | 2014-05-02 17:04:23 |+-----------+-------------------+----------+-----------------------+---------------------+---------------------+1 row in set (0.00 sec)
|
slavemysql>showslavestatus/G ***************************1.row*************************** Slave_IO_State:Waitingformastertosendevent Master_Host:192.168.70.2 Master_User:repl Master_Port:3306 Connect_Retry:60 Master_Log_File:master-bin.000005 Read_Master_Log_Pos:120 Relay_Log_File:slave-relay-bin.000002 Relay_Log_Pos:267 Relay_Master_Log_File:master-bin.000005 Slave_IO_Running:Yes Slave_SQL_Running:Yes mastermysql>select*fromheartbeat; +-----------+-------------------+----------+-----------------------+---------------------+---------------------+ |server_id|file |position|relay_master_log_file|exec_master_log_pos|ts | +-----------+-------------------+----------+-----------------------+---------------------+---------------------+ | 1|master-bin.000002| 5115935|NULL | NULL|2014-05-0217:04:23| +-----------+-------------------+----------+-----------------------+---------------------+---------------------+ 1rowinset(0.01sec) slavemysql>select*fromheartbeat; +-----------+-------------------+----------+-----------------------+---------------------+---------------------+ |server_id|file |position|relay_master_log_file|exec_master_log_pos|ts | +-----------+-------------------+----------+-----------------------+---------------------+---------------------+ | 1|master-bin.000002| 5115935|NULL | NULL|2014-05-0217:04:23| +-----------+-------------------+----------+-----------------------+---------------------+---------------------+ 1rowinset(0.00sec) |
But, if I’m not careful on MySQL 5.6, and rebuild the table, the new format does clearly bite me:
master mysql> set sql_log_bin=0;Query OK, 0 rows affected (0.00 sec)master mysql> alter table percona.heartbeat force;Query OK, 1 row affected, 1 warning (0.18 sec)Records: 1Duplicates: 0Warnings: 1master mysql> show warnings;+-------+------+-------------------------------------------------------------------------------------+| Level | Code | Message |+-------+------+-------------------------------------------------------------------------------------+| Note| 1880 | TIME/TIMESTAMP/DATETIME columns of old format have been upgraded to the new format. |+-------+------+-------------------------------------------------------------------------------------+1 row in set (0.00 sec)slave mysql> show slave status/G*************************** 1. row ***************************... Slave_IO_Running: YesSlave_SQL_Running: No... Last_Errno: 1677 Last_Error: Column 5 of table 'percona.heartbeat' cannot be converted from type '' to type 'datetime'... Last_SQL_Errno: 1677 Last_SQL_Error: Column 5 of table 'percona.heartbeat' cannot be converted from type '' to type 'datetime'Replicate_Ignore_Server_Ids: Master_Server_Id: 11 row in set (0.00 sec)
|
mastermysql>setsql_log_bin=0; QueryOK,0rowsaffected(0.00sec) mastermysql>altertablepercona.heartbeatforce; QueryOK,1rowaffected,1warning(0.18sec) Records:1 Duplicates:0 Warnings:1 mastermysql>showwarnings; +-------+------+-------------------------------------------------------------------------------------+ |Level|Code|Message | +-------+------+-------------------------------------------------------------------------------------+ |Note |1880|TIME/TIMESTAMP/DATETIMEcolumnsofoldformathavebeenupgradedtothenewformat.| +-------+------+-------------------------------------------------------------------------------------+ 1rowinset(0.00sec) slavemysql>showslavestatus/G ***************************1.row*************************** ...Slave_IO_Running:Yes Slave_SQL_Running:No ...Last_Errno:1677 Last_Error:Column5oftable'percona.heartbeat'cannotbeconvertedfromtype''totype'datetime' ...Last_SQL_Errno:1677 Last_SQL_Error:Column5oftable'percona.heartbeat'cannotbeconvertedfromtype''totype'datetime' Replicate_Ignore_Server_Ids: Master_Server_Id:1 1rowinset(0.00sec) |
TL;DR
What does all this teach us?
While the MySQL version is important, for RBR what matters most is the actual current format for each column. Your master and slave(s) MUST have the same column formats for RBR to work right.
So, the new temporal formats do not necessarily break RBR replication back to 5.5, provided:
- All base MySQL 5.6 enhancements to replication are disabled (binlog checksums and the RBR v2 format)
- Tables with temporal formats are preserved in their 5.5 formats until all 5.5 nodes are retired.
- You can avoid creating any new tables on the MySQL 5.6 master with temporal formats
However, I want to make it clear that MySQL 5.6 to 5.5 replication is technically unsupported. I have not exhausted all possibilities for problems with 5.6 to 5.5 RBR replication, just this specific one. If you choose to make an upgrade strategy that relies on backwards replication in this way, be prepared for it to not work and test it thoroughly in advance. The purpose of this post is to simply point out that data type formats, in and of themselves, do not necessarily break RBR backwards compatibility.
上一篇： 让你的 Web 应用程序飞起来
下一篇： TSRC挑战赛： PHP防御绕过挑战实录