我们经常看到表格的一个 int 列,由于值超出范围,需要先更改为 unsigned-int,然后再更改为 unsigned-bigint。有时,甚至可能存在阻止我们直接更改表或在主节点上应用 pt-online-schema-change 工具,这需要轮换解决方案:首先将更改应用到副本上,将写入切换到副本,然后将更改应用于前一个主节点。在这种情况下,MySQL 将不得不将 unsigned-int 复制到 unsigned-bigint 一段时间。
有人可能认为 MySQL 应该能够将 unsigned-int 复制到 unsigned-bigint 是显而易见和直接的,因为 unsigned-bigint 具有更大的大小(8 字节),它涵盖了 unsigned-int(4 字节)。这部分是正确的,但在实践中有一些技巧。本博客将通过场景向您展示这些技巧。
让我们了解从主库中的 unsigned-int 复制到副本中的 unsigned-bigint 时可能面临的场景和问题。对于场景一和二,我们将只关注 binlog_format=ROW,因为使用 binlog_format=STATEMENT,“如果在源上运行的语句也将在副本上成功执行,它也应该成功复制” – MySQL doc。但是,对于场景三,我们测试了 binlog_format=ROW 和 binlog_format=STATEMENT。
首先,摆好表。在这里,我们有一个表test_replication_differ_type ,主数据库中为 unsigned int ,副本中为unsigned bigint 。
主服务器版本:MySQL 8.0.28
CREATE TABLE `test_replication_differ_type` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`replicate_col` int(10) unsigned DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
从服务器版本:MySQL 8.0.28
CREATE TABLE `test_replication_differ_type` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`replicate_col` bigint(20) unsigned DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
场景一:在主中插入一个unsigned int并复制到unsigned bigint
mysql> insert into test.test_replication_differ_type(id,replicate_col) values(1,1);
Query OK, 1 row affected (0.00 sec)
mysql> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
+----+---------------+
root@localhost [test]> show replica status\G
<strong>Last_SQL_Errno: 1677</strong>
Last_SQL_Error: Column 1 of table 'test.test_replication_differ_type' cannot be converted from type 'int' to type 'bigint(20)'
分析及解决方案:
我们立即看到阻止我们复制的错误。现在有一个配置选项“slave_type_conversions”,它控制副本上使用的类型转换模式。 此复制错误的原因是,在副本上,默认情况下未设置slave_type_conversions变量。
root@localhost [test]> show variables like 'slave_type%';
+------------------------+-------+
| Variable_name | Value |
+------------------------+-------+
| slave_type_conversions | |
将slave_type_conversions设置为ALL_LOSSY模式将不起作用,因为: ”要求有损转换或根本不允许转换;例如,仅启用此模式允许将 INT 列转换为 TINYINT(有损转换),但不允许将 TINYINT 列转换为 INT 列(非有损)。” —— MySQL 文档
root@localhost [test]> set global slave_type_conversions='ALL_LOSSY';
Query OK, 0 rows affected (0.00 sec)
root@localhost [test]> start slave;
Query OK, 0 rows affected (0.01 sec)
root@localhost [test]> show slave status\G
Last_Errno: 1677
Last_Error: Column 1 of table 'test.test_replication_differ_type' cannot be converted from type 'int' to type 'bigint(20)'
将slave_type_conversions设置为ALL_NON_LOSSY模式将起作用,因为该模式:也就是说,它允许在目标类型的范围比源类型更宽的情况下进行转换。” —— MySQL 文档
root@localhost [test]> set global slave_type_conversions='ALL_NON_LOSSY';
Query OK, 0 rows affected (0.00 sec)
root@localhost [test]> start slave;
Query OK, 0 rows affected (0.01 sec)
root@localhost [test]> show slave status\G
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Seconds_Behind_Master: 0root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
+----+---------------+
不要被名字搞糊涂了,ALL_LOSSY和ALL_NON_LOSSY不是独占的,相反,它们可以并行添加以允许两种模式:
root@localhost [test]> set global slave_type_conversions='ALL_NON_LOSSY, ALL_LOSSY';
mysql> insert into test.test_replication_differ_type(id,replicate_col) values(2,2);
Query OK, 1 row affected (0.00 sec)
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
+----+---------------+
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
+----+---------------+
因此,如果您想使用不同的数据类型进行复制,请考虑在全局变量slave_type_conversions中适当地使用模式ALL_NON_LOSSY和/或ALL_LOSSY 。
场景二: 插入一个超出有符号整数范围的值并复制到无符号大整数
有符号整数的范围值为 (-2147483648, 2147483647)。让我们尝试将 2147483647+1 插入到主节点的 unsigned int 列中,并观察它是如何复制的。应该没事吧?
mysql> insert into test.test_replication_differ_type(id, replicate_col) values(3, 2147483647);
Query OK, 1 row affected (0.00 sec)
mysql> insert into test.test_replication_differ_type(id, replicate_col) values(4, 2147483648);
Query OK, 1 row affected (0.00 sec)
mysql> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | <strong>2147483648</strong> |
+----+---------------+
4 rows in set (0.00 sec)
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | <strong> 0</strong> |
+----+---------------+
4 rows in set (0.00 sec)
哦,我们在主节点上插入了 2147483648 ,但在副本上得到了0 。
分析及解决方案:
查看primary的binlog文件,可以看到unsigned int值2147483648写成如下图。@2=-2147483648 (2147483648)
root@db2:/home/vagrant# mysqlbinlog -vvv /var/lib/mysql/mysql-bin.000010 > test_convert.sql
root@db2:/home/vagrant# cat test_convert.sql
# at 2635
#220821 4:55:56 server id 2 end_log_pos 2679 CRC32 0x85dfff8a Write_rows: table id 113 flags: STMT_END_F
BINLOG '
3LoBYxMCAAAASAAAAEsKAAAAAHEAAAAAAAEABHRlc3QAHHRlc3RfcmVwbGljYXRpb25fZGlmZmVy
X3R5cGUAAgMDAAKLYMRs
3LoBYx4CAAAALAAAAHcKAAAAAHEAAAAAAAEAAgAC//wEAAAAAAAAgIr/34U=
'/*!*/;
### INSERT INTO `test`.`test_replication_differ_type`
### SET
### @1=4 /* INT meta=0 nullable=0 is_null=0 */
<strong>### @2=-2147483648 (2147483648) /* INT meta=0 nullable=1 is_null=0 */</strong>
# at 2679
#220821 4:55:56 server id 2 end_log_pos 2710 CRC32 0x532d66ec Xid = 89
COMMIT/*!*/;
但是,“当提升整数类型时,不会保留其符号性。默认情况下,副本将所有这些值视为有符号”,因此-2147483648 ( 2147483648 )被视为有符号值-2147483648 ,最终变为0 ,因为unsigned bigint的最小值为0 。
我们需要通过向变量slave_type_conversions添加模式ALL_UNSIGNED来告诉 MySQL 副本将值-2147483648 ( 2147483648 ) 视为unsigned int而不是signed int ,如下例所示:
root@localhost [test]> set global slave_type_conversions='ALL_NON_LOSSY,ALL_UNSIGNED';
Query OK, 0 rows affected (0.00 sec)
mysql> insert into test.test_replication_differ_type(id,replicate_col) values(5,2147483649);
Query OK, 1 row affected (0.00 sec)
mysql> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | 2147483648 |
| <strong> 5 | 2147483649</strong> |
+----+---------------+
5 rows in set (0.00 sec)
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | 0 |
| <strong> 5 | 2147483649</strong> |
+----+---------------+
5 rows in set (0.00 sec)
我们也可以将模式添加为ALL_SIGNED ,但这一次,它不像ALL_LOSSY和ALL_NON_LOSSY是并行的。ALL_SIGNED比ALL_UNSIGNED具有更高的优先级,这意味着如果可能,MySQL 将首先将值视为有符号,否则将值视为无符号。
root@localhost [test]> set global slave_type_conversions='ALL_NON_LOSSY,ALL_LOSSY,ALL_UNSIGNED,ALL_SIGNED';
Query OK, 0 rows affected (0.00 sec)
mysql> insert into test.test_replication_differ_type(id,replicate_col) values(6,2147483650);
Query OK, 1 row affected (0.01 sec)mysql> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | 2147483648 |
| 5 | 2147483649 |
| <strong> 6 | 2147483650</strong> |
+----+---------------+
6 rows in set (0.00 sec)
MySQL
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | 0 |
| 5 | 2147483649 |
| <strong>6 | 0 |</strong>
+----+---------------+
6 rows in set (0.00 sec)
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | 0 |
| 5 | 2147483649 |
| <strong>6 | 0 |</strong>
+----+---------------+
6 rows in set (0.00 sec)
场景三: 副本具有额外的列,并且并非所有公共列都具有相同的数据类型。
文档中写道:“此外,当表的副本副本的列数多于源副本的列数时,表的公用列必须在两个表中使用相同的数据类型。” ——MySQL 文档
让我们验证这是否是行为。
root@localhost [test]> set global slave_type_conversions='ALL_NON_LOSSY,ALL_LOSSY,ALL_UNSIGNED';
Query OK, 0 rows affected (0.00 sec)root@localhost [test]> alter table test.test_replication_differ_type add column extra_col int unsigned;
Query OK, 0 rows affected (0.04 sec)
Records: 0 Duplicates: 0 Warnings: 0
root@localhost [test]> show create table test.test_replication_differ_type\G
*************************** 1. row ***************************
Table: test_replication_differ_type
Create Table: CREATE TABLE `test_replication_differ_type` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`replicate_col` bigint(20) unsigned DEFAULT NULL,
<strong>`extra_col` int(10) unsigned DEFAULT NULL,</strong>
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=9 DEFAULT CHARSET=latin1
1 row in set (0.00 sec)
mysql> show create table test.test_replication_differ_type\G
*************************** 1. row ***************************
Table: test_replication_differ_type
Create Table: CREATE TABLE `test_replication_differ_type` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`replicate_col` int(10) unsigned DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)mysql> insert into test.test_replication_differ_type(id,replicate_col) values(7,2147483651);
Query OK, 1 row affected (0.00 sec)
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+
| id | replicate_col |
+----+---------------+
| 1 | 1 |
| 2 | 2 |
| 3 | 2147483647 |
| 4 | 2147483648 |
| 5 | 2147483649 |
| 6 | 2147483650 |
<strong>| 7 | 2147483651 |</strong>
+----+---------------+
7 rows in set (0.00 sec)
root@localhost [test]> select * from test.test_replication_differ_type;
+----+---------------+-----------+
| id | replicate_col | extra_col |
+----+---------------+-----------+
| 1 | 1 | NULL |
| 2 | 2 | NULL |
| 3 | 2147483647 | NULL |
| 4 | 0 | NULL |
| 5 | 2147483649 | NULL |
| 6 | 0 | NULL |
<strong>| 7 | 2147483651 | NULL |</strong>
+----+---------------+-----------+
7 rows in set (0.00 sec)
现在,正如我们的实验所说,文档中的陈述是不正确的。事实是,即使副本上的replicate_col 具有数据类型bigint ( 20 ) unsigned而主副本上的 int ( 10 ) unsigned ,并且副本具有额外的列 extra_col,它仍然可以很好地复制。
我已经向Oracle MySQL提交了错误报告。
在这篇博客中,我说明了如何设置变量 slave_type_conversions,ALL_NON_LOSSY 和 ALL_LOSSY 模式的目的和区别,以及 ALL_UNSIGNED 和 ALL_SIGNED。
具体来说,为了让 MySQL 正确地从 unsigned-int 复制到 unsigned-bigint,我们需要设置变量 slave_type_conversions=‘ALL_NON_LOSSY,ALL_UNSIGNED’ OR slave_type_conversions=‘ALL_NON_LOSSY,ALL_LOSSY,ALL_UNSIGNED’
另外请注意,当replica比primary多了一个column时,只要common column同名且顺序相同,并且在extra column之前,即使common column数据类型与primary不同,复制仍然可以顺利进行。
原文标题:MySQL Replicate From Unsigned-int to Unsigned-bigint