欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  数据库

MySQL 大数据操作注意事项_MySQL

程序员文章站 2022-06-01 08:09:18
...

MySQL 大数据操作注意事项

http://netkiller.github.io/journal/mysql.parallel.html

Mr. Neo Chen (netkiller), 陈景峰(BG7NYT)


中国广东省深圳市龙华新区民治街道溪山美地
518131
+86 13113668890
+86 755 29812080

版权 © 2011, 2012, 2013, 2014 http://netkiller.github.io

版权声明

转载请与作者联系,转载时请务必标明文章原始出处和作者信息及本声明。

MySQL 大数据操作注意事项_MySQL
文档出处:
http://netkiller.github.io
http://netkiller.sourceforge.net

2014-05-16

摘要 我的系列文档
Netkiller Architect 手札 Netkiller Developer 手札 Netkiller PHP 手札 Netkiller Python 手札 Netkiller Testing 手札 Netkiller Cryptography 手札
Netkiller Linux 手札 Netkiller Debian 手札 Netkiller CentOS 手札 Netkiller FreeBSD 手札 Netkiller Shell 手札 Netkiller Security 手札
Netkiller Web 手札 Netkiller Monitoring 手札 Netkiller Storage 手札 Netkiller Mail 手札 Netkiller Docbook 手札 Netkiller Version 手札
Netkiller Database 手札 Netkiller PostgreSQL 手札 Netkiller MySQL 手札 Netkiller NoSQL 手札 Netkiller LDAP 手札 Netkiller Network 手札
Netkiller Cisco IOS 手札 Netkiller H3C 手札 Netkiller Multimedia 手札 Netkiller Perl 手札 Netkiller Amateur Radio 手札

目录
  • 1. 关于 delete
  • 2. 关于 update
  • 3. 关于创建索引
  • 4. 关于 OPTIMIZE
  • 5. 确保SELECT不被受阻

1. 关于 delete

delete from mytable 必死无疑,你需要分批删除,尽量缩小每个批次删除的记录数,delete 是可以并行执行的,你可以同时运行多个删除操作

mysql> show processlist;+--------+-----------------+---------------------+-----------+---------+-------+-----------------------------+--------------------------------------------------------+| Id     | User            | Host                | db        | Command | Time  | State                       | Info                                                   |+--------+-----------------+---------------------+-----------+---------+-------+-----------------------------+--------------------------------------------------------+|      1 | event_scheduler | localhost           | NULL      | Daemon  |    52 | Waiting for next activation | NULL                                                   || 115986 | dba             | localhost           | example   | Query   |     0 | NULL                        | show processlist                                       || 117446 | dba             | localhost           | example   | Query   |    20 | updating                    | delete from mytable where OPEN_TIME like '2011.11.28%' || 117525 | dba             | localhost           | example   | Query   |     2 | updating                    | delete from mytable where OPEN_TIME like '2011.12.02%' || 117526 | dba             | localhost           | example   | Query   |    49 | updating                    | delete from mytable where OPEN_TIME like '2011.12.12%' || 117527 | dba             | localhost           | example   | Query   |     6 | updating                    | delete from mytable where OPEN_TIME like '2011.12.21%' || 117528 | dba             | localhost           | example   | Query   |    64 | updating                    | delete from mytable where OPEN_TIME like '2011.12.30%' || 117546 | dba             | localhost           | example   | Query   |    33 | updating                    | delete from mytable where OPEN_TIME like '2011.11.10%' |+--------+-----------------+---------------------+-----------+---------+-------+-----------------------------+--------------------------------------------------------+23 rows in set (0.00 sec)

2. 关于 update

在电商领域常常遇到一个问题“调价”,经常需要调整一批商品的价格, 程序猿一条预警搞定有没有?

update goods set price=price+10 where category_id = xxx

在开发,测试环境是可以通过测试的,一旦部署到生产环境,必死无疑

3. 关于创建索引

大表创建索引需要很久的时间,通常要经历 manage keys 与 copy to tmp table 的过程

mysql> show processlist;+--------+-----------------+---------------------+----------+---------+-------+-----------------------------+------------------------------------------------------------------+| Id     | User            | Host                | db       | Command | Time  | State                       | Info                                                             |+--------+-----------------+---------------------+----------+---------+-------+-----------------------------+------------------------------------------------------------------+|      1 | event_scheduler | localhost           | NULL     | Daemon  |    47 | Waiting for next activation | NULL                                                             || 115986 | dba             | localhost           | example  | Query   |     0 | NULL                        | show processlist                                                 || 118814 | dba             | 192.168.6.20:50459  | example  | Query   |     8 | copy to tmp table           | ALTER TABLE `mytable` ADD INDEX `modifiy_time` (`MODIFY_TIME`)   |+--------+-----------------+---------------------+----------+---------+-------+-----------------------------+------------------------------------------------------------------+17 rows in set (0.00 sec)

删除索引,也需要经理 copy to tmp table 过程,漫长的等待

mysql> show processlist;+--------+-----------------+---------------------+--------------+---------+-------+-----------------------------+-------------------------------------------------+| Id     | User            | Host                | db           | Command | Time  | State                       | Info                                            |+--------+-----------------+---------------------+--------------+---------+-------+-----------------------------+-------------------------------------------------+|      1 | event_scheduler | localhost           | NULL         | Daemon  |    11 | Waiting for next activation | NULL                                            || 115986 | dba             | localhost           | example      | Query   |     0 | NULL                        | show processlist                                || 118814 | dba             | 192.168.6.20:50459  | example      | Query   |     4 | copy to tmp table           | ALTER TABLE `mytable`	DROP INDEX `modifiy_time` |+--------+-----------------+---------------------+--------------+---------+-------+-----------------------------+-------------------------------------------------+17 rows in set (0.00 sec)

所以数据设计要深思熟虑,做到提前未雨绸缪,不要亡羊补牢

4. 关于 OPTIMIZE

OPTIMIZE 的操作是将当前表复制到临时表操作后再删除当前表,最后将临时表改名

mysql> show processlist;+--------+-----------------+---------------------+---------------------------+---------+-------+-----------------------------+--------------------------+| Id     | User            | Host                | db                        | Command | Time  | State                       | Info                     |+--------+-----------------+---------------------+---------------------------+---------+-------+-----------------------------+--------------------------+|      1 | event_scheduler | localhost           | NULL                      | Daemon  |    14 | Waiting for next activation | NULL                     || 115835 | dba             | 192.168.6.20:49664  | example                   | Query   |     9 | copy to tmp table           | OPTIMIZE TABLE `mytable` || 115986 | dba             | localhost           | example                   | Query   |     0 | NULL                        | show processlist         |+--------+-----------------+---------------------+---------------------------+---------+-------+-----------------------------+--------------------------+17 rows in set (0.00 sec)

5. 确保SELECT不被受阻

使用各种手段保证select操作不被受阻,只要select一直可以查询网站前端就能提供80%的功能,一旦select受阻一切都是浮云。

保证 select 操作优先于其他操作

UPDATE [LOW_PRIORITY] [IGNORE] tbl_name  SET col_name1=expr1 [, col_name2=expr2 ...]  [WHERE where_definition]  [ORDER BY ...]  [LIMIT row_count]

update的时候增加 LOW_PRIORITY 参数,可以降低更新语句的优先级。

my.cnf

[mysqld]		low_priority_updates=1

或者启动是添加--low-priority-updates参数

全局开启

SET @@global.low_priority_updates = 1;

适用于本次会话连接

SET @@session.low_priority_updates = 1;