Redis快速入门（三）之持久化（persistence）

前言：本章为学习总结Redis持久化，版本为4.0.10。

Redis持久化简介：

    当使用Redis进行数据操作的时候，不可避免的能会遇到一些意外的情况导致Redis停止工作，诸如宕机，电源断电等因素。此时Redis提供了RDB以及AOF两种方式来进行数据的备份，两者各有优缺点。可以选择其中的一个单独使用，也支持两者同时使用。通常的情况下AOF保存的数据会比RDB完整，此外在两者同时使用的情况下会优先载入AOF来进行数据恢复。下面介绍下两者的优缺点及其实现。

Redis持久化之RDB：

  RDB简介：

    RDB（Redis DataBase）通过快照（SnapShotting）来实现在指定的时候内对数据进行保存。可以通过配置文件来修改RDB的触发，文件名等等。同也可以通过手动调用SAVE（阻塞）或者BGSAVE命令来触发RDB。

################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
#   save <seconds> <changes>
#
#   Will save the DB if both the given number of seconds and the given
#   number of write operations against the DB occurred.
#
#   In the example below the behaviour will be to save:
#   after 900 sec (15 min) if at least 1 key changed
#   after 300 sec (5 min) if at least 10 keys changed
#   after 60 sec if at least 10000 keys changed
#
#   Note: you can disable saving completely by commenting out all "save" lines.
#
#   It is also possible to remove all the previously configured save
#   points by adding a save directive with a single empty string argument
#   like in the following example:
#
#   save ""  
#   配置RDB的触发条件使用save ""或者全部注释则关闭RDB
save 900 1
save 300 10
save 60 10000

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
# 如果bgsave发生错误，停止Redis数据写入，保证数据的一致性。
stop-writes-on-bgsave-error yes

# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
# 使用LZF算法，当dump.rdb文件过大的时候消耗CPU进行压缩
rdbcompression yes

# Since version 5 of RDB a CRC64 checksum is placed at the end of the file.
# This makes the format more resistant to corruption but there is a performance
# hit to pay (around 10%) when saving and loading RDB files, so you can disable it
# for maximum performances.
#
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
# 使用CRC64算法来进行数据校验，但是在会有10%的性能消耗。
rdbchecksum yes

# The filename where to dump the DB
# 设置 生成的RDB文件名
dbfilename dump.rdb

# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./

  RDB工作方式：

    首先Redis在需要保存dump.rdb文件的时候（满足配置中的触发条件）会先fork一个子进程。这个子进程可以说是复制了当前进程，两者拥有一样的环境。此时主进程继续执行数据的读写操作，而子进程则负责把数据写到临时的RDB文件中，当子进程完成对临时的RDB文件的写入后，会用这个临时文件来替换旧的RDB文件并删除旧的文件，这个过程就是写时复制（Copy-on-Write）。

  RDB的优缺点：

    RDB的优点是恢复大数据集的时候速度更快。

    RDB的缺点是保存的数据不完整，因为其每次触发都有时间间隔因此一旦发生故障，而部分数据还没有保存到快照中那么恢复的时候会缺少数据；此外在数据量集大的时候fock的过程耗时加大会影响到客户端的请求响应。

Redis持久化之AOF：

  AOF简介：

    通过前文可以知道RDB在保存数据的时候会有完整性问题，因此Redis支持了另一个持久化方式也就是AOF（Append-Only File）来保证数据的完整 。每当Redis执行一次改变数据的命令时候，这个命令就会追加到aof文件末尾中，当恢复数据的时候根据aof中的指令一条条进行恢复。此外随着Redis的运行aof文件会越来越大当超过了配置文件中的值后会触发日志重写，所谓的重写指一些命令的优化，例如对一个数据执行了100次Incr仅仅是为了保存数据,那实际上只要一条set语句就可以实现其余99条就是多余的，也可以手动使用命令BGREWRITEAOF来执行日志重写。下面是AOF的配置模块。

############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.
# 开启AOF，默认关闭
appendonly no

# The name of the append only file (default: "appendonly.aof")
# 保存Redis指令的aof文件，用此文件来恢复数据
appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".
# AOF触发条件，默认每秒fsync一次 everysec，还有从不fsync以及每个指令fsync一次（always）
# appendfsync always
appendfsync everysec
# appendfsync no

# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.
# 重写的时候是否使用appendfsync 
no-appendfsync-on-rewrite no

# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.
# 当aof文件过大的时候AOF会重写aof文件，当当前的文件大小是上次重写后大小的一倍并且内存大于64MB的时候触发重写
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

# An AOF file may be found to be truncated at the end during the Redis
# startup process, when the AOF data gets loaded back into memory.
# This may happen when the system where Redis is running
# crashes, especially when an ext4 filesystem is mounted without the
# data=ordered option (however this can't happen when Redis itself
# crashes or aborts but the operating system still works correctly).
#
# Redis can either exit with an error when this happens, or load as much
# data as possible (the default now) and start if the AOF file is found
# to be truncated at the end. The following option controls this behavior.
#
# If aof-load-truncated is set to yes, a truncated AOF file is loaded and
# the Redis server starts emitting a log to inform the user of the event.
# Otherwise if the option is set to no, the server aborts with an error
# and refuses to start. When the option is set to no, the user requires
# to fix the AOF file using the "redis-check-aof" utility before to restart
# the server.
#
# Note that if the AOF file will be found to be corrupted in the middle
# the server will still exit with an error. This option only applies when
# Redis will try to read more data from the AOF file but not enough bytes
# will be found.
aof-load-truncated yes

# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes

AOF的工作方式：

   一开始Redis在执行数据操作的时候当涉及到改变数据的命令时就会爸把命令写到aof文件中。当触发重写的时候AOF跟RDB一样利用了写时复制（Copy-On-Write）。AOF会fork一个子进程，子进程会把新的aof文件（重写后的）写到一个临时文件中，此时父进程一边把指令放到一个内存缓存中一边把指令写入到旧的aof文件，这样可以防止在重写的过程出现意外保证现有数据的完整性。当子进程完成重写后给父进程发送一个信号，父进程接收到信号后把数据写到新的aof文件的末尾，最后用新的aof文件替换旧的文件，完成重写过程。

AOF的文件修复：

    当往aof文件中写入数据，有可能会发生数据写到一半或者其他各种导致指令不够完整无法形成一条正常的指令。这个时候就无法还原数据。这个时候使用指令redis-check-aof --fix appendonly.aof可以实现文件的修复。Redis就把文件中不规范的语句全部删掉。这个时候重启Redies服务就可以看到数据恢复。

AOF的优缺点：

    AOF的优点是提供了三种策略来针对数据的存储，使用默认的每秒fsync策略Redis的性能仍然很好同时当出现故障的时候最多只会丢失一秒的数据，如果你指定always策略，性能比较差但是能保证数据完整；针对FLUSHALL的时候RDB会立刻清除掉dump.rdb文件，而AOF只是在aof文件的末尾加上FLUSHALL命令，只要把这条指令删除就可以立刻恢复数据。

    AOF的缺点是aof的文件会大于rdb文件，在恢复数据的速度会慢于RDB；在处理巨大的写入载入时RDB可以提供更有保证的最大延迟时间。