C基础 带你手写 redis adlist 双向链表
程序员文章站
2022-10-04 11:57:28
引言 - 导航栏目 有些朋友可能对 redis 充满着数不尽的求知欲, 也许是 redis 属于工作, 交流(面试)的大头戏, 不得不 ... 而自己当下对于 redis 只是停留在会用层面, 细节层面几乎没有涉猎. 为了更快的融于大 家, 这里尝试抛砖引玉. 先带大家手写个 redis 中最简单的 ......
引言 - 导航栏目
有些朋友可能对 redis 充满着数不尽的求知欲, 也许是 redis 属于工作, 交流(面试)的大头戏,
不得不 ... 而自己当下对于 redis 只是停留在会用层面, 细节层面几乎没有涉猎. 为了更快的融于大
家, 这里尝试抛砖引玉. 先带大家手写个 redis 中最简单的数据结构, adlist 双向链表. 让我们一
起对 redis 有个初步的认知. 本文会从下面几个标题展开解读(吐槽), 欢迎交流和指正.
1. redis adlist 解析
2. redis config.h 分析
3. redis setproctitle.c 分析
4. redis atomicvar.h 分析
5. redis zmalloc 分析
6. redis makefile 解析
redis 大头是 c 写的, 而 c 啥也不缺, 就缺手写, ok 开始废话手写之旅吧 :)
前言 - redis adlist
全篇示例代码都有手写过, 不过为了素材正规, 这里直接原封不动的引用
github.com/antirez/redis 中相关代码.
1. redis adlist 解析
1 /* adlist.h - a generic doubly linked list implementation
2 *
3 * copyright (c) 2006-2012, salvatore sanfilippo <antirez at gmail dot com>
4 * all rights reserved.
5 *
6 * redistribution and use in source and binary forms, with or without
7 * modification, are permitted provided that the following conditions are met:
8 *
9 * * redistributions of source code must retain the above copyright notice,
10 * this list of conditions and the following disclaimer.
11 * * redistributions in binary form must reproduce the above copyright
12 * notice, this list of conditions and the following disclaimer in the
13 * documentation and/or other materials provided with the distribution.
14 * * neither the name of redis nor the names of its contributors may be used
15 * to endorse or promote products derived from this software without
16 * specific prior written permission.
17 *
18 * this software is provided by the copyright holders and contributors "as is"
19 * and any express or implied warranties, including, but not limited to, the
20 * implied warranties of merchantability and fitness for a particular purpose
21 * are disclaimed. in no event shall the copyright owner or contributors be
22 * liable for any direct, indirect, incidental, special, exemplary, or
23 * consequential damages (including, but not limited to, procurement of
24 * substitute goods or services; loss of use, data, or profits; or business
25 * interruption) however caused and on any theory of liability, whether in
26 * contract, strict liability, or tort (including negligence or otherwise)
27 * arising in any way out of the use of this software, even if advised of the
28 * possibility of such damage.
29 */
30
31 #ifndef __adlist_h__
32 #define __adlist_h__
33
34 /* node, list, and iterator are the only data structures used currently. */
35
36 typedef struct listnode {
37 struct listnode *prev;
38 struct listnode *next;
39 void *value;
40 } listnode;
41
42 typedef struct listiter {
43 listnode *next;
44 int direction;
45 } listiter;
46
47 typedef struct list {
48 listnode *head;
49 listnode *tail;
50 void *(*dup)(void *ptr);
51 void (*free)(void *ptr);
52 int (*match)(void *ptr, void *key);
53 unsigned long len;
54 } list;
55
56 /* functions implemented as macros */
57 #define listlength(l) ((l)->len)
58 #define listfirst(l) ((l)->head)
59 #define listlast(l) ((l)->tail)
60 #define listprevnode(n) ((n)->prev)
61 #define listnextnode(n) ((n)->next)
62 #define listnodevalue(n) ((n)->value)
63
64 #define listsetdupmethod(l,m) ((l)->dup = (m))
65 #define listsetfreemethod(l,m) ((l)->free = (m))
66 #define listsetmatchmethod(l,m) ((l)->match = (m))
67
68 #define listgetdupmethod(l) ((l)->dup)
69 #define listgetfreemethod(l) ((l)->free)
70 #define listgetmatchmethod(l) ((l)->match)
71
72 /* prototypes */
73 list *listcreate(void);
74 void listrelease(list *list);
75 void listempty(list *list);
76 list *listaddnodehead(list *list, void *value);
77 list *listaddnodetail(list *list, void *value);
78 list *listinsertnode(list *list, listnode *old_node, void *value, int after);
79 void listdelnode(list *list, listnode *node);
80 listiter *listgetiterator(list *list, int direction);
81 listnode *listnext(listiter *iter);
82 void listreleaseiterator(listiter *iter);
83 list *listdup(list *orig);
84 listnode *listsearchkey(list *list, void *key);
85 listnode *listindex(list *list, long index);
86 void listrewind(list *list, listiter *li);
87 void listrewindtail(list *list, listiter *li);
88 void listrotate(list *list);
89 void listjoin(list *l, list *o);
90
91 /* directions for iterators */
92 #define al_start_head 0
93 #define al_start_tail 1
94
95 #endif /* __adlist_h__ */
首先手写的是 adlist.h 双向链表的头文件, 对于这个头文件有几点要聊一聊的.
1.1' redis 中头文件格式目前没有统一
#ifndef __adlist_h__ #endif #ifndef __redis_help_h #endif #ifndef __zmalloc_h #endif
可能也是, redis 这个项目维护和开发都十年多了. 代码风格在变(千奇百怪)也是正常.
这里推荐第三种写法 -> __{不带后缀文件名}_h
1.2' adlist.h 中函数命名随意
void listreleaseiterator(listiter *iter); list *listdup(list *orig); listnode *listsearchkey(list *list, void *key); listnode *listindex(list *list, long index); void listrewind(list *list, listiter *li); void listrewindtail(list *list, listiter *li); void listrotate(list *list); void listjoin(list *l, list *o);
命名随意不是个好习惯, 推荐参数名强区分. 例如下面这样固定格式
extern void listreleaseiterator(listiter * iter); extern list * listdup(list * l); extern listnode * listsearchkey(list * l, void * key); extern listnode * listindex(list * l, long index); extern void listrewind(list * l, listiter * iter); extern void listrewindtail(list * l, listiter * iter); extern void listrotate(list * l); extern void listjoin(list * l, list * o);
写完 adlist.h 接口定义部分, 相信有些人对待实现的 adlist.c 也有了大致轮廓了吧 :)
1 /* adlist.c - a generic doubly linked list implementation
2 *
3 * copyright (c) 2006-2010, salvatore sanfilippo <antirez at gmail dot com>
4 * all rights reserved.
5 *
6 * redistribution and use in source and binary forms, with or without
7 * modification, are permitted provided that the following conditions are met:
8 *
9 * * redistributions of source code must retain the above copyright notice,
10 * this list of conditions and the following disclaimer.
11 * * redistributions in binary form must reproduce the above copyright
12 * notice, this list of conditions and the following disclaimer in the
13 * documentation and/or other materials provided with the distribution.
14 * * neither the name of redis nor the names of its contributors may be used
15 * to endorse or promote products derived from this software without
16 * specific prior written permission.
17 *
18 * this software is provided by the copyright holders and contributors "as is"
19 * and any express or implied warranties, including, but not limited to, the
20 * implied warranties of merchantability and fitness for a particular purpose
21 * are disclaimed. in no event shall the copyright owner or contributors be
22 * liable for any direct, indirect, incidental, special, exemplary, or
23 * consequential damages (including, but not limited to, procurement of
24 * substitute goods or services; loss of use, data, or profits; or business
25 * interruption) however caused and on any theory of liability, whether in
26 * contract, strict liability, or tort (including negligence or otherwise)
27 * arising in any way out of the use of this software, even if advised of the
28 * possibility of such damage.
29 */
30
31
32 #include <stdlib.h>
33 #include "adlist.h"
34 #include "zmalloc.h"
35
36 /* create a new list. the created list can be freed with
37 * alfreelist(), but private value of every node need to be freed
38 * by the user before to call alfreelist().
39 *
40 * on error, null is returned. otherwise the pointer to the new list. */
41 list *listcreate(void)
42 {
43 struct list *list;
44
45 if ((list = zmalloc(sizeof(*list))) == null)
46 return null;
47 list->head = list->tail = null;
48 list->len = 0;
49 list->dup = null;
50 list->free = null;
51 list->match = null;
52 return list;
53 }
54
55 /* remove all the elements from the list without destroying the list itself. */
56 void listempty(list *list)
57 {
58 unsigned long len;
59 listnode *current, *next;
60
61 current = list->head;
62 len = list->len;
63 while(len--) {
64 next = current->next;
65 if (list->free) list->free(current->value);
66 zfree(current);
67 current = next;
68 }
69 list->head = list->tail = null;
70 list->len = 0;
71 }
72
73 /* free the whole list.
74 *
75 * this function can't fail. */
76 void listrelease(list *list)
77 {
78 listempty(list);
79 zfree(list);
80 }
81
82 /* add a new node to the list, to head, containing the specified 'value'
83 * pointer as value.
84 *
85 * on error, null is returned and no operation is performed (i.e. the
86 * list remains unaltered).
87 * on success the 'list' pointer you pass to the function is returned. */
88 list *listaddnodehead(list *list, void *value)
89 {
90 listnode *node;
91
92 if ((node = zmalloc(sizeof(*node))) == null)
93 return null;
94 node->value = value;
95 if (list->len == 0) {
96 list->head = list->tail = node;
97 node->prev = node->next = null;
98 } else {
99 node->prev = null;
100 node->next = list->head;
101 list->head->prev = node;
102 list->head = node;
103 }
104 list->len++;
105 return list;
106 }
107
108 /* add a new node to the list, to tail, containing the specified 'value'
109 * pointer as value.
110 *
111 * on error, null is returned and no operation is performed (i.e. the
112 * list remains unaltered).
113 * on success the 'list' pointer you pass to the function is returned. */
114 list *listaddnodetail(list *list, void *value)
115 {
116 listnode *node;
117
118 if ((node = zmalloc(sizeof(*node))) == null)
119 return null;
120 node->value = value;
121 if (list->len == 0) {
122 list->head = list->tail = node;
123 node->prev = node->next = null;
124 } else {
125 node->prev = list->tail;
126 node->next = null;
127 list->tail->next = node;
128 list->tail = node;
129 }
130 list->len++;
131 return list;
132 }
133
134 list *listinsertnode(list *list, listnode *old_node, void *value, int after) {
135 listnode *node;
136
137 if ((node = zmalloc(sizeof(*node))) == null)
138 return null;
139 node->value = value;
140 if (after) {
141 node->prev = old_node;
142 node->next = old_node->next;
143 if (list->tail == old_node) {
144 list->tail = node;
145 }
146 } else {
147 node->next = old_node;
148 node->prev = old_node->prev;
149 if (list->head == old_node) {
150 list->head = node;
151 }
152 }
153 if (node->prev != null) {
154 node->prev->next = node;
155 }
156 if (node->next != null) {
157 node->next->prev = node;
158 }
159 list->len++;
160 return list;
161 }
162
163 /* remove the specified node from the specified list.
164 * it's up to the caller to free the private value of the node.
165 *
166 * this function can't fail. */
167 void listdelnode(list *list, listnode *node)
168 {
169 if (node->prev)
170 node->prev->next = node->next;
171 else
172 list->head = node->next;
173 if (node->next)
174 node->next->prev = node->prev;
175 else
176 list->tail = node->prev;
177 if (list->free) list->free(node->value);
178 zfree(node);
179 list->len--;
180 }
181
182 /* returns a list iterator 'iter'. after the initialization every
183 * call to listnext() will return the next element of the list.
184 *
185 * this function can't fail. */
186 listiter *listgetiterator(list *list, int direction)
187 {
188 listiter *iter;
189
190 if ((iter = zmalloc(sizeof(*iter))) == null) return null;
191 if (direction == al_start_head)
192 iter->next = list->head;
193 else
194 iter->next = list->tail;
195 iter->direction = direction;
196 return iter;
197 }
198
199 /* release the iterator memory */
200 void listreleaseiterator(listiter *iter) {
201 zfree(iter);
202 }
203
204 /* create an iterator in the list private iterator structure */
205 void listrewind(list *list, listiter *li) {
206 li->next = list->head;
207 li->direction = al_start_head;
208 }
209
210 void listrewindtail(list *list, listiter *li) {
211 li->next = list->tail;
212 li->direction = al_start_tail;
213 }
214
215 /* return the next element of an iterator.
216 * it's valid to remove the currently returned element using
217 * listdelnode(), but not to remove other elements.
218 *
219 * the function returns a pointer to the next element of the list,
220 * or null if there are no more elements, so the classical usage patter
221 * is:
222 *
223 * iter = listgetiterator(list,<direction>);
224 * while ((node = listnext(iter)) != null) {
225 * dosomethingwith(listnodevalue(node));
226 * }
227 *
228 * */
229 listnode *listnext(listiter *iter)
230 {
231 listnode *current = iter->next;
232
233 if (current != null) {
234 if (iter->direction == al_start_head)
235 iter->next = current->next;
236 else
237 iter->next = current->prev;
238 }
239 return current;
240 }
241
242 /* duplicate the whole list. on out of memory null is returned.
243 * on success a copy of the original list is returned.
244 *
245 * the 'dup' method set with listsetdupmethod() function is used
246 * to copy the node value. otherwise the same pointer value of
247 * the original node is used as value of the copied node.
248 *
249 * the original list both on success or error is never modified. */
250 list *listdup(list *orig)
251 {
252 list *copy;
253 listiter iter;
254 listnode *node;
255
256 if ((copy = listcreate()) == null)
257 return null;
258 copy->dup = orig->dup;
259 copy->free = orig->free;
260 copy->match = orig->match;
261 listrewind(orig, &iter);
262 while((node = listnext(&iter)) != null) {
263 void *value;
264
265 if (copy->dup) {
266 value = copy->dup(node->value);
267 if (value == null) {
268 listrelease(copy);
269 return null;
270 }
271 } else
272 value = node->value;
273 if (listaddnodetail(copy, value) == null) {
274 listrelease(copy);
275 return null;
276 }
277 }
278 return copy;
279 }
280
281 /* search the list for a node matching a given key.
282 * the match is performed using the 'match' method
283 * set with listsetmatchmethod(). if no 'match' method
284 * is set, the 'value' pointer of every node is directly
285 * compared with the 'key' pointer.
286 *
287 * on success the first matching node pointer is returned
288 * (search starts from head). if no matching node exists
289 * null is returned. */
290 listnode *listsearchkey(list *list, void *key)
291 {
292 listiter iter;
293 listnode *node;
294
295 listrewind(list, &iter);
296 while((node = listnext(&iter)) != null) {
297 if (list->match) {
298 if (list->match(node->value, key)) {
299 return node;
300 }
301 } else {
302 if (key == node->value) {
303 return node;
304 }
305 }
306 }
307 return null;
308 }
309
310 /* return the element at the specified zero-based index
311 * where 0 is the head, 1 is the element next to head
312 * and so on. negative integers are used in order to count
313 * from the tail, -1 is the last element, -2 the penultimate
314 * and so on. if the index is out of range null is returned. */
315 listnode *listindex(list *list, long index) {
316 listnode *n;
317
318 if (index < 0) {
319 index = (-index)-1;
320 n = list->tail;
321 while(index-- && n) n = n->prev;
322 } else {
323 n = list->head;
324 while(index-- && n) n = n->next;
325 }
326 return n;
327 }
328
329 /* rotate the list removing the tail node and inserting it to the head. */
330 void listrotate(list *list) {
331 listnode *tail = list->tail;
332
333 if (listlength(list) <= 1) return;
334
335 /* detach current tail */
336 list->tail = tail->prev;
337 list->tail->next = null;
338 /* move it as head */
339 list->head->prev = tail;
340 tail->prev = null;
341 tail->next = list->head;
342 list->head = tail;
343 }
344
345 /* add all the elements of the list 'o' at the end of the
346 * list 'l'. the list 'other' remains empty but otherwise valid. */
347 void listjoin(list *l, list *o) {
348 if (o->head)
349 o->head->prev = l->tail;
350
351 if (l->tail)
352 l->tail->next = o->head;
353 else
354 l->head = o->head;
355
356 if (o->tail) l->tail = o->tail;
357 l->len += o->len;
358
359 /* setup other as an empty list. */
360 o->head = o->tail = null;
361 o->len = 0;
362 }
是的, 就是这样, 就是这样简单.
我们稍微多讲点, 其实对于 listcreate 可以写的更加简约, 不是吗?
struct list * listcreate(void) {
return zcalloc(sizeof(struct list));
}
好了, 那我们继续交流(吐槽)吧.
1.3' 代码括号 { } 位置随意
这不是个好习惯, 毕竟谁也不喜欢两面派. 大项目还是得需要在大方向上统一风格和约束.
1.4' struct listiter::direction 不一定是个很好的设计
direction 通过与 al_start_head or al_start_tail 宏进行运行时比对, 来区分遍历的方向. 觉得
有点浪费. 内心更倾向于干掉运行时比对, 从一开始用户就应该知道该怎么遍历更好, 毕竟这是所有数据结构
的标杆.
❤ 恭喜大家, 到这我们关于 redis adlist 最基础最简单的数据结构已经手写分析完毕, 后面可以不用看了.
谢谢大家捧场 ~
正文 - adlist 周边
简单愉快的背后总会有些更深的不可捉摸. 离开了奶头乐, 我们将从 adlist.c 中一行代码, 正式开启
我们此次探险之旅.
#include "zmalloc.h"
2. redis config.h 分析
同样在 zmallo.c 中我们发现了如下两行代码, 这就是我们要说的一个主体之一 config.h
#include "config.h" #include "atomicvar.h"
config.h 主要作用是用于确定程序的运行环境, 例如是什么操作系统, 是什么字节序, 要不要启用某些功能
/*
* copyright (c) 2009-2012, salvatore sanfilippo <antirez at gmail dot com>
* all rights reserved.
*
* redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* * redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
* * redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * neither the name of redis nor the names of its contributors may be used
* to endorse or promote products derived from this software without
* specific prior written permission.
*
* this software is provided by the copyright holders and contributors "as is"
* and any express or implied warranties, including, but not limited to, the
* implied warranties of merchantability and fitness for a particular purpose
* are disclaimed. in no event shall the copyright owner or contributors be
* liable for any direct, indirect, incidental, special, exemplary, or
* consequential damages (including, but not limited to, procurement of
* substitute goods or services; loss of use, data, or profits; or business
* interruption) however caused and on any theory of liability, whether in
* contract, strict liability, or tort (including negligence or otherwise)
* arising in any way out of the use of this software, even if advised of the
* possibility of such damage.
*/
#ifndef __config_h
#define __config_h
#ifdef __apple__
#include <availabilitymacros.h>
#endif
#ifdef __linux__
#include <linux/version.h>
#include <features.h>
#endif
/* define redis_fstat to fstat or fstat64() */
#if defined(__apple__) && !defined(mac_os_x_version_10_6)
#define redis_fstat fstat64
#define redis_stat stat64
#else
#define redis_fstat fstat
#define redis_stat stat
#endif
/* test for proc filesystem */
#ifdef __linux__
#define have_proc_stat 1
#define have_proc_maps 1
#define have_proc_smaps 1
#define have_proc_somaxconn 1
#endif
/* test for task_info() */
#if defined(__apple__)
#define have_taskinfo 1
#endif
/* test for backtrace() */
#if defined(__apple__) || (defined(__linux__) && defined(__glibc__)) || \
defined(__freebsd__) || (defined(__openbsd__) && defined(use_backtrace))\
|| defined(__dragonfly__)
#define have_backtrace 1
#endif
/* msg_nosignal. */
#ifdef __linux__
#define have_msg_nosignal 1
#endif
/* test for polling api */
#ifdef __linux__
#define have_epoll 1
#endif
#if (defined(__apple__) && defined(mac_os_x_version_10_6)) || defined(__freebsd__) || defined(__openbsd__) || defined (__netbsd__)
#define have_kqueue 1
#endif
#ifdef __sun
#include <sys/feature_tests.h>
#ifdef _dtrace_version
#define have_evport 1
#endif
#endif
/* define redis_fsync to fdatasync() in linux and fsync() for all the rest */
#ifdef __linux__
#define redis_fsync fdatasync
#else
#define redis_fsync fsync
#endif
/* define rdb_fsync_range to sync_file_range() on linux, otherwise we use
* the plain fsync() call. */
#ifdef __linux__
#if defined(__glibc__) && defined(__glibc_prereq)
#if (linux_version_code >= 0x020611 && __glibc_prereq(2, 6))
#define have_sync_file_range 1
#endif
#else
#if (linux_version_code >= 0x020611)
#define have_sync_file_range 1
#endif
#endif
#endif
#ifdef have_sync_file_range
#define rdb_fsync_range(fd,off,size) sync_file_range(fd,off,size,sync_file_range_wait_before|sync_file_range_write)
#else
#define rdb_fsync_range(fd,off,size) fsync(fd)
#endif
/* check if we can use setproctitle().
* bsd systems have support for it, we provide an implementation for
* linux and osx. */
#if (defined __netbsd__ || defined __freebsd__ || defined __openbsd__)
#define use_setproctitle
#endif
#if ((defined __linux && defined(__glibc__)) || defined __apple__)
#define use_setproctitle
#define init_setproctitle_replacement
void spt_init(int argc, char *argv[]);
void setproctitle(const char *fmt, ...);
#endif
/* byte ordering detection */
#include <sys/types.h> /* this will likely define byte_order */
#ifndef byte_order
#if (bsd >= 199103)
# include <machine/endian.h>
#else
#if defined(linux) || defined(__linux__)
# include <endian.h>
#else
#define little_endian 1234 /* least-significant byte first (vax, pc) */
#define big_endian 4321 /* most-significant byte first (ibm, net) */
#define pdp_endian 3412 /* lsb first in word, msw first in long (pdp)*/
#if defined(__i386__) || defined(__x86_64__) || defined(__amd64__) || \
defined(vax) || defined(ns32000) || defined(sun386) || \
defined(mipsel) || defined(_mipsel) || defined(bit_zero_on_right) || \
defined(__alpha__) || defined(__alpha)
#define byte_order little_endian
#endif
#if defined(sel) || defined(pyr) || defined(mc68000) || defined(sparc) || \
defined(is68k) || defined(tahoe) || defined(ibm032) || defined(ibm370) || \
defined(mipseb) || defined(_mipseb) || defined(_ibmr2) || defined(dgux) ||\
defined(apollo) || defined(__convex__) || defined(_cray) || \
defined(__hppa) || defined(__hp9000) || \
defined(__hp9000s300) || defined(__hp9000s700) || \
defined (bit_zero_on_left) || defined(m68k) || defined(__sparc)
#define byte_order big_endian
#endif
#endif /* linux */
#endif /* bsd */
#endif /* byte_order */
/* sometimes after including an os-specific header that defines the
* endianess we end with __byte_order but not with byte_order that is what
* the redis code uses. in this case let's define everything without the
* underscores. */
#ifndef byte_order
#ifdef __byte_order
#if defined(__little_endian) && defined(__big_endian)
#ifndef little_endian
#define little_endian __little_endian
#endif
#ifndef big_endian
#define big_endian __big_endian
#endif
#if (__byte_order == __little_endian)
#define byte_order little_endian
#else
#define byte_order big_endian
#endif
#endif
#endif
#endif
#if !defined(byte_order) || \
(byte_order != big_endian && byte_order != little_endian)
/* you must determine what the correct bit order is for
* your compiler - the next line is an intentional error
* which will force your compiles to bomb until you fix
* the above macros.
*/
#error "undefined or invalid byte_order"
#endif
#if (__i386 || __amd64 || __powerpc__) && __gnuc__
#define gnuc_version (__gnuc__ * 10000 + __gnuc_minor__ * 100 + __gnuc_patchlevel__)
#if defined(__clang__)
#define have_atomic
#endif
#if (defined(__glibc__) && defined(__glibc_prereq))
#if (gnuc_version >= 40100 && __glibc_prereq(2, 6))
#define have_atomic
#endif
#endif
#endif
/* make sure we can test for arm just checking for __arm__, since sometimes
* __arm is defined but __arm__ is not. */
#if defined(__arm) && !defined(__arm__)
#define __arm__
#endif
#if defined (__aarch64__) && !defined(__arm64__)
#define __arm64__
#endif
/* make sure we can test for sparc just checking for __sparc__. */
#if defined(__sparc) && !defined(__sparc__)
#define __sparc__
#endif
#if defined(__sparc__) || defined(__arm__)
#define use_aligned_access
#endif
#endif
从宏定义中可以看出来, redis 依赖 linux unix 类型的操作系统. 如果当时 redis 一心只为
linux 服务, 预计开发和维护的心智负担会小很多(纯属意淫). 那开始扯皮吧.
2.1' 宏排版差评, 写起来辣眼睛
我们以 byte_order 为例子, 不放给其排排版, 对对齐, 方便肉眼阅读.
// sometimes after including an os-specific header that defines the // endianess we end with __byte_order but not with byte_order that is what // the redis code uses. in this case let's define everything without the // underscores. #ifndef byte_order # ifdef __byte_order # if defined __little_endian && defined __big_endian # ifndef little_endian # define little_engian __little_endian # endif # ifndef big_endian # define big_engian __big_engian # endif # if __byte_order == __little_engian # define byte_order little_engian # else # define byte_order big_engian # endif # endif # endif #endif
大家看看这样, 是不是清爽了很多.
而对于 config.h 我们不继续展开 config.c 了, 因为项目运行起点的就是 config. 这要再深入下去
基本就 redis all in 了. 附赠聊聊边角料 setproctitle 设置进程标题的话题.
#if (defined __linux && defined __glibc__) || (defined __apple__) #define use_setproctitle #define init_setproctitle_replacement extern void spt_init(int argc, char * argv[]); extern void setproctitle(const char * fmt, ...); #endif
3. redis setproctitle.c 分析
1 /* ==========================================================================
2 * setproctitle.c - linux/darwin setproctitle.
3 * --------------------------------------------------------------------------
4 * copyright (c) 2010 william ahern
5 * copyright (c) 2013 salvatore sanfilippo
6 * copyright (c) 2013 stam he
7 *
8 * permission is hereby granted, free of charge, to any person obtaining a
9 * copy of this software and associated documentation files (the
10 * "software"), to deal in the software without restriction, including
11 * without limitation the rights to use, copy, modify, merge, publish,
12 * distribute, sublicense, and/or sell copies of the software, and to permit
13 * persons to whom the software is furnished to do so, subject to the
14 * following conditions:
15 *
16 * the above copyright notice and this permission notice shall be included
17 * in all copies or substantial portions of the software.
18 *
19 * the software is provided "as is", without warranty of any kind, express
20 * or implied, including but not limited to the warranties of
21 * merchantability, fitness for a particular purpose and noninfringement. in
22 * no event shall the authors or copyright holders be liable for any claim,
23 * damages or other liability, whether in an action of contract, tort or
24 * otherwise, arising from, out of or in connection with the software or the
25 * use or other dealings in the software.
26 * ==========================================================================
27 */
28 #ifndef _gnu_source
29 #define _gnu_source
30 #endif
31
32 #include <stddef.h> /* null size_t */
33 #include <stdarg.h> /* va_list va_start va_end */
34 #include <stdlib.h> /* malloc(3) setenv(3) clearenv(3) setproctitle(3) getprogname(3) */
35 #include <stdio.h> /* vsnprintf(3) snprintf(3) */
36
37 #include <string.h> /* strlen(3) strchr(3) strdup(3) memset(3) memcpy(3) */
38
39 #include <errno.h> /* errno program_invocation_name program_invocation_short_name */
40
41 #if !defined(have_setproctitle)
42 #if (defined __netbsd__ || defined __freebsd__ || defined __openbsd__ || defined __dragonfly__)
43 #define have_setproctitle 1
44 #else
45 #define have_setproctitle 0
46 #endif
47 #endif
48
49
50 #if !have_setproctitle
51 #if (defined __linux || defined __apple__)
52
53 extern char **environ;
54
55 static struct {
56 /* original value */
57 const char *arg0;
58
59 /* title space available */
60 char *base, *end;
61
62 /* pointer to original nul character within base */
63 char *nul;
64
65 _bool reset;
66 int error;
67 } spt;
68
69
70 #ifndef spt_min
71 #define spt_min(a, b) (((a) < (b))? (a) : (b))
72 #endif
73
74 static inline size_t spt_min(size_t a, size_t b) {
75 return spt_min(a, b);
76 } /* spt_min() */
77
78
79 /*
80 * for discussion on the portability of the various methods, see
81 * http://lists.freebsd.org/pipermail/freebsd-stable/2008-june/043136.html
82 */
83 static int spt_clearenv(void) {
84 #if __glibc__
85 clearenv();
86
87 return 0;
88 #else
89 extern char **environ;
90 static char **tmp;
91
92 if (!(tmp = malloc(sizeof *tmp)))
93 return errno;
94
95 tmp[0] = null;
96 environ = tmp;
97
98 return 0;
99 #endif
100 } /* spt_clearenv() */
101
102
103 static int spt_copyenv(char *oldenv[]) {
104 extern char **environ;
105 char *eq;
106 int i, error;
107
108 if (environ != oldenv)
109 return 0;
110
111 if ((error = spt_clearenv()))
112 goto error;
113
114 for (i = 0; oldenv[i]; i++) {
115 if (!(eq = strchr(oldenv[i], '=')))
116 continue;
117
118 *eq = '\0';
119 error = (0 != setenv(oldenv[i], eq + 1, 1))? errno : 0;
120 *eq = '=';
121
122 if (error)
123 goto error;
124 }
125
126 return 0;
127 error:
128 environ = oldenv;
129
130 return error;
131 } /* spt_copyenv() */
132
133
134 static int spt_copyargs(int argc, char *argv[]) {
135 char *tmp;
136 int i;
137
138 for (i = 1; i < argc || (i >= argc && argv[i]); i++) {
139 if (!argv[i])
140 continue;
141
142 if (!(tmp = strdup(argv[i])))
143 return errno;
144
145 argv[i] = tmp;
146 }
147
148 return 0;
149 } /* spt_copyargs() */
150
151
152 void spt_init(int argc, char *argv[]) {
153 char **envp = environ;
154 char *base, *end, *nul, *tmp;
155 int i, error;
156
157 if (!(base = argv[0]))
158 return;
159
160 nul = &base[strlen(base)];
161 end = nul + 1;
162
163 for (i = 0; i < argc || (i >= argc && argv[i]); i++) {
164 if (!argv[i] || argv[i] < end)
165 continue;
166
167 end = argv[i] + strlen(argv[i]) + 1;
168 }
169
170 for (i = 0; envp[i]; i++) {
171 if (envp[i] < end)
172 continue;
173
174 end = envp[i] + strlen(envp[i]) + 1;
175 }
176
177 if (!(spt.arg0 = strdup(argv[0])))
178 goto syerr;
179
180 #if __glibc__
181 if (!(tmp = strdup(program_invocation_name)))
182 goto syerr;
183
184 program_invocation_name = tmp;
185
186 if (!(tmp = strdup(program_invocation_short_name)))
187 goto syerr;
188
189 program_invocation_short_name = tmp;
190 #elif __apple__
191 if (!(tmp = strdup(getprogname())))
192 goto syerr;
193
194 setprogname(tmp);
195 #endif
196
197
198 if ((error = spt_copyenv(envp)))
199 goto error;
200
201 if ((error = spt_copyargs(argc, argv)))
202 goto error;
203
204 spt.nul = nul;
205 spt.base = base;
206 spt.end = end;
207
208 return;
209 syerr:
210 error = errno;
211 error:
212 spt.error = error;
213 } /* spt_init() */
214
215
216 #ifndef spt_maxtitle
217 #define spt_maxtitle 255
218 #endif
219
220 void setproctitle(const char *fmt, ...) {
221 char buf[spt_maxtitle + 1]; /* use buffer in case argv[0] is passed */
222 va_list ap;
223 char *nul;
224 int len, error;
225
226 if (!spt.base)
227 return;
228
229 if (fmt) {
230 va_start(ap, fmt);
231 len = vsnprintf(buf, sizeof buf, fmt, ap);
232 va_end(ap);
233 } else {
234 len = snprintf(buf, sizeof buf, "%s", spt.arg0);
235 }
236
237 if (len <= 0)
238 { error = errno; goto error; }
239
240 if (!spt.reset) {
241 memset(spt.base, 0, spt.end - spt.base);
242 spt.reset = 1;
243 } else {
244 memset(spt.base, 0, spt_min(sizeof buf, spt.end - spt.base));
245 }
246
247 len = spt_min(len, spt_min(sizeof buf, spt.end - spt.base) - 1);
248 memcpy(spt.base, buf, len);
249 nul = &spt.base[len];
250
251 if (nul < spt.nul) {
252 *spt.nul = '.';
253 } else if (nul == spt.nul && &nul[1] < spt.end) {
254 *spt.nul = ' ';
255 *++nul = '\0';
256 }
257
258 return;
259 error:
260 spt.error = error;
261 } /* setproctitle() */
262
263
264 #endif /* __linux || __apple__ */
265 #endif /* !have_setproctitle */
3.1' spt_clearenv -> spt_copyenv -> setenv -> goto error -> environ = oldenv memory leak
感兴趣的朋友可以一块交流. 想了解更多也可以参阅我和这个博主之间的交互(设置进程名称)
4. redis atomicvar.h 分析
1 /* this file implements atomic counters using __atomic or __sync macros if
2 * available, otherwise synchronizing different threads using a mutex.
3 *
4 * the exported interface is composed of three macros:
5 *
6 * atomicincr(var,count) -- increment the atomic counter
7 * atomicgetincr(var,oldvalue_var,count) -- get and increment the atomic counter
8 * atomicdecr(var,count) -- decrement the atomic counter
9 * atomicget(var,dstvar) -- fetch the atomic counter value
10 * atomicset(var,value) -- set the atomic counter value
11 *
12 * the variable 'var' should also have a declared mutex with the same
13 * name and the "_mutex" postfix, for instance:
14 *
15 * long myvar;
16 * pthread_mutex_t myvar_mutex;
17 * atomicset(myvar,12345);
18 *
19 * if atomic primitives are available (tested in config.h) the mutex
20 * is not used.
21 *
22 * never use return value from the macros, instead use the atomicgetincr()
23 * if you need to get the current value and increment it atomically, like
24 * in the followign example:
25 *
26 * long oldvalue;
27 * atomicgetincr(myvar,oldvalue,1);
28 * dosomethingwith(oldvalue);
29 *
30 * ----------------------------------------------------------------------------
31 *
32 * copyright (c) 2015, salvatore sanfilippo <antirez at gmail dot com>
33 * all rights reserved.
34 *
35 * redistribution and use in source and binary forms, with or without
36 * modification, are permitted provided that the following conditions are met:
37 *
38 * * redistributions of source code must retain the above copyright notice,
39 * this list of conditions and the following disclaimer.
40 * * redistributions in binary form must reproduce the above copyright
41 * notice, this list of conditions and the following disclaimer in the
42 * documentation and/or other materials provided with the distribution.
43 * * neither the name of redis nor the names of its contributors may be used
44 * to endorse or promote products derived from this software without
45 * specific prior written permission.
46 *
47 * this software is provided by the copyright holders and contributors "as is"
48 * and any express or implied warranties, including, but not limited to, the
49 * implied warranties of merchantability and fitness for a particular purpose
50 * are disclaimed. in no event shall the copyright owner or contributors be
51 * liable for any direct, indirect, incidental, special, exemplary, or
52 * consequential damages (including, but not limited to, procurement of
53 * substitute goods or services; loss of use, data, or profits; or business
54 * interruption) however caused and on any theory of liability, whether in
55 * contract, strict liability, or tort (including negligence or otherwise)
56 * arising in any way out of the use of this software, even if advised of the
57 * possibility of such damage.
58 */
59
60 #include <pthread.h>
61
62 #ifndef __atomic_var_h
63 #define __atomic_var_h
64
65 /* to test redis with helgrind (a valgrind tool) it is useful to define
66 * the following macro, so that __sync macros are used: those can be detected
67 * by helgrind (even if they are less efficient) so that no false positive
68 * is reported. */
69 // #define __atomic_var_force_sync_macros
70
71 #if !defined(__atomic_var_force_sync_macros) && defined(__atomic_relaxed) && !defined(__sun) && (!defined(__clang__) || !defined(__apple__) || __apple_build_version__ > 4210057)
72 /* implementation using __atomic macros. */
73
74 #define atomicincr(var,count) __atomic_add_fetch(&var,(count),__atomic_relaxed)
75 #define atomicgetincr(var,oldvalue_var,count) do { \
76 oldvalue_var = __atomic_fetch_add(&var,(count),__atomic_relaxed); \
77 } while(0)
78 #define atomicdecr(var,count) __atomic_sub_fetch(&var,(count),__atomic_relaxed)
79 #define atomicget(var,dstvar) do { \
80 dstvar = __atomic_load_n(&var,__atomic_relaxed); \
81 } while(0)
82 #define atomicset(var,value) __atomic_store_n(&var,value,__atomic_relaxed)
83 #define redis_atomic_api "atomic-builtin"
84
85 #elif defined(have_atomic)
86 /* implementation using __sync macros. */
87
88 #define atomicincr(var,count) __sync_add_and_fetch(&var,(count))
89 #define atomicgetincr(var,oldvalue_var,count) do { \
90 oldvalue_var = __sync_fetch_and_add(&var,(count)); \
91 } while(0)
92 #define atomicdecr(var,count) __sync_sub_and_fetch(&var,(count))
93 #define atomicget(var,dstvar) do { \
94 dstvar = __sync_sub_and_fetch(&var,0); \
95 } while(0)
96 #define atomicset(var,value) do { \
97 while(!__sync_bool_compare_and_swap(&var,var,value)); \
98 } while(0)
99 #define redis_atomic_api "sync-builtin"
100
101 #else
102 /* implementation using pthread mutex. */
103
104 #define atomicincr(var,count) do { \
105 pthread_mutex_lock(&var ## _mutex); \
106 var += (count); \
107 pthread_mutex_unlock(&var ## _mutex); \
108 } while(0)
109 #define atomicgetincr(var,oldvalue_var,count) do { \
110 pthread_mutex_lock(&var ## _mutex); \
111 oldvalue_var = var; \
112 var += (count); \
113 pthread_mutex_unlock(&var ## _mutex); \
114 } while(0)
115 #define atomicdecr(var,count) do { \
116 pthread_mutex_lock(&var ## _mutex); \
117 var -= (count); \
118 pthread_mutex_unlock(&var ## _mutex); \
119 } while(0)
120 #define atomicget(var,dstvar) do { \
121 pthread_mutex_lock(&var ## _mutex); \
122 dstvar = var; \
123 pthread_mutex_unlock(&var ## _mutex); \
124 } while(0)
125 #define atomicset(var,value) do { \
126 pthread_mutex_lock(&var ## _mutex); \
127 var = value; \
128 pthread_mutex_unlock(&var ## _mutex); \
129 } while(0)
130 #define redis_atomic_api "pthread-mutex"
131
132 #endif
133 #endif /* __atomic_var_h */
atomicvar.h 原子库操作封装思路有三种, c11 stdatomic.h 和 gcc sync 操作, 还有 posix pthread.h . 不过使用
pthread.h 封装的"原子操作", 不是那么通用, 因为和业务强绑定. 只能在 redis 项目下用. 原因是它依赖事先定义
好变量 var##_mutex
pthread_mutex_lock(&var ## _mutex); \
可以找到例子, 例如 src/lazyfree.c 中有段代码如下
#include "server.h" #include "bio.h" #include "atomicvar.h" #include "cluster.h" static size_t lazyfree_objects = 0; pthread_mutex_t lazyfree_objects_mutex = pthread_mutex_initializer;
事先定义 lazyfree_objects 和 lazyfree_objects_mutex 才能使用 pthread 封装的原子操作宏. 额外的优化
可以通过 __sync_lock_test_and_set 替代 while __sync_bool_compare_and_swap 费力操作
5. redis zmalloc 分析
1 /* zmalloc - total amount of allocated memory aware version of malloc()
2 *
3 * copyright (c) 2009-2010, salvatore sanfilippo <antirez at gmail dot com>
4 * all rights reserved.
5 *
6 * redistribution and use in source and binary forms, with or without
7 * modification, are permitted provided that the following conditions are met:
8 *
9 * * redistributions of source code must retain the above copyright notice,
10 * this list of conditions and the following disclaimer.
11 * * redistributions in binary form must reproduce the above copyright
12 * notice, this list of conditions and the following disclaimer in the
13 * documentation and/or other materials provided with the distribution.
14 * * neither the name of redis nor the names of its contributors may be used
15 * to endorse or promote products derived from this software without
16 * specific prior written permission.
17 *
18 * this software is provided by the copyright holders and contributors "as is"
19 * and any express or implied warranties, including, but not limited to, the
20 * implied warranties of merchantability and fitness for a particular purpose
21 * are disclaimed. in no event shall the copyright owner or contributors be
22 * liable for any direct, indirect, incidental, special, exemplary, or
23 * consequential damages (including, but not limited to, procurement of
24 * substitute goods or services; loss of use, data, or profits; or business
25 * interruption) however caused and on any theory of liability, whether in
26 * contract, strict liability, or tort (including negligence or otherwise)
27 * arising in any way out of the use of this software, even if advised of the
28 * possibility of such damage.
29 */
30
31 #ifndef __zmalloc_h
32 #define __zmalloc_h
33
34 /* double expansion needed for stringification of macro values. */
35 #define __xstr(s) __str(s)
36 #define __str(s) #s
37
38 #if defined(use_tcmalloc)
39 #define zmalloc_lib ("tcmalloc-" __xstr(tc_version_major) "." __xstr(tc_version_minor))
40 #include <google/tcmalloc.h>
41 #if (tc_version_major == 1 && tc_version_minor >= 6) || (tc_version_major > 1)
42 #define have_malloc_size 1
43 #define zmalloc_size(p) tc_malloc_size(p)
44 #else
45 #error "newer version of tcmalloc required"
46 #endif
47
48 #elif defined(use_jemalloc)
49 #define zmalloc_lib ("jemalloc-" __xstr(jemalloc_version_major) "." __xstr(jemalloc_version_minor) "." __xstr(jemalloc_version_bugfix))
50 #include <jemalloc/jemalloc.h>
51 #if (jemalloc_version_major == 2 && jemalloc_version_minor >= 1) || (jemalloc_version_major > 2)
52 #define have_malloc_size 1
53 #define zmalloc_size(p) je_malloc_usable_size(p)
54 #else
55 #error "newer version of jemalloc required"
56 #endif
57
58 #elif defined(__apple__)
59 #include <malloc/malloc.h>
60 #define have_malloc_size 1
61 #define zmalloc_size(p) malloc_size(p)
62 #endif
63
64 #ifndef zmalloc_lib
65 #define zmalloc_lib "libc"
66 #ifdef __glibc__
67 #include <malloc.h>
68 #define have_malloc_size 1
69 #define zmalloc_size(p) malloc_usable_size(p)
70 #endif
71 #endif
72
73 /* we can enable the redis defrag capabilities only if we are using jemalloc
74 * and the version used is our special version modified for redis having
75 * the ability to return per-allocation fragmentation hints. */
76 #if defined(use_jemalloc) && defined(jemalloc_frag_hint)
77 #define have_defrag
78 #endif
79
80 void *zmalloc(size_t size);
81 void *zcalloc(size_t size);
82 void *zrealloc(void *ptr, size_t size);
83 void zfree(void *ptr);
84 char *zstrdup(const char *s);
85 size_t zmalloc_used_memory(void);
86 void zmalloc_set_oom_handler(void (*oom_handler)(size_t));
87 size_t zmalloc_get_rss(void);
88 int zmalloc_get_allocator_info(size_t *allocated, size_t *active, size_t *resident);
89 void set_jemalloc_bg_thread(int enable);
90 int jemalloc_purge();
91 size_t zmalloc_get_private_dirty(long pid);
92 size_t zmalloc_get_smap_bytes_by_field(char *field, long pid);
93 size_t zmalloc_get_memory_size(void);
94 void zlibc_free(void *ptr);
95
96 #ifdef have_defrag
97 void zfree_no_tcache(void *ptr);
98 void *zmalloc_no_tcache(size_t size);
99 #endif
100
101 #ifndef have_malloc_size
102 size_t zmalloc_size(void *ptr);
103 size_t zmalloc_usable(void *ptr);
104 #else
105 #define zmalloc_usable(p) zmalloc_size(p)
106 #endif
107
108 #ifdef redis_test
109 int zmalloc_test(int argc, char **argv);
110 #endif
111
112 #endif /* __zmalloc_h */
内存模块支持外部库丰富, 自然写的就有点啰嗦. 我们这里有个诀窍, 我们假定只使用 use_jemalloc ,
然后代码一路走下去, 是不是吼方便呢.
1 /* zmalloc - total amount of allocated memory aware version of malloc()
2 *
3 * copyright (c) 2009-2010, salvatore sanfilippo <antirez at gmail dot com>
4 * all rights reserved.
5 *
6 * redistribution and use in source and binary forms, with or without
7 * modification, are permitted provided that the following conditions are met:
8 *
9 * * redistributions of source code must retain the above copyright notice,
10 * this list of conditions and the following disclaimer.
11 * * redistributions in binary form must reproduce the above copyright
12 * notice, this list of conditions and the following disclaimer in the
13 * documentation and/or other materials provided with the distribution.
14 * * neither the name of redis nor the names of its contributors may be used
15 * to endorse or promote products derived from this software without
16 * specific prior written permission.
17 *
18 * this software is provided by the copyright holders and contributors "as is"
19 * and any express or implied warranties, including, but not limited to, the
20 * implied warranties of merchantability and fitness for a particular purpose
21 * are disclaimed. in no event shall the copyright owner or contributors be
22 * liable for any direct, indirect, incidental, special, exemplary, or
23 * consequential damages (including, but not limited to, procurement of
24 * substitute goods or services; loss of use, data, or profits; or business
25 * interruption) however caused and on any theory of liability, whether in
26 * contract, strict liability, or tort (including negligence or otherwise)
27 * arising in any way out of the use of this software, even if advised of the
28 * possibility of such damage.
29 */
30
31 #include <stdio.h>
32 #include <stdlib.h>
33 #include <stdint.h>
34
35 /* this function provide us access to the original libc free(). this is useful
36 * for instance to free results obtained by backtrace_symbols(). we need
37 * to define this function before including zmalloc.h that may shadow the
38 * free implementation if we use jemalloc or another non standard allocator. */
39 void zlibc_free(void *ptr) {
40 free(ptr);
41 }
42
43 #include <string.h>
44 #include <pthread.h>
45 #include "config.h"
46 #include "zmalloc.h"
47 #include "atomicvar.h"
48
49 #ifdef have_malloc_size
50 #define prefix_size (0)
51 #else
52 #if defined(__sun) || defined(__sparc) || defined(__sparc__)
53 #define prefix_size (sizeof(long long))
54 #else
55 #define prefix_size (sizeof(size_t))
56 #endif
57 #endif
58
59 /* explicitly override malloc/free etc when using tcmalloc. */
60 #if defined(use_tcmalloc)
61 #define malloc(size) tc_malloc(size)
62 #define calloc(count,size) tc_calloc(count,size)
63 #define realloc(ptr,size) tc_realloc(ptr,size)
64 #define free(ptr) tc_free(ptr)
65 #elif defined(use_jemalloc)
66 #define malloc(size) je_malloc(size)
67 #define calloc(count,size) je_calloc(count,size)
68 #define realloc(ptr,size) je_realloc(ptr,size)
69 #define free(ptr) je_free(ptr)
70 #define mallocx(size,flags) je_mallocx(size,flags)
71 #define dallocx(ptr,flags) je_dallocx(ptr,flags)
72 #endif
73
74 #define update_zmalloc_stat_alloc(__n) do { \
75 size_t _n = (__n); \
76 if (_n&(sizeof(long)-1)) _n += sizeof(long)-(_n&(sizeof(long)-1)); \
77 atomicincr(used_memory,__n); \
78 } while(0)
79
80 #define update_zmalloc_stat_free(__n) do { \
81 size_t _n = (__n); \
82 if (_n&(sizeof(long)-1)) _n += sizeof(long)-(_n&(sizeof(long)-1)); \
83 atomicdecr(used_memory,__n); \
84 } while(0)
85
86 static size_t used_memory = 0;
87 pthread_mutex_t used_memory_mutex = pthread_mutex_initializer;
88
89 static void zmalloc_default_oom(size_t size) {
90 fprintf(stderr, "zmalloc: out of memory trying to allocate %zu bytes\n",
91 size);
92 fflush(stderr);
93 abort();
94 }
95
96 static void (*zmalloc_oom_handler)(size_t) = zmalloc_default_oom;
97
98 void *zmalloc(size_t size) {
99 void *ptr = malloc(size+prefix_size);
100
101 if (!ptr) zmalloc_oom_handler(size);
102 #ifdef have_malloc_size
103 update_zmalloc_stat_alloc(zmalloc_size(ptr));
104 return ptr;
105 #else
106 *((size_t*)ptr) = size;
107 update_zmalloc_stat_alloc(size+prefix_size);
108 return (char*)ptr+prefix_size;
109 #endif
110 }
111
112 /* allocation and free functions that bypass the thread cache
113 * and go straight to the allocator arena bins.
114 * currently implemented only for jemalloc. used for online defragmentation. */
115 #ifdef have_defrag
116 void *zmalloc_no_tcache(size_t size) {
117 void *ptr = mallocx(size+prefix_size, mallocx_tcache_none);
118 if (!ptr) zmalloc_oom_handler(size);
119 update_zmalloc_stat_alloc(zmalloc_size(ptr));
120 return ptr;
121 }
122
123 void zfree_no_tcache(void *ptr) {
124 if (ptr == null) return;
125 update_zmalloc_stat_free(zmalloc_size(ptr));
126 dallocx(ptr, mallocx_tcache_none);
127 }
128 #endif
129
130 void *zcalloc(size_t size) {
131 void *ptr = calloc(1, size+prefix_size);
132
133 if (!ptr) zmalloc_oom_handler(size);
134 #ifdef have_malloc_size
135 update_zmalloc_stat_alloc(zmalloc_size(ptr));
136 return ptr;
137 #else
138 *((size_t*)ptr) = size;
139 update_zmalloc_stat_alloc(size+prefix_size);
140 return (char*)ptr+prefix_size;
141 #endif
142 }
143
144 void *zrealloc(void *ptr, size_t size) {
145 #ifndef have_malloc_size
146 void *realptr;
147 #endif
148 size_t oldsize;
149 void *newptr;
150
151 if (size == 0 && ptr != null) {
152 zfree(ptr);
153 return null;
154 }
155 if (ptr == null) return zmalloc(size);
156 #ifdef have_malloc_size
157 oldsize = zmalloc_size(ptr);
158 newptr = realloc(ptr,size);
159 if (!newptr) zmalloc_oom_handler(size);
160
161 update_zmalloc_stat_free(oldsize);
162 update_zmalloc_stat_alloc(zmalloc_size(newptr));
163 return newptr;
164 #else
165 realptr = (char*)ptr-prefix_size;
166 oldsize = *((size_t*)realptr);
167 newptr = realloc(realptr,size+prefix_size);
168 if (!newptr) zmalloc_oom_handler(size);
169
170 *((size_t*)newptr) = size;
171 update_zmalloc_stat_free(oldsize+prefix_size);
172 update_zmalloc_stat_alloc(size+prefix_size);
173 return (char*)newptr+prefix_size;
174 #endif
175 }
176
177 /* provide zmalloc_size() for systems where this function is not provided by
178 * malloc itself, given that in that case we store a header with this
179 * information as the first bytes of every allocation. */
180 #ifndef have_malloc_size
181 size_t zmalloc_size(void *ptr) {
182 void *realptr = (char*)ptr-prefix_size;
183 size_t size = *((size_t*)realptr);
184 /* assume at least that all the allocations are padded at sizeof(long) by
185 * the underlying allocator. */
186 if (size&(sizeof(long)-1)) size += sizeof(long)-(size&(sizeof(long)-1));
187 return size+prefix_size;
188 }
189 size_t zmalloc_usable(void *ptr) {
190 return zmalloc_size(ptr)-prefix_size;
191 }
192 #endif
193
194 void zfree(void *ptr) {
195 #ifndef have_malloc_size
196 void *realptr;
197 size_t oldsize;
198 #endif
199
200 if (ptr == null) return;
201 #ifdef have_malloc_size
202 update_zmalloc_stat_free(zmalloc_size(ptr));
203 free(ptr);
204 #else
205 realptr = (char*)ptr-prefix_size;
206 oldsize = *((size_t*)realptr);
207 update_zmalloc_stat_free(oldsize+prefix_size);
208 free(realptr);
209 #endif
210 }
211
212 char *zstrdup(const char *s) {
213 size_t l = strlen(s)+1;
214 char *p = zmalloc(l);
215
216 memcpy(p,s,l);
217 return p;
218 }
219
220 size_t zmalloc_used_memory(void) {
221 size_t um;
222 atomicget(used_memory,um);
223 return um;
224 }
225
226 void zmalloc_set_oom_handler(void (*oom_handler)(size_t)) {
227 zmalloc_oom_handler = oom_handler;
228 }
229
230 /* get the rss information in an os-specific way.
231 *
232 * warning: the function zmalloc_get_rss() is not designed to be fast
233 * and may not be called in the busy loops where redis tries to release
234 * memory expiring or swapping out objects.
235 *
236 * for this kind of "fast rss reporting" usages use instead the
237 * function redisestimaterss() that is a much faster (and less precise)
238 * version of the function. */
239
240 #if defined(have_proc_stat)
241 #include <unistd.h>
242 #include <sys/types.h>
243 #include <sys/stat.h>
244 #include <fcntl.h>
245
246 size_t zmalloc_get_rss(void) {
247 int page = sysconf(_sc_pagesize);
248 size_t rss;
249 char buf[4096];
250 char filename[256];
251 int fd, count;
252 char *p, *x;
253
254 snprintf(filename,256,"/proc/%d/stat",getpid());
255 if ((fd = open(filename,o_rdonly)) == -1) return 0;
256 if (read(fd,buf,4096) <= 0) {
257 close(fd);
258 return 0;
259 }
260 close(fd);
261
262 p = buf;
263 count = 23; /* rss is the 24th field in /proc/<pid>/stat */
264 while(p && count--) {
265 p = strchr(p,' ');
266 if (p) p++;
267 }
268 if (!p) return 0;
269 x = strchr(p,' ');
270 if (!x) return 0;
271 *x = '\0';
272
273 rss = strtoll(p,null,10);
274 rss *= page;
275 return rss;
276 }
277 #elif defined(have_taskinfo)
278 #include <unistd.h>
279 #include <stdio.h>
280 #include <stdlib.h>
281 #include <sys/types.h>
282 #include <sys/sysctl.h>
283 #include <mach/task.h>
284 #include <mach/mach_init.h>
285
286 size_t zmalloc_get_rss(void) {
287 task_t task = mach_port_null;
288 struct task_basic_info t_info;
289 mach_msg_type_number_t t_info_count = task_basic_info_count;
290
291 if (task_for_pid(current_task(), getpid(), &task) != kern_success)
292 return 0;
293 task_info(task, task_basic_info, (task_info_t)&t_info, &t_info_count);
294
295 return t_info.resident_size;
296 }
297 #elif defined(__freebsd__)
298 #include <sys/types.h>
299 #include <sys/sysctl.h>
300 #include <sys/user.h>
301 #include <unistd.h>
302
303 size_t zmalloc_get_rss(void) {
304 struct kinfo_proc info;
305 size_t infolen = sizeof(info);
306 int mib[4];
307 mib[0] = ctl_kern;
308 mib[1] = kern_proc;
309 mib[2] = kern_proc_pid;
310 mib[3] = getpid();
311
312 if (sysctl(mib, 4, &info, &infolen, null, 0) == 0)
313 return (size_t)info.ki_rssize;
314
315 return 0l;
316 }
317 #else
318 size_t zmalloc_get_rss(void) {
319 /* if we can't get the rss in an os-specific way for this system just
320 * return the memory usage we estimated in zmalloc()..
321 *
322 * fragmentation will appear to be always 1 (no fragmentation)
323 * of course... */
324 return zmalloc_used_memory();
325 }
326 #endif
327
328 #if defined(use_jemalloc)
329
330 int zmalloc_get_allocator_info(size_t *allocated,
331 size_t *active,
332 size_t *resident) {
333 uint64_t epoch = 1;
334 size_t sz;
335 *allocated = *resident = *active = 0;
336 /* update the statistics cached by mallctl. */
337 sz = sizeof(epoch);
338 je_mallctl("epoch", &epoch, &sz, &epoch, sz);
339 sz = sizeof(size_t);
340 /* unlike rss, this does not include rss from shared libraries and other non
341 * heap mappings. */
342 je_mallctl("stats.resident", resident, &sz, null, 0);
343 /* unlike resident, this doesn't not include the pages jemalloc reserves
344 * for re-use (purge will clean that). */
345 je_mallctl("stats.active", active, &sz, null, 0);
346 /* unlike zmalloc_used_memory, this matches the stats.resident by taking
347 * into account all allocations done by this process (not only zmalloc). */
348 je_mallctl("stats.allocated", allocated, &sz, null, 0);
349 return 1;
350 }
351
352 void set_jemalloc_bg_thread(int enable) {
353 /* let jemalloc do purging asynchronously, required when there's no traffic
354 * after flushdb */
355 char val = !!enable;
356 je_mallctl("background_thread", null, 0, &val, 1);
357 }
358
359 int jemalloc_purge() {
360 /* return all unused (reserved) pages to the os */
361 char tmp[32];
362 unsigned narenas = 0;
363 size_t sz = sizeof(unsigned);
364 if (!je_mallctl("arenas.narenas", &narenas, &sz, null, 0)) {
365 sprintf(tmp, "arena.%d.purge", narenas);
366 if (!je_mallctl(tmp, null, 0, null, 0))
367 return 0;
368 }
369 return -1;
370 }
371
372 #else
373
374 int zmalloc_get_allocator_info(size_t *allocated,
375 size_t *active,
376 size_t *resident) {
377 *allocated = *resident = *active = 0;
378 return 1;
379 }
380
381 void set_jemalloc_bg_thread(int enable) {
382 ((void)(enable));
383 }
384
385 int jemalloc_purge() {
386 return 0;
387 }
388
389 #endif
390
391 /* get the sum of the specified field (converted form kb to bytes) in
392 * /proc/self/smaps. the field must be specified with trailing ":" as it
393 * apperas in the smaps output.
394 *
395 * if a pid is specified, the information is extracted for such a pid,
396 * otherwise if pid is -1 the information is reported is about the
397 * current process.
398 *
399 * example: zmalloc_get_smap_bytes_by_field("rss:",-1);
400 */
401 #if defined(have_proc_smaps)
402 size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
403 char line[1024];
404 size_t bytes = 0;
405 int flen = strlen(field);
406 file *fp;
407
408 if (pid == -1) {
409 fp = fopen("/proc/self/smaps","r");
410 } else {
411 char filename[128];
412 snprintf(filename,sizeof(filename),"/proc/%ld/smaps",pid);
413 fp = fopen(filename,"r");
414 }
415
416 if (!fp) return 0;
417 while(fgets(line,sizeof(line),fp) != null) {
418 if (strncmp(line,field,flen) == 0) {
419 char *p = strchr(line,'k');
420 if (p) {
421 *p = '\0';
422 bytes += strtol(line+flen,null,10) * 1024;
423 }
424 }
425 }
426 fclose(fp);
427 return bytes;
428 }
429 #else
430 size_t zmalloc_get_smap_bytes_by_field(char *field, long pid) {
431 ((void) field);
432 ((void) pid);
433 return 0;
434 }
435 #endif
436
437 size_t zmalloc_get_private_dirty(long pid) {
438 return zmalloc_get_smap_bytes_by_field("private_dirty:",pid);
439 }
440
441 /* returns the size of physical memory (ram) in bytes.
442 * it looks ugly, but this is the cleanest way to achieve cross platform results.
443 * cleaned up from:
444 *
445 * http://nadeausoftware.com/articles/2012/09/c_c_tip_how_get_physical_memory_size_system
446 *
447 * note that this function:
448 * 1) was released under the following cc attribution license:
449 * http://creativecommons.org/licenses/by/3.0/deed.en_us.
450 * 2) was originally implemented by david robert nadeau.
451 * 3) was modified for redis by matt stancliff.
452 * 4) this note exists in order to comply with the original license.
453 */
454 size_t zmalloc_get_memory_size(void) {
455 #if defined(__unix__) || defined(__unix) || defined(unix) || \
456 (defined(__apple__) && defined(__mach__))
457 #if defined(ctl_hw) && (defined(hw_memsize) || defined(hw_physmem64))
458 int mib[2];
459 mib[0] = ctl_hw;
460 #if defined(hw_memsize)
461 mib[1] = hw_memsize; /* osx. --------------------- */
462 #elif defined(hw_physmem64)
463 mib[1] = hw_physmem64; /* netbsd, openbsd. --------- */
464 #endif
465 int64_t size = 0; /* 64-bit */
466 size_t len = sizeof(size);
467 if (sysctl( mib, 2, &size, &len, null, 0) == 0)
468 return (size_t)size;
469 return 0l; /* failed? */
470
471 #elif defined(_sc_phys_pages) && defined(_sc_pagesize)
472 /* freebsd, linux, openbsd, and solaris. -------------------- */
473 return (size_t)sysconf(_sc_phys_pages) * (size_t)sysconf(_sc_pagesize);
474
475 #elif defined(ctl_hw) && (defined(hw_physmem) || defined(hw_realmem))
476 /* dragonfly bsd, freebsd, netbsd, openbsd, and osx. -------- */
477 int mib[2];
478 mib[0] = ctl_hw;
479 #if defined(hw_realmem)
480 mib[1] = hw_realmem; /* freebsd. ----------------- */
481 #elif defined(hw_physmem)
482 mib[1] = hw_physmem; /* others. ------------------ */
483 #endif
484 unsigned int size = 0; /* 32-bit */
485 size_t len = sizeof(size);
486 if (sysctl(mib, 2, &size, &len, null, 0) == 0)
487 return (size_t)size;
488 return 0l; /* failed? */
489 #else
490 return 0l; /* unknown method to get the data. */
491 #endif
492 #else
493 return 0l; /* unknown os. */
494 #endif
495 }
496
497 #ifdef redis_test
498 #define unused(x) ((void)(x))
499 int zmalloc_test(int argc, char **argv) {
500 void *ptr;
501
502 unused(argc);
503 unused(argv);
504 printf("initial used memory: %zu\n", zmalloc_used_memory());
505 ptr = zmalloc(123);
506 printf("allocated 123 bytes; used: %zu\n", zmalloc_used_memory());
507 ptr = zrealloc(ptr, 456);
508 printf("reallocated to 456 bytes; used: %zu\n", zmalloc_used_memory());
509 zfree(ptr);
510 printf("freed pointer; used: %zu\n", zmalloc_used_memory());
511 return 0;
512 }
513 #endif
这里提了一个优化, 减少代码的无效操作.
整体代码非常简单. prefix_size 用于记录申请内存块大小用的, 额外的, 写写, 自然就全明白了. 相比其它
模块多了 zmalloc_test 单元测试. 调用的地方可以看 src/server.c
void _serverpanic(const char *file, int line, const char *msg, ...) {
va_list ap;
va_start(ap,msg);
char fmtmsg[256];
vsnprintf(fmtmsg,sizeof(fmtmsg),msg,ap);
va_end(ap);
bugreportstart();
serverlog(ll_warning,"------------------------------------------------");
serverlog(ll_warning,"!!! software failure. press left mouse button to continue");
serverlog(ll_warning,"guru meditation: %s #%s:%d",fmtmsg,file,line);
#ifdef have_backtrace
serverlog(ll_warning,"(forcing sigsegv in order to print the stack trace)");
#endif
serverlog(ll_warning,"------------------------------------------------");
*((char*)-1) = 'x';
}
#define serverpanic(...) _serverpanic(__file__,__line__,__va_args__),_exit(1)
void redisoutofmemoryhandler(size_t allocation_size) {
serverlog(ll_warning,"out of memory allocating %zu bytes!",
allocation_size);
serverpanic("redis aborting for out of memory");
}
void redissetproctitle(char *title) {
#ifdef use_setproctitle
char *server_mode = "";
if (server.cluster_enabled) server_mode = " [cluster]";
else if (server.sentinel_mode) server_mode = " [sentinel]";
setproctitle("%s %s:%d%s",
title,
server.bindaddr_count ? server.bindaddr[0] : "*",
server.port ? server.port : server.tls_port,
server_mode);
#else
unused(title);
#endif
}
int main(int argc, char **argv) {
struct timeval tv;
int j;
#ifdef redis_test
if (argc == 3 && !strcasecmp(argv[1], "test")) {
if (!strcasecmp(argv[2], "ziplist")) {
return ziplisttest(argc, argv);
} else if (!strcasecmp(argv[2], "quicklist")) {
quicklisttest(argc, argv);
} else if (!strcasecmp(argv[2], "intset")) {
return intsettest(argc, argv);
} else if (!strcasecmp(argv[2], "zipmap")) {
return zipmaptest(argc, argv);
} else if (!strcasecmp(argv[2], "sha1test")) {
return sha1test(argc, argv);
} else if (!strcasecmp(argv[2], "util")) {
return utiltest(argc, argv);
} else if (!strcasecmp(argv[2], "endianconv")) {
return endianconvtest(argc, argv);
} else if (!strcasecmp(argv[2], "crc64")) {
return crc64test(argc, argv);
} else if (!strcasecmp(argv[2], "zmalloc")) {
return zmalloc_test(argc, argv);
}
return -1; /* test not found */
}
#endif
/* we need to initialize our libraries, and the server configuration. */
#ifdef init_setproctitle_replacement
spt_init(argc, argv);
#endif
setlocale(lc_collate,"");
tzset(); /* populates 'timezone' global. */
zmalloc_set_oom_handler(redisoutofmemoryhandler);
srand(time(null)^getpid());
gettimeofday(&tv,null);
...
..
.
通过这些预计你也看出点门道了吧. 关于 adlist 代码应该也了解七七八八了. 最后我们进入操练编译环节,
打通边角最后的一公里.
6. redis makefile 解析
加了注释, 方便大家自行逐个阅读 mkreleasehdr.sh 和 makefile, 感悟编译的不容易.
1 #!/bin/sh
2
3 # git log 第一个八位hash 值
4 git_sha1=`(git show-ref --head --hash=8 2>/dev/null || echo 00000000) | head -n1`
5
6 # 显示提交, 提交和工作树等之间的变化, 禁止外部差异驱动程序. 系统默认的 git diff
7 git_dirty=`git diff --no-ext-diff 2>/dev/null | wc -l`
8
9 # {网络节点上的主机名}-{时间戳}
10 build_id=`uname -n`"-"`date +%s`
11 # -n 判断 "$source_date_epoch" 是否不是空串, 不是为真
12 if [ -n "$source_date_epoch" ] then
13 # source_date_epoch=1574154953 -> date -u -d "@1574154953" +%s -> 1574154953 还是传入的时间戳
14 # source_date_epoch=src -> date -u -r "src" +%s -> 1574153469 显示指定文件的最后修改时间
15 # date -u +%s -> 1574155246 -> 输出或者设置协调的通用时间
16 build_id=$(date -u -d "@$source_date_epoch" +%s 2>/dev/null || date -u -r "$source_date_epoch" +%s 2>/dev/null || date -u +%s)
17 fi
18
19 # 测试 release.h 文件是否存在, 不存在就创建
20 test -f release.h || touch release.h
21 # 判断 release.h 是否已经创建 ok, ok 的话 exit 退出
22 (cat release.h | grep sha1 | grep $git_sha1) && \
23 (cat release.h | grep dirty | grep $git_dirty) && exit 0 # already up-to-date
24
25 # release.h 写入宏配置 redis_git_sha1 redis_git_dirty redis_build_id
26 echo "#define redis_git_sha1 \"$git_sha1\"" > release.h
27 echo "#define redis_git_dirty \"$git_dirty\"" >> release.h
28 echo "#define redis_build_id \"$build_id\"" >> release.h
29
30 # touch 更新 文件访问时间 atime 文件内容更改时间 mtime 文件状态改动时间 ctime
31 touch release.c # force recompile release.c
1 # redis makefile
2 # copyright (c) 2009 salvatore sanfilippo <antirez at gmail dot com>
3 # this file is released under the bsd license, see the copying file
4 #
5 # the makefile composes the final final_cflags and final_ldflags using
6 # what is needed for redis plus the standard cflags and ldflags passed.
7 # however when building the dependencies (jemalloc, lua, hiredis, ...)
8 # cflags and ldflags are propagated to the dependencies, so to pass
9 # flags only to be used when compiling / linking redis itself redis_cflags
10 # and redis_ldflags are used instead (this is the case of 'make gcov').
11 #
12 # dependencies are stored in the makefile.dep file. to rebuild this file
13 # just use 'make dep', but this is only needed by developers.
14
15 # makefile run sh ./mkreleasehdr.sh
16 release_hdr := $(shell sh -c './mkreleasehdr.sh')
17 # 定义直接展示式变量 uname_s := 内核名称, 例如 uname -s -> linux
18 uname_s := $(shell sh -c 'uname -s 2>/dev/null || echo not')
19 # 定义直接展示式变量 uname_m := 主机的硬件架构名称, 例如 uname -s -> x86_64
20 uname_m := $(shell sh -c 'uname -m 2>/dev/null || echo not')
21 # 定义条件赋值变量, 当 optimization 不存在, 会有 optimization :=-o2, 否则不改变 optimization
22 optimization?=-o2
23 # 定义递归展开式变量 dependency_targets =hiredis linenoise lua
24 # makefile 展开时候 = 后面如果有空格, 会保留.
25 dependency_targets=hiredis linenoise lua
26 # 定义直接展示式变量 nodeps :=clean distclean
27 nodeps:=clean distclean
28
29 # default settings
30 # -std=c11 使用 c11 标准
31 # -pedantic 以ansi/iso c标准列出的所有警告
32 # -dredis_static='' 进行 redis_static='' 宏定义
33 std=-std=c11 -pedantic -dredis_static=''
34 # ifneq ($1, $2) $1 != $2 为真
35 # $(findstring clang,$(cc)) 在 $(cc) 中查找 clang 字符串, 找到了返回 clang, 否则返回空串
36 # $(findstring freebsd,$(uname_s)) 在 $(uname_s) 中查找 freebsd 字符串 ...
37 # -wno-c11-extensions 删除 c11 扩展警告
38 ifneq (,$(findstring clang,$(cc)))
39 ifneq (,$(findstring freebsd,$(uname_s)))
40 std+=-wno-c11-extensions
41 endif
42 endif
43 # -wall 显示编译后所有警告
44 # -w 类似-wall 会显示警告, 但是只显示编译器认为会出现错误的警告
45 # -wno-missing-field-initializers 禁止结构初始化未指定所有字段警告
46 warn=-wall -w -wno-missing-field-initializers
47 # 定义递归展开式变量 opt =$(optimization) ?=-o2
48 opt=$(optimization)
49
50 # # 定义条件赋值变量, 当 prefix 不存在, 会有 prefix :=/usr/local, 否则维持 prefix 变量不变
51 prefix?=/usr/local
52 install_bin=$(prefix)/bin
53 install=install
54
55 # default allocator defaults to jemalloc if it's not an arm
56 malloc=libc
57 # ifneq ($(uname_m),armv6l) $(uname_m) 硬件架构不是 armv6l 才为真
58 # ifneq ($(uname_m),armv7l) $(uname_m) 硬件架构不是 armv7l 才为真
59 # ifeq ($(uname_s),linux) $(uname_s) 内核名称是 linux 才为真
60 ifneq ($(uname_m),armv6l)
61 ifneq ($(uname_m),armv7l)
62 ifeq ($(uname_s),linux)
63 malloc=jemalloc
64 endif
65 endif
66 endif
67
68 # to get arm stack traces if redis crashes we need a special c flag.
69 # $(filter aarch64 armv,$(uname_m)) $(uname_m) 硬件架构中如果是 aarch64 or armv 就保留
70 # -funwind-tables 开启便于做 backtrace
71 # unwind table 表记录了与函数相关的信息, 函数的起始地址, 函数的结束地址, 一个 info block 指针
72 ifneq (,$(filter aarch64 armv,$(uname_m)))
73 cflags+=-funwind-tables
74 else
75 ifneq (,$(findstring armv,$(uname_m)))
76 cflags+=-funwind-tables
77 endif
78 endif
79
80 # backwards compatibility for selecting an allocator
81 ifeq ($(use_tcmalloc),yes)
82 malloc=tcmalloc
83 endif
84
85 ifeq ($(use_tcmalloc_minimal),yes)
86 malloc=tcmalloc_minimal
87 endif
88
89 # ifeq ($(use_jemalloc),yes) $(use_jemalloc) == yes 会让 malloc=jemalloc
90 ifeq ($(use_jemalloc),yes)
91 malloc=jemalloc
92 endif
93
94 ifeq ($(use_jemalloc),no)
95 malloc=libc
96 endif
97
98 # override default settings if possible
99 # - 忽略错误
100 # include .make-settings 导入 .make-settings makefile 文件
101 -include .make-settings
102
103 # 以 linux jemalloc 为例
104 # final_cflags=-std=c11 -pedantic -dredis_static='' -wall -w -wno-missing-field-initializers -o2 -g -ggdb
105 # final_ldflags= -g -ggdb
106 # -g 操作系统的原生格式(native format)生成调试信息, 其它调试器也可以用
107 # -ggdb 使 gcc 为 gdb 生成专用的更为丰富的调试信息
108 # -lm 链接 math.h 相关库
109 final_cflags=$(std) $(warn) $(opt) $(debug) $(cflags) $(redis_cflags)
110 final_ldflags=$(ldflags) $(redis_ldflags) $(debug)
111 final_libs=-lm
112 debug=-g -ggdb
113
114 ifeq ($(uname_s),sunos)
115 # sunos
116 ifneq ($(@@),32bit)
117 cflags+= -m64
118 ldflags+= -m64
119 endif
120 debug=-g
121 debug_flags=-g
122 export cflags ldflags debug debug_flags
123 install=cp -pf
124 final_cflags+= -d__extensions__ -d_xpg6
125 final_libs+= -ldl -lnsl -lsocket -lresolv -lpthread -lrt
126 else
127 ifeq ($(uname_s),darwin)
128 # darwin
129 final_libs+= -ldl
130 openssl_cflags=-i/usr/local/opt/openssl/include
131 openssl_ldflags=-l/usr/local/opt/openssl/lib
132 else
133 ifeq ($(uname_s),aix)
134 # aix
135 final_ldflags+= -wl,-bexpall
136 final_libs+=-ldl -pthread -lcrypt -lbsd
137 else
138 ifeq ($(uname_s),openbsd)
139 # openbsd
140 final_libs+= -lpthread
141 ifeq ($(use_backtrace),yes)
142 final_cflags+= -duse_backtrace -i/usr/local/include
143 final_ldflags+= -l/usr/local/lib
144 final_libs+= -lexecinfo
145 endif
146
147 else
148 ifeq ($(uname_s),freebsd)
149 # freebsd
150 final_libs+= -lpthread -lexecinfo
151 else
152 ifeq ($(uname_s),dragonfly)
153 # freebsd
154 final_libs+= -lpthread -lexecinfo
155 else
156 # all the other oses (notably linux)
157 final_ldflags+= -rdynamic
158 final_libs+=-ldl -pthread -lrt
159 endif
160 endif
161 endif
162 endif
163 endif
164 endif
165 # include paths to dependencies
166 final_cflags+= -i../deps/hiredis -i../deps/linenoise -i../deps/lua/src
167
168 ifeq ($(malloc),tcmalloc)
169 final_cflags+= -duse_tcmalloc
170 final_libs+= -ltcmalloc
171 endif
172
173 ifeq ($(malloc),tcmalloc_minimal)
174 final_cflags+= -duse_tcmalloc
175 final_libs+= -ltcmalloc_minimal
176 endif
177
178 # ifeq ($(malloc),jemalloc) $(malloc) == jemalloc 为真才进入条件分支
179 # dependency_targets += jemalloc -> dependency_targets = hiredis linenoise lua jemalloc
180 # final_cflags 继续 += -duse_jemalloc -i../deps/jemalloc/include 引入 use_jemalloc 宏和指定头文件路径
181 # final_libs := ../deps/jemalloc/lib/libjemalloc.a -lm
182 ifeq ($(malloc),jemalloc)
183 dependency_targets+= jemalloc
184 final_cflags+= -duse_jemalloc -i../deps/jemalloc/include
185 final_libs := ../deps/jemalloc/lib/libjemalloc.a $(final_libs)
186 endif
187
188 # build_tls -> use_openssl
189 ifeq ($(build_tls),yes)
190 final_cflags+=-duse_openssl $(openssl_cflags)
191 final_ldflags+=$(openssl_ldflags)
192 final_libs += ../deps/hiredis/libhiredis_ssl.a -lssl -lcrypto
193 endif
194
195 redis_cc=$(quiet_cc)$(cc) $(final_cflags)
196 redis_ld=$(quiet_link)$(cc) $(final_ldflags)
197 redis_install=$(quiet_install)$(install)
198
199 # cccolor="\033[34m" 蓝色字
200 # linkcolor="\033[34;1m" 蓝色字 高亮
201 # srccolor="\033[33m" 黄色字
202 # bincolor="\033[37;1m" 白色字 高亮
203 # makecolor="\033[32;1m" 绿色字 高亮
204 # endcolor="\033[0m" 关闭所有属性
205 cccolor="\033[34m"
206 linkcolor="\033[34;1m"
207 srccolor="\033[33m"
208 bincolor="\033[37;1m"
209 makecolor="\033[32;1m"
210 endcolor="\033[0m"
211
212 # ifndef 不存在 v 那么就为真
213 # @printf makefile 屏幕上不再回显执行的命令
214 # %b 相对应的参数被视为含有要被处理的转义序列之字符串
215 # 1>&2 标准输出重定向到标准错误
216 # $@ 表示规则中的目标
217 # quiet_cc 编译, quiet_link 链接, quiet_install 安装
218 ifndef v
219 quiet_cc = @printf ' %b %b\n' $(cccolor)cc$(endcolor) $(srccolor)$@$(endcolor) 1>&2;
220 quiet_link = @printf ' %b %b\n' $(linkcolor)link$(endcolor) $(bincolor)$@$(endcolor) 1>&2;
221 quiet_install = @printf ' %b %b\n' $(linkcolor)install$(endcolor) $(bincolor)$@$(endcolor) 1>&2;
222 endif
223
224 redis_server_name=redis-server
225 redis_sentinel_name=redis-sentinel
226 redis_server_obj=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o
227 redis_cli_name=redis-cli
228 redis_cli_obj=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o anet.o ae.o crc64.o siphash.o crc16.o
229 redis_benchmark_name=redis-benchmark
230 redis_benchmark_obj=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o siphash.o redis-benchmark.o
231 redis_check_rdb_name=redis-check-rdb
232 redis_check_aof_name=redis-check-aof
233
234 all: $(redis_server_name) $(redis_sentinel_name) $(redis_cli_name) $(redis_benchmark_name) $(redis_check_rdb_name) $(redis_check_aof_name)
235 @echo ""
236 @echo "hint: it's a good idea to run 'make test' ;)"
237 @echo ""
238
239 # 构建 makefile.dep 文件
240 # -m 不是输出预编译过程的结果, 而是输出一个用于make的规则, 该规则描述了这个源文件的依赖关系.
241 # 预编译器输出的这个 make 规则包含名字与原文件相同的目标文件, 冒号和所有 include 文件的名字
242 # -mm 与 -m 相似, 只是不包含系统头文件
243 makefile.dep:
244 -$(redis_cc) -mm *.c > makefile.dep 2> /dev/null || true
245
246 # $(words <text>) 单词个数统计函数
247 # makecmdgoals 会存放你所指定的终极目标的列表，如果在命令行上，你没有指定目标，那么，这个变量是空值
248 # nodeps:=clean distclean
249 ifeq (0, $(words $(findstring $(makecmdgoals), $(nodeps))))
250 -include makefile.dep
251 endif
252
253 .phony: all
254
255 # 构建 .make-settings
256 persist-settings: distclean
257 echo std=$(std) >> .make-settings
258 echo warn=$(warn) >> .make-settings
259 echo opt=$(opt) >> .make-settings
260 echo malloc=$(malloc) >> .make-settings
261 echo cflags=$(cflags) >> .make-settings
262 echo ldflags=$(ldflags) >> .make-settings
263 echo redis_cflags=$(redis_cflags) >> .make-settings
264 echo redis_ldflags=$(redis_ldflags) >> .make-settings
265 echo prev_final_cflags=$(final_cflags) >> .make-settings
266 echo prev_final_ldflags=$(final_ldflags) >> .make-settings
267 -(cd ../deps && $(make) $(dependency_targets))
268
269 .phony: persist-settings
270
271 # prerequisites target
272 .make-prerequisites:
273 @touch $@
274
275 # clean everything, persist settings and build dependencies if anything changed
276 ifneq ($(strip $(prev_final_cflags)), $(strip $(final_cflags)))
277 .make-prerequisites: persist-settings
278 endif
279
280 # $(strip strint) 去空格函数
281 ifneq ($(strip $(prev_final_ldflags)), $(strip $(final_ldflags)))
282 .make-prerequisites: persist-settings
283 endif
284
285 # redis-server
286 $(redis_server_name): $(redis_server_obj)
287 $(redis_ld) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/lua/src/liblua.a $(final_libs)
288
289 # redis-sentinel
290 $(redis_sentinel_name): $(redis_server_name)
291 $(redis_install) $(redis_server_name) $(redis_sentinel_name)
292
293 # redis-check-rdb
294 $(redis_check_rdb_name): $(redis_server_name)
295 $(redis_install) $(redis_server_name) $(redis_check_rdb_name)
296
297 # redis-check-aof
298 $(redis_check_aof_name): $(redis_server_name)
299 $(redis_install) $(redis_server_name) $(redis_check_aof_name)
300
301 # redis-cli
302 $(redis_cli_name): $(redis_cli_obj)
303 $(redis_ld) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/linenoise/linenoise.o $(final_libs)
304
305 # redis-benchmark
306 $(redis_benchmark_name): $(redis_benchmark_obj)
307 $(redis_ld) -o $@ $^ ../deps/hiredis/libhiredis.a $(final_libs)
308
309 dict-benchmark: dict.c zmalloc.c sds.c siphash.c
310 $(redis_cc) $(final_cflags) $^ -d dict_benchmark_main -o $@ $(final_libs)
311
312 # because the jemalloc.h header is generated as a part of the jemalloc build,
313 # building it should complete before building any other object. instead of
314 # depending on a single artifact, build all dependencies first.
315 %.o: %.c .make-prerequisites
316 $(redis_cc) -c $<
317
318 clean:
319 rm -rf $(redis_server_name) $(redis_sentinel_name) $(redis_cli_name) $(redis_benchmark_name) $(redis_check_rdb_name) $(redis_check_aof_name) *.o *.gcda *.gcno *.gcov redis.info lcov-html makefile.dep dict-benchmark
320
321 .phony: clean
322
323 distclean: clean
324 -(cd ../deps && $(make) distclean)
325 -(rm -f .make-*)
326
327 .phony: distclean
328
329 test: $(redis_server_name) $(redis_check_aof_name)
330 @(cd ..; ./runtest)
331
332 test-sentinel: $(redis_sentinel_name)
333 @(cd ..; ./runtest-sentinel)
334
335 check: test
336
337 lcov:
338 $(make) gcov
339 @(set -e; cd ..; ./runtest --clients 1)
340 @geninfo -o redis.info .
341 @genhtml --legend -o lcov-html redis.info
342
343 test-sds: sds.c sds.h
344 $(redis_cc) sds.c zmalloc.c -dsds_test_main $(final_libs) -o /tmp/sds_test
345 /tmp/sds_test
346
347 .phony: lcov
348
349 bench: $(redis_benchmark_name)
350 ./$(redis_benchmark_name)
351
352 32bit:
353 @echo ""
354 @echo "warning: if it fails under linux you probably need to install libc6-dev-i386"
355 @echo ""
356 $(make) cflags="-m32" ldflags="-m32"
357
358 gcov:
359 $(make) redis_cflags="-fprofile-arcs -ftest-coverage -dcoverage_test" redis_ldflags="-fprofile-arcs -ftest-coverage"
360
361 noopt:
362 $(make) optimization="-o0"
363
364 valgrind:
365 $(make) optimization="-o0" malloc="libc"
366
367 helgrind:
368 $(make) optimization="-o0" malloc="libc" cflags="-d__atomic_var_force_sync_macros"
369
370 # 生成 src/help.h 头文件
371 # @../utils/generate-command-help.rb > help.h 通过 ruby 脚本请求
372 # https://raw.githubusercontent.com/antirez/redis-doc/master/commands.json 构建相关 c 中全局区变量信息
373 src/help.h:
374 @../utils/generate-command-help.rb > help.h
375
376 install: all
377 @mkdir -p $(install_bin)
378 $(redis_install) $(redis_server_name) $(install_bin)
379 $(redis_install) $(redis_benchmark_name) $(install_bin)
380 $(redis_install) $(redis_cli_name) $(install_bin)
381 $(redis_install) $(redis_check_rdb_name) $(install_bin)
382 $(redis_install) $(redis_check_aof_name) $(install_bin)
383 @ln -sf $(redis_server_name) $(install_bin)/$(redis_sentinel_name)
384
385 uninstall:
386 rm -f $(install_bin)/{$(redis_server_name),$(redis_benchmark_name),$(redis_cli_name),$(redis_check_rdb_name),$(redis_check_aof_name),$(redis_sentinel_name)}
感谢大家 ☆ redis 最简单的数据结构 adlist 双向链表就带大家写到这了. 有缘再见 ~
后记 - 如果
错误是难免, 欢迎大家指正交流, 共同提高 ~
if《如果》——rudyard kipling（拉迪亚德.吉普林） if you can keep your head when all about you are losing theirs and blaming it on you; if you can trust yourself when all men doubt you, but make allowance for their doubting too; if you can wait and not be tired by waiting, or, being lied about, don't deal in lies, or, being hated, don't give way to hating, and yet don't look too good, nor talk too wise; 如果所有人都失去理智，咒骂你， 你仍能保持头脑清醒； 如果所有人都怀疑你， 你仍能坚信自己，让所有的怀疑动摇； 如果你要等待，不要因此厌烦， 为人所骗，不要因此骗人， 为人所恨，不要因此抱恨， 不要太乐观，不要自以为是； if you can dream - and not make dreams your master; if you can think - and not make thoughts your aim; if you can meet with triumph and disaster and treat those two imposters just the same; if you can bear to hear the truth you've spoken twisted by knaves to make a trap for fools, or watch the things you gave your life to broken, and stoop and build ‘em up with worn-out tools; 如果你是个追梦人——不要被梦主宰； 如果你是个爱思考的人——不要以思想者自居； 如果你遇到骄傲和挫折 把两者当骗子看待； 如果你能忍受，你曾讲过的事实 被恶棍扭曲，用于蒙骗*； 或者，看着你用毕生去看护的东西被破坏， 俯下身去，用破旧的工具把它修补； if you can make one heap of all your winnings and risk it on one turn of pitch-and-toss, and lose, and start again at your beginnings and never breath a word about your loss; if you can force your heart and nerve and sinew to serve your turn long after they are gone, and so hold on when there is nothing in you except the will which says to them: "hold on"; 如果在你赢得无数桂冠之后， 然后孤注一掷再搏一次， 失败过后，东山再起， 不要抱怨你的失败； 如果你能迫使自己， 在别人走后，长久坚守阵地， 在你心中已空荡荡无一物， 只有意志告诉你“坚持！”； if you can talk with crowds and keep your virtue, or walk with kings - nor lose the common touch; if neither foes nor loving friends can hurt you; if all men count with you, but none too much; if you can fill the unforgiving minute with sixty seconds' worth of distance run - yours is the earth and everything that's in it, 如果你与人交谈，能保持风度， 伴王同行，能保持距离； 如果仇敌和好友都不害你； 如果所有人都指望你，却无人全心全意； 如果你花六十秒进行短程跑， 填满那不可饶恕的一分钟—— 你就可以拥有一个世界， 这个世界的一切都是你的， 更重要的是，孩子，你是个顶天立地的人。
上一篇： [Go] 分页计算页码的主要逻辑