InnoDB Memcached Plugin源码实现调研
背景 MySQL 5.6版本,新增了一个NoSQL的接口,通过将memcached嵌入到MySQL系统之中,用户可以直接使用memcached接口直接操作MySQL中的InnoDB表,绕过MySQL Server层面的SQL解析,优化,甚至绕过InnoDB Handler层,直接操作InnoDB内部的方法,从而达到更优的
背景
MySQL 5.6版本,新增了一个NoSQL的接口,通过将memcached嵌入到MySQL系统之中,用户可以直接使用memcached接口直接操作MySQL中的InnoDB表,绕过MySQL Server层面的SQL解析,优化,甚至绕过InnoDB Handler层,直接操作InnoDB内部的方法,从而达到更优的响应时间与效率。关于此功能的官方介绍,请见:InnoDB Integration with memcached 。嵌入memcached之后,整个MySQL的架构如下图所示:
本文接下来的部分,将从源码的角度,详细分析InnoDB Integration with memcached的实现细节问题。
导读
- InnoDB引擎为了支持Memcached API,在Handler层面进行的改动,请见(一);
- Memcached Plugin的初始化流程,请见(二);
- InnoDB提供的Callback方法在InnoDB Handlerton中的data中以及Memcached Plugin的data中,是如何传递的呢?请见(三);
- InnoDB Memcached Engine提供的方法集合,请见(四);
- InnoDB Memcached Engine提供的方法,如何调用到InnoDB Engine提供的方法,请见(五);
- InnoDB Memcached Engine中,存在两个Engine实例:InnoDB Engine vs Default Engine,二者的功能异同,请见(六);
- InnoDB Memcached Engine中,需要配置Container Table,Container表详解,请见(七);
- InnoDB Memcached Engine的初始化与使用流程,可参考engine_testapp.c测试文件;
(一) InnoDB Handler层面改动
InnoDB为了支持Memcached接口,在其Handler层面提供了新的Callback方法,这些方法的定义如下:
????????????/** Set up InnoDB API callback function array */
????????????ib_cb_t innodb_api_cb[] = {
????????????????(ib_cb_t) ib_cursor_open_table,
????????????????(ib_cb_t) ib_cursor_read_row,
????????????????(ib_cb_t) ib_cursor_insert_row,
????????????????(ib_cb_t) ib_cursor_delete_row,
????????????????(ib_cb_t) ib_cursor_update_row,
????????????????(ib_cb_t) ib_cursor_next,
????????????????(ib_cb_t) ib_cursor_last,
????????????????(ib_cb_t) ib_tuple_get_n_cols,
????????????????(ib_cb_t) ib_col_set_value,
????????????????(ib_cb_t) ib_col_get_value,
????????????????(ib_cb_t) ib_col_get_meta,
????????????????(ib_cb_t) ib_trx_begin,
????????????????(ib_cb_t) ib_trx_commit,
????????????????(ib_cb_t) ib_trx_rollback,
????????????????(ib_cb_t) ib_trx_start,
????????????????…
????????????};
同时,innodb_api_cb[]数组,被存储在InnoDB Handlerton的data字段中,可以向上传递给InnoDB Memcached Engine。InnoDB Memcached Engine可以调用这些Callback方法,达到直接操作InnoDB Engine的目的。
????????ha_innodb.cc::innobase_init();
????????????…
????????????innobase_hton->data = &innodb_api_cb;
(二) Memcached Plugin的初始化流程
Memcached Plugin,同样注册为MySQL的一个插件(memcached_mysql.cc),此插件的定义如下:
????????mysql_declare_plugin(daemon_memcached)
????????{
????????????MYSQL_DAEMON_PLUGIN,
????????????&daemon_memcached_plugin,
????????????”daemon_memcached”,
????????????”Oracle Corporation”,
????????????”Memcached Daemon”,
????????????PLUGIN_LICENSE_GPL,
????????????daemon_memcached_plugin_init,????????/* Plugin Init */
????????????daemon_memcached_plugin_deinit,????/* Plugin Deinit */
????????????0×0100 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/* 1.0 */,
????????????NULL, ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? /* status variables */
????????????daemon_memcached_sys_var, ? ? ? ? ? ? ?/* system variables */
????????????NULL ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?/* config options */
????????}
????????mysql_declare_plugin_end;
Memcached Plugin插件,初始化为daemon_memcached_plugin_init函数,该函数同样需要一个Handlerton输入,初始化Memcached Plugin插件,尤其是初始化InnoDB Memcached Engine。详细的初始化流程如下所示:
????struct mysql_memcached_context * con;
????// plugin为InnoDB的Handlerton,plugin->data指向InnoDB Handlerton中的data,
????// 在InnoDB的init函数中被初始化为innodb_api_cb数组。innodb_api_cb数组中,
????// 注册了各种InnoDB提供的方法。
????…
????con->memcached_conf.m_innodb_api_cb = plugin->data;
????…
????// 创建Memcached Plugin后台线程,线程的主函数为daemon_memcached_main(),
????// 线程传入参数,即为从InnoDB Handlerton处获取的InnoDB提供的Callback方法
????pthread_create(&con->memcached_thread, &attr,
????????????????daemon_memcached_main, (void *)&con->memcached_conf);
????????…
????????// 创建、加载InnoDB Memcached Engine
????????Engine_loader.c::load_engine();
????????????// Create InnoDB Memcached Engine
????????????// 注册InnoDB Memcached Engine的各种方法,例如:
????????????// 初始化方法:innodb_eng->engine.initialize = innodb_initialize;
????????????innodb_engine.c::create_instance();
????????????????…
????????// 初始化InnoDB Memcached Engine
????????Engine_loader.c::init_engine(engine_handle,engine_config…);
????????????Innodb_engine.c::innodb_initialize(engine, config_str);
????????????????// 注册InnoDB Engine提供的内部方法至InnoDB Memcached Engine,
????????????????// 包括:读取、插入、删除、更新、事务创建/提交/回滚等操作。
????????????????// 至此,后续的InnoDB Memcached Engine的所有操作,均可被映射
????????????????// 为InnoDB Storage Engine提供的内部方法进行实际操作。
????????????????Innodb_api.c::register_innodb_cb();
????????????????// Fetch InnoDB specific settings
????????????????innodb_eng->meta_info = innodb_config(NULL, 0, &innodb_eng->meta_hash);
????????????????????// Find the table and column info that used for memcached data
????????????????????// Cantainers表:InnoDB内部存储元数据的系统表,
????????????????????// MCI_CFG_DB_NAME;MCI_CFG_CONTAINER_TABLE;…
????????????????????innodb_config.c::innodb_config_container();
????????????????…
????????????????// 开启InnoDB Memcached Engine的后台自动提交线程
????????????????innodb_engine.c::innodb_bk_thread();
????????????????????…
????????…
????// 在InnoDB Handlerton中存储更多Memcached Plugin所需要的信息
????plugin->data = (void *)con;
(三) Memcached Plugin如何获取使用InnoDB Handlerton中的innodb_api_cb?
// 在此函数中完成innodb_api_cb的传递
sql_plugin.cc::plugin_initialize();
????// 此函数指针,直接调用到Plugin注册的init方法
????(*plugin_type_initialize[plugin->plugin->type])(plugin);
????…
????// 判断出当前是InnoDB引擎完成了初始化,则将InnoDB Handlerton中
? ? // 的data拷贝到临时变量中
????if (strcmp(plugin->name.str, “InnoDB”) == 0)
????????innodb_callback_data = ((Handlerton *)plugin->data)->data;
????// 判断出当前Plugin为Memcached Plugin,则将临时变量赋值到memcached plugin
? ? // 的data中,完成InnoDB Handlerton提供的Callback方法的传递
????else if (plugin->plugin->init)
????????if (strcmp(plugin->name.str, “daemon_memcached”) == 0)
????????????plugin->data = (void *)innodb_callback_data;
????????// 初始化Memcached Plugin
????????plugin->plugin->init();
(四) InnoDB Memcached Plugin提供的方法集合
????????innodb_engine.c::create_instance();
????????????innodb_eng->engine.interface.interface = 1;
????????????innodb_eng->engine.get_info = innodb_get_info;
????????????innodb_eng->engine.initialize = innodb_initialize;
????????????innodb_eng->engine.destroy = innodb_destroy;
????????????innodb_eng->engine.allocate = innodb_allocate;
????????????innodb_eng->engine.remove = innodb_remove;
????????????innodb_eng->engine.release = innodb_release;
????????????innodb_eng->engine.clean_engine= innodb_clean_engine;
????????????innodb_eng->engine.get = innodb_get;
????????????innodb_eng->engine.get_stats = innodb_get_stats;
????????????innodb_eng->engine.reset_stats = innodb_reset_stats;
????????????innodb_eng->engine.store = innodb_store;
????????????innodb_eng->engine.arithmetic = innodb_arithmetic;
????????????innodb_eng->engine.flush = innodb_flush;
????????????innodb_eng->engine.unknown_command = innodb_unknown_command;
????????????innodb_eng->engine.item_set_cas = item_set_cas;
????????????innodb_eng->engine.get_item_info = innodb_get_item_info;
????????????innodb_eng->engine.get_stats_struct = NULL;
????????????innodb_eng->engine.errinfo = NULL;
????????????innodb_eng->engine.bind = innodb_bind;
所有InnoDB Memcached Engine提供的方法,都是类Memcached方法。例如:get/remove/store方法等。
(五) 一个InnoDB Memcached Plugin的操作的流程
通过Memcached Plugin进行一个删除操作的函数处理流程,如下:
注1:删除操作,对应的InnoDB Handlerton提供的方法为:????remove()
注2:删除操作,对应的InnoDB Memcached Engine的方法为:????ib_cursor_delete_row()
// 通过Memcached接口删除操作的处理流程
memcached.c::process_delete_command();
????settings.engine.v1->remove(settings.engine.v0, c, key, nkey, 0, 0);
????// InnoDB Memcached Engine层面提供的删除方法
????innodb_engine.c::innodb_remove(ENGINE_HANDLE*, cookie, key, nkey, …);
????????…
????????// 初始化InnoDB的连接,传入参数分析:
????????// CONN_MODE_WRITE:当前为写操作
????????// IB_LOCK_X:????????当前操作需要对记录行加X锁
????????conn_data = innodb_conn_init(innodb_eng, cookie, CONN_MODE_WRITE, …);
????????????if (conn_option == CONN_MODE_WRITE)
????????????????// 开始一个事务
????????????????innodb_api.c::innodb_cb_trx_begin();
????????????????????ib_cb_trx_begin() -> api0api.cc::ib_trx_begin();
????????????????// 打开当前表,并设置游标
????????????????innodb_api.c::innodb_api_begin();
????????????????????…
????????// 进行真正的删除操作
????????innodb_api.c::innodb_api_delete(innodb_eng, conn_data, key, nkey);
????????????// 根据传入的key,将游标定位到正确的位置
????????????innodb_api.c::innodb_api_search();
????????????????…
????????????// 如果开启了binlog,则需要将定位到的记录拷贝出来
????????????if (engine->enable_binlog)
????????????????…
????????????// 删除记录
????????????????ib_cb_delete_row() -> api0api.cc::ib_cursor_delete_row();
????????????// 记录binlog
????????????????handler_api.cc::handler_binlog_row();
????????????// 删除构造出来的Tuple对象
????????????ib_cb_tuple_delete() -> api0api.cc::ib_tuple_delete();
(六) InnoDB Engine vs Default Engine
在InnoDB Engine结构的内部(innodb_engine.h::struct innodb_engine),有两个实例化的Engine Handle,分别为:
????????ENGINE_HANDLE_V1????engine;
????????ENGINE_HANDLE*????????default_engine;
两个engine handle有何区别:
-
ENGINE_HANDLE_V1为InnoDB Engine Handle,封装了所有InnoDB引擎提供的方法;
-
ENGINE_HANDLE(default_engine)为Memcached自带的默认Engine,封装了所有标准Memcached所提供的方案,包括:slab分配,数据的存取、失效等等。
-
InnoDB Engine与Default Engine是否启用?是否同时启用?通过参数控制:
????/** Tells if we will used Memcached default engine or InnoDB Memcached engine to handle the request */
????typedef enum meta_cache_opt {
????????META_CACHE_OPT_INNODB = 1,????????/*!
????????META_CACHE_OPT_DEFAULT, ? ? ? ? ? ? /*!
????????META_CACHE_OPT_MIX, ? ? ? ? ? ? ? ? ? ? ? ?/*!
????????META_CACHE_OPT_DISABLE, ? ? ? ? ? ? ? /*!
????????META_CACHE_NUM_OPT ? ? ? ? ? ? ? ? ? ? ? ?/*!
????} meta_cache_opt_t;
默认的参数值为META_CACHE_OPT_INNODB,仅仅启用InnoDB Engine,数据通过InnoDB引擎提供的Callback方法操作;若同时启用了Memcached Default Engine(META_CACHE_OPT_MIX),那么数据读取时,首先从Default Engine中读取;记录删除时,也需要先删除Default Engine中的记录;
(七) Container表详解
InnoDB Memcached Engine,包含一个Container Table,用于存储Memcached与InnoDB Table之间的映射关系。
Container Table,必须包含9列,分别是:
????/** Columns in the “containers” system table, this maps the Memcached
????operation to a consistent InnoDB table */
????typedef enum container {
????????CONTAINER_NAME, ? ? ? ? /*!
????????CONTAINER_DB, ? ? ? ? ? ? ? ?/*!
????????CONTAINER_TABLE,????????/*!
????????CONTAINER_KEY, ? ? ? ? ? ? /*!
????????CONTAINER_VALUE, ? ? ? /*!
????????CONTAINER_FLAG, ? ? ? ? ?/*!
????????CONTAINER_CAS, ? ? ? ? ? ? /*!
????????CONTAINER_EXP, ? ? ? ? ? ? /*!
????????CONTAINER_NUM_COLS????/*!
????} container_t;
其中:
-
CONTAINER_TABLE为InnoDB表名,CONTAINER_DB为对应的Database名;
-
CONTAINER_KEY为Memcached的Key对应的InnoDB表中的列名(只能一列);
-
CONTAINER_VALUE为Memcached的Value对应的InnoDB表中的列名(可以有多列,以” ;,|\n”作为分隔符,详见innodb_config.c::innodb_config_parse_value_col()函数);
-
CONTAINER Table的最后一列,为CONTAINER_KEY列对应的InnoDB表的唯一索引名(必须存在);
-
CONTAINER Table,在innodb_engine.cc::innodb_initialize函数被读取,将其中的每一项都解析并存储到InnoDB Engine的meta_hash结构之中:
????innodb_config.c::innodb_config(NULL, 0, &innodb_eng->meta_hash);
????????innodb_config_meta_hash_init();
????????????…
????????????// 打开CONTAINER Table
????????????innodb_api.c::innodb_api_begin(…, MCI_CFG_CONTAINER_TABLE, …);
????????????innodb_api.c::innodb_cb_read_row();
????????????innodb_config.c::innodb_config_add_item();
????????????????…
????????????????// 针对Value列,需要解析此列对应于InnoDB Table的哪些列
????????????????// InnoDB中的不同列,以” ;,|\n”作为分隔符
????????????????if (i == CONTAINER_VALUE)
????????????????????innodb_config_parse_value_col();
????????????????…
在innodb_engine.cc::innodb_initialize函数,完成Container Table的读取以及Meta Hash的填充之后,后续的Memcached方法,才可以根据规则操作,完成记录在Memcached与InnoDB引擎间的传递。
背景 MySQL 5.6版本,新增了一个NoSQL的接口,通过将memcached嵌入到MySQL系统之中,用户可以直接使用memcached接口直接操作MySQL中的InnoDB表,绕过MySQL Server层面的SQL解析,优化,甚至绕过InnoDB Handler层,直接操作InnoDB内部的方法,从而达到更优的响应时间与效率。关于此功能的官方介绍,请见:InnoDB Integration with memcached 。嵌入memcached之后,整个MySQL的架构如下图所示: 本文接下来的部分,将从源码的角度,详细分析InnoDB Integration with memcached的实现细节问题。 导读 InnoDB引擎为了支持Memcached API,在Handler层面进行的改动,请见(一); Memcached Plugin的初始化流程,请见(二); InnoDB提供的Callback方法在InnoDB Handlerton中的data中以及Memcached Plugin的data中,是如何传递的呢?请见(三); InnoDB Memcached Engine提供的方法集合,请见(四); InnoDB Memcached Engine提供的方法,如何调用到InnoDB Engine提供的方法,请见(五); InnoDB Memcached Engine中,存在两个Engine实例:InnoDB … 继续阅读下一篇: 第二种分页类