欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

protobuff的erlang nif封装

程序员文章站 2022-07-15 08:16:23
...

    服务器协议一直以来都使用erlang-protobuff,项目地址:https://github.com/ngerakines/erlang_protobuffs,但觉得效率不行,于是想优化一下,思路就是把协议编解码放到底层c去做。正好可以使用云风写的pbc:https://github.com/cloudwu/pbc,于是拿来封装一下nif。虽然看起来很简单,但中间也遇到了一些问题:

1. pbc是非线程安全的,解决办法是初始化时为每个线程创建一个pbc_env环境

2. pbc以字符串key作为消息的字段,但erlang的record结构没有key,只是一个存放term的数组,怎么对应起来

3. c层如何构造一个初始的erlang消息结构,然后解完消息之后,返回给erlang层使用

4. 这个初始的erlang结构都是固定的,怎么把这个erlang结构在c层保存起来一直使用

 

最终做出来效率提升好几倍,而且可以同时支持record和map结构, 代码已提交到github,https://github.com/616050468/erlang_pbc

 

erlang_pbc

erlang_pbc is erlang nif of protobuff-c which is developed by cloudwu, want to know protobuff-c more, please see:https://github.com/cloudwu/pbc

erlang_pbc is faster than erlang-protobuff, support record and map. if you use map, you don't need any erlang code.

Quick Example

package msg;

message attriMessage {
    optional int32 index = 1;
    optional int32 value = 2;
}

message longMessage {
    optional int64 dbID = 1;
    optional int32 sceneID = 2;
    optional int32 sceneLine = 3;
    optional int32 x = 4;
    optional int32 y = 5;
    optional int32 z = 6;
    repeated attriMessage attriList = 7;
    optional string name = 8;
    optional int32 career = 9;
    optional bool sex = 10;
    optional string signature = 11;
    optional int32 vip_level = 12;
    optional string telphone = 13;
}
Eshell V6.1  (abort with ^G)
1> test_msg:init_pbc().
[ok,ok,ok,ok,ok,ok,ok]
2> pbc_nif:get_default("msg.attriMessage").
{{'msg.attriMessage',0,0},#{"index" => 0,"value" => 0}}
3>
3> Bin = pbc_nif:encode("msg.attriMessage", {'msg.attriMessage', 100, 100}).
<<8,100,16,100>>
4>
4> pbc_nif:decode("msg.attriMessage", Bin, 0).
{'msg.attriMessage',100,100}
5>
5> pbc_nif:decode("msg.attriMessage", Bin, 1).
#{"index" => 100,"value" => 100}

Profile

please see test_msg.erl to know more....

test data in personal note-book:

Intel(R) Xeon(R) CPU E3-1231 v3 @3.40GHz
RAM:  8.00GB
OS TYPE: 64-bit
Eshell V6.1  (abort with ^G)
1> test_msg:profile().

begin profile encode 
encode_short_erlang (10.0w): 139ms
encode_short_pbc_map (10.0w): 58ms
encode_short_pbc_record (10.0w): 52ms
encode_long_erlang (10.0w): 850ms
encode_long_pbc_map (10.0w): 187ms
encode_long_pbc_record (10.0w): 152ms
encode_big_erlang (10.0w): 13084ms
encode_big_pbc_map (10.0w): 2203ms
encode_big_pbc_record (10.0w): 1739ms

begin profile decode 
decode_short_erlang (10.0w): 164ms
decode_short_pbc_map (10.0w): 76ms
decode_short_pbc_record (10.0w): 60ms
decode_long_erlang (10.0w): 1342ms
decode_long_pbc_map (10.0w): 424ms
decode_long_pbc_record (10.0w): 175ms
decode_big_erlang (10.0w): 20312ms
decode_big_pbc_map (10.0w): 6099ms
decode_big_pbc_record (10.0w): 2556ms
ok

compile

Windows: open project use VS and build the project.

Linux: (use premake4 to create makefile)

premake4 gmake