欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

自行实现比dotcore/dotnet更方便更高性能的对象二进制序列化

程序员文章站 2022-07-07 22:46:21
二进制序列化可以方便快捷的将对象进行持久化或者网络传输,并且体积小、性能高,应用面甚至还要高于json的序列化;开始之前,先来看看dotcore/dotne自带的二进制序列化:C#中对象序列化和反序列化一般是通过BinaryFormatter类来实现的二进制序列化、反序列化的。 BinaryForm ......

        二进制序列化可以方便快捷的将对象进行持久化或者网络传输,并且体积小、性能高,应用面甚至还要高于json的序列化;开始之前,先来看看dotcore/dotne自带的二进制序列化:C#中对象序列化和反序列化一般是通过BinaryFormatter类来实现的二进制序列化、反序列化的。

        BinaryFormatter序列化:

1 System.Runtime.Serialization.Formatters.Binary.BinaryFormatter serializer = new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
2 
3 System.IO.MemoryStream memStream = new System.IO.MemoryStream();
4 
5 serializer.Serialize(memStream, request);

        BinaryFormatter反序列化:

 1  memStream.Position=0;
 2 
 3  System.Runtime.Serialization.Formatters.Binary.BinaryFormatter deserializer =
 4 
 5  new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
 6 
 7  object newobj = deserializer.Deserialize(memStream);
 8 
 9  memStream.Close();
10 
11  return newobj;

        用着多了就发现BinaryFormatter有很多地方不妥,下面就来数数这个序列化的“三宗罪”:

        1.类名上面要加上[Serializable],不加不给序列化;正常的用法应该是序列化一个对象,不需的地方加上NonSerialized才合理吧;

        2.序列化byte[]结果非常大,使用System.Text.Encoding.UTF8.GetString(bytes)查看下,发现里面有一大堆的元数据;对比看看google的protobuf,pb为什么在网络上应用的越来越多,这和他本身序列化完后体积小有着绝大部门的原因;

        3.序列化对象需要完全一致,连类的命名空间都要相同,这点对于分面式开发的应用来说也是不可接受的;

        既然BinaryFormatter不好用,那就只能动手自行实现一个解决上述问题的二进制序列化方案;首先去掉[Serializable]这个标签,接着主要是分析对象,并定义对象序列化后的数据结构;这里的想法是按长度加内容的方式来定义,举个例子:使用int作为长度,来保存一个int值,序列化完应该是:4,0,0,0,1,0,0,0这样的一组bytes,同理可以将int、short、long、float、double、datetime、enum、array、string、class、generic等按照这个格式进行序列化,这里主要使用的是BitConverter、反射等来实现序列化与反序列化;

        序列化实现如下:

自行实现比dotcore/dotnet更方便更高性能的对象二进制序列化
  1         public static byte[] Serialize(object param)
  2         {
  3             List<byte> datas = new List<byte>();
  4 
  5             var len = 0;
  6 
  7             byte[] data = null;
  8 
  9             if (param == null)
 10             {
 11                 len = 0;
 12             }
 13             else
 14             {
 15                 if (param is string)
 16                 {
 17                     data = Encoding.UTF8.GetBytes((string)param);
 18                 }
 19                 else if (param is byte)
 20                 {
 21                     data = new byte[] { (byte)param };
 22                 }
 23                 else if (param is bool)
 24                 {
 25                     data = BitConverter.GetBytes((bool)param);
 26                 }
 27                 else if (param is short)
 28                 {
 29                     data = BitConverter.GetBytes((short)param);
 30                 }
 31                 else if (param is int)
 32                 {
 33                     data = BitConverter.GetBytes((int)param);
 34                 }
 35                 else if (param is long)
 36                 {
 37                     data = BitConverter.GetBytes((long)param);
 38                 }
 39                 else if (param is float)
 40                 {
 41                     data = BitConverter.GetBytes((float)param);
 42                 }
 43                 else if (param is double)
 44                 {
 45                     data = BitConverter.GetBytes((double)param);
 46                 }
 47                 else if (param is DateTime)
 48                 {
 49                     var str = "wl" + ((DateTime)param).Ticks;
 50                     data = Encoding.UTF8.GetBytes(str);
 51                 }
 52                 else if (param is Enum)
 53                 {
 54                     var enumValType = Enum.GetUnderlyingType(param.GetType());
 55 
 56                     if (enumValType == typeof(byte))
 57                     {
 58                         data = new byte[] { (byte)param };
 59                     }
 60                     else if (enumValType == typeof(short))
 61                     {
 62                         data = BitConverter.GetBytes((Int16)param);
 63                     }
 64                     else if (enumValType == typeof(int))
 65                     {
 66                         data = BitConverter.GetBytes((Int32)param);
 67                     }
 68                     else
 69                     {
 70                         data = BitConverter.GetBytes((Int64)param);
 71                     }
 72                 }
 73                 else if (param is byte[])
 74                 {
 75                     data = (byte[])param;
 76                 }
 77                 else
 78                 {
 79                     var type = param.GetType();
 80 
 81 
 82                     if (type.IsGenericType || type.IsArray)
 83                     {
 84                         if (TypeHelper.DicTypeStrs.Contains(type.Name))
 85                             data = SerializeDic((System.Collections.IDictionary)param);
 86                         else if (TypeHelper.ListTypeStrs.Contains(type.Name) || type.IsArray)
 87                             data = SerializeList((System.Collections.IEnumerable)param);
 88                         else
 89                             data = SerializeClass(param, type);
 90                     }
 91                     else if (type.IsClass)
 92                     {
 93                         data = SerializeClass(param, type);
 94                     }
 95 
 96                 }
 97                 if (data != null)
 98                     len = data.Length;
 99             }
100             datas.AddRange(BitConverter.GetBytes(len));
101             if (len > 0)
102             {
103                 datas.AddRange(data);
104             }
105             return datas.Count == 0 ? null : datas.ToArray();
106         }
View Code

        反序列化实现如下:

自行实现比dotcore/dotnet更方便更高性能的对象二进制序列化
  1         public static object Deserialize(Type type, byte[] datas, ref int offset)
  2         {
  3             dynamic obj = null;
  4 
  5             var len = 0;
  6 
  7             byte[] data = null;
  8 
  9             len = BitConverter.ToInt32(datas, offset);
 10             offset += 4;
 11             if (len > 0)
 12             {
 13                 data = new byte[len];
 14                 Buffer.BlockCopy(datas, offset, data, 0, len);
 15                 offset += len;
 16 
 17                 if (type == typeof(string))
 18                 {
 19                     obj = Encoding.UTF8.GetString(data);
 20                 }
 21                 else if (type == typeof(byte))
 22                 {
 23                     obj = (data);
 24                 }
 25                 else if (type == typeof(bool))
 26                 {
 27                     obj = (BitConverter.ToBoolean(data, 0));
 28                 }
 29                 else if (type == typeof(short))
 30                 {
 31                     obj = (BitConverter.ToInt16(data, 0));
 32                 }
 33                 else if (type == typeof(int))
 34                 {
 35                     obj = (BitConverter.ToInt32(data, 0));
 36                 }
 37                 else if (type == typeof(long))
 38                 {
 39                     obj = (BitConverter.ToInt64(data, 0));
 40                 }
 41                 else if (type == typeof(float))
 42                 {
 43                     obj = (BitConverter.ToSingle(data, 0));
 44                 }
 45                 else if (type == typeof(double))
 46                 {
 47                     obj = (BitConverter.ToDouble(data, 0));
 48                 }
 49                 else if (type == typeof(decimal))
 50                 {
 51                     obj = (BitConverter.ToDouble(data, 0));
 52                 }
 53                 else if (type == typeof(DateTime))
 54                 {
 55                     var dstr = Encoding.UTF8.GetString(data);
 56                     var ticks = long.Parse(dstr.Substring(2));
 57                     obj = (new DateTime(ticks));
 58                 }
 59                 else if (type.BaseType == typeof(Enum))
 60                 {
 61                     var numType = Enum.GetUnderlyingType(type);
 62 
 63                     if (numType == typeof(byte))
 64                     {
 65                         obj = Enum.ToObject(type, data[0]);
 66                     }
 67                     else if (numType == typeof(short))
 68                     {
 69                         obj = Enum.ToObject(type, BitConverter.ToInt16(data, 0));
 70                     }
 71                     else if (numType == typeof(int))
 72                     {
 73                         obj = Enum.ToObject(type, BitConverter.ToInt32(data, 0));
 74                     }
 75                     else
 76                     {
 77                         obj = Enum.ToObject(type, BitConverter.ToInt64(data, 0));
 78                     }
 79                 }
 80                 else if (type == typeof(byte[]))
 81                 {
 82                     obj = (byte[])data;
 83                 }
 84                 else if (type.IsGenericType)
 85                 {
 86                     if (TypeHelper.ListTypeStrs.Contains(type.Name))
 87                     {
 88                         obj = DeserializeList(type, data);
 89                     }
 90                     else if (TypeHelper.DicTypeStrs.Contains(type.Name))
 91                     {
 92                         obj = DeserializeDic(type, data);
 93                     }
 94                     else
 95                     {
 96                         obj = DeserializeClass(type, data);
 97                     }
 98                 }
 99                 else if (type.IsClass)
100                 {
101                     obj = DeserializeClass(type, data);
102                 }
103                 else if (type.IsArray)
104                 {
105                     obj = DeserializeArray(type, data);
106                 }
107                 else
108                 {
109                     throw new RPCPamarsException("ParamsSerializeUtil.Deserialize 未定义的类型:" + type.ToString());
110                 }
111 
112             }
113             return obj;
114         }
View Code

        其他详细的代码可以查看

        功能基本实现了,下面对比一下10000次的实体序列化与反序列化测试结果:

        实体代码:

 1             var groupInfo = new GroupInfo()
 2             {
 3                 GroupID = 1,
 4                 IsTemporary = false,
 5                 Name = "yswenli group",
 6                 Created = DateTimeHelper.Now,
 7                 Creator = new UserInfo()
 8                 {
 9 
10                     ID = 1,
11                     Birthday = DateTimeHelper.Now.AddYears(-100),
12                     UserName = "yswenli"
13                 },
14                 Users = new System.Collections.Generic.List<UserInfo>()
15                 {
16                     new UserInfo()
17                     {
18 
19                         ID = 1,
20                         Birthday = DateTimeHelper.Now.AddYears(-100),
21                         UserName = "yswenli"
22                     }
23                 }
24             };

        测试代码:

自行实现比dotcore/dotnet更方便更高性能的对象二进制序列化
 1         public static byte[] SerializeBinary(object request)
 2         {
 3 
 4             System.Runtime.Serialization.Formatters.Binary.BinaryFormatter serializer =
 5 
 6             new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
 7 
 8             using (System.IO.MemoryStream memStream = new System.IO.MemoryStream())
 9             {
10                 serializer.Serialize(memStream, request);
11 
12                 return memStream.ToArray();
13             }
14         }
15 
16 
17         public static object DeSerializeBinary(byte[] data)
18         {
19             using (System.IO.MemoryStream memStream = new System.IO.MemoryStream(data))
20             {
21                 System.Runtime.Serialization.Formatters.Binary.BinaryFormatter deserializer =
22 
23                 new System.Runtime.Serialization.Formatters.Binary.BinaryFormatter();
24 
25                 return deserializer.Deserialize(memStream);
26             }
27         }
28 
29         static void SerializeTest()
30         {
31             var groupInfo = new GroupInfo()
32             {
33                 GroupID = 1,
34                 IsTemporary = false,
35                 Name = "yswenli group",
36                 Created = DateTimeHelper.Now,
37                 Creator = new UserInfo()
38                 {
39 
40                     ID = 1,
41                     Birthday = DateTimeHelper.Now.AddYears(-100),
42                     UserName = "yswenli"
43                 },
44                 Users = new System.Collections.Generic.List<UserInfo>()
45                 {
46                     new UserInfo()
47                     {
48 
49                         ID = 1,
50                         Birthday = DateTimeHelper.Now.AddYears(-100),
51                         UserName = "yswenli"
52                     }
53                 }
54             };
55 
56             var count = 100000;
57             var len1 = 0;
58             var len2 = 0;
59 
60             Stopwatch sw = new Stopwatch();
61             sw.Start();
62 
63             List<byte[]> list = new List<byte[]>();
64             for (int i = 0; i < count; i++)
65             {
66                 var bytes = SerializeBinary(groupInfo);
67                 len1 = bytes.Length;
68                 list.Add(bytes);
69             }
70             ConsoleHelper.WriteLine($"BinaryFormatter实体序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");
71 
72             sw.Restart();
73             for (int i = 0; i < count; i++)
74             {
75                 var obj = DeSerializeBinary(list[i]);
76             }
77             ConsoleHelper.WriteLine($"BinaryFormatter实体反序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");
78             ConsoleHelper.WriteLine($"BinaryFormatter序列化生成bytes大小:{len1 * count * 1.0 / 1024 / 1024} Mb");
79             list.Clear();
80             sw.Restart();
81 
82             for (int i = 0; i < count; i++)
83             {
84                 var bytes = RPC.Serialize.ParamsSerializeUtil.Serialize(groupInfo);
85                 len2 = bytes.Length;
86                 list.Add(bytes);
87             }
88             ConsoleHelper.WriteLine($"ParamsSerializeUtil实体序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");
89             sw.Restart();
90             for (int i = 0; i < count; i++)
91             {
92                 int os = 0;
93 
94                 var obj = RPC.Serialize.ParamsSerializeUtil.Deserialize(groupInfo.GetType(), list[i], ref os);
95             }
96             ConsoleHelper.WriteLine($"ParamsSerializeUtil实体反序列化平均:{count * 1000 / sw.ElapsedMilliseconds} 次/秒");
97             ConsoleHelper.WriteLine($"ParamsSerializeUtil序列化生成bytes大小:{len2 * count * 1.0 / 1024 / 1024} Mb");
98             sw.Stop();
99         }
View Code

        运行结果:

自行实现比dotcore/dotnet更方便更高性能的对象二进制序列化