一个TCP长连接设备管理后台工程(四)---jtt808协议解析

协议解析

从前面内容我们可以发现，808协议是一个很典型的协议格式：

固定字段+变长字段

其中固定字段用来检测一个帧格式的完整性和有效性，所以一般会包含一下内容：帧头+变长字段对应的长度+校验。由于这一段的数据格式固定，目的单一，所以处理起来比较简单。

变长字段的长度是由固定字段终端某一个子字段的值决定的，而且这部分的格式比较多变，需要灵活处理。这一字段我们通常称为Body或者Apdu。

我们首先说明变长字段的处理流程。

Body处理

正因为Body字段格式灵活，所以为了提高代码的复用性和拓展性，我们需要对Body的处理机制进行抽象，提取出一个相对通用的接口出来。

有经验的工程师都知道，一个协议格式处理，无非就是编码和解码。编码我们称之为Marshal，解码我们称之为Unmarshal。对于不同的格式，我们只需要提供不同的Marshal和Unmarshal实现即可。

从前面分析可以知道，我们现在面对的一种格式是类似于Plain的格式，这种格式没有基本的分割符，下面我们就对这种编码来实现Marshal和Unmarshal。我们将这部分逻辑定义为一个codec包

package codecfunc Unmarshal(data []byte, v interface{}) (int, error){}
func Marshal(v interface{}) ([]byte, error){}

参考官方库解析json的流程，很快我们就想到了用反射来实现这两个功能。

首先我们来分析Unmarshal，我们需要按照v的类型，将data数据按照对应的长度和类型赋值。举个最简单的例子:

func TestSimple(t *testing.T) {type Body struct {Age1 int8Age2 int16}data := []byte{0x01, 0x02, 0x03}pack := Body{}i, err := Unmarshal(data, &pack)if err != nil {t.Errorf("err:%s", err.Error())}t.Log("len:", i)t.Log("pack:", pack)
}

$ go test -v server/codec -run TestSimple
=== RUN   TestSimple
--- PASS: TestSimple (0.00s)codec_test.go:20: len: 3codec_test.go:21: pack: {1 515}
PASS
ok      server/codec    0.002s

对于Body结构体，第一个字段是int8，占用一个字节，所以分配的值是0x01。第二个字段是int16，占用两个字节，分配的值是0x02,0x03，然后把这两个字节按照大端格式组合成一个int16就行了。所以结果就是Age1字段为1(0x01)，Age2字段为515(0x0203)

所以处理的关键是，我们要识别出v interface{}的类型，然后计算该类型对应的大小，再将data中对应大小的数据段组合成对应类型值复制给v中的对应字段。

v interface{}的类型多变，可能会涉及到结构体嵌套等，所以会存在递归处理，当然第一步我们需要获取到v的类型：

rv := reflect.ValueOf(v)
switch rv.Kind() {case reflect.Int8://case reflect.Uint8://case reflect.Int16://case reflect.Uint16://case reflect.Int32://case reflect.Uint32://case reflect.Int64://case reflect.Uint64://case reflect.Float32://case reflect.Float64://case reflect.String://case reflect.Slice://case reflect.Struct://需要对struct中的每个元素进行解析
}

其他的类型都比较好处理，需要说明的是struct类型，首先我们要能够遍历struct中的各个元素，于是我们找到了：

fieldCount := v.NumField()
v.Field(i)

NumField()能够获取结构体内部元素个数，然后Field(i)通过指定index就可以获取到指定的元素了。获取到了元素后，我们就需要最这个元素进行再次的Unmarshal，也就是递归。但是此时我们通过v.Field(i)获取到的是reflect.Value类型，而不是interface{}类型了，所以递归的入参我们使用reflect.Value。另外还需要考虑的一个问题是data数据的索引问题，一次调用Unmarshal就会消耗掉一定字节的data数据，消耗的长度应该能够被获取到，以方便下一次调用Unmarshal时，能够对入参的data数据索引做正确的设定。因此，Unmarshal函数需要返回一个当前当用后所占用的字节长度。比如int8就是一个字节，struct就是各个字段字节之和。

func Unmarshal(data []byte, v interface{})  (int,error) {rv := reflect.ValueOf(v)if rv.Kind() != reflect.Ptr || rv.IsNil() {return 0,fmt.Errorf("error")}return refUnmarshal(data, reflect.ValueOf(v))
}func refUnmarshal(data []byte, v reflect.Value)  (int,error) {var usedLen int = 0if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:usedLen = usedLen + 1case reflect.Uint8:usedLen = usedLen + 1case reflect.Int16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}usedLen = usedLen + 2case reflect.Uint16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}usedLen = usedLen + 2case reflect.Int32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}usedLen = usedLen + 4case reflect.Uint32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}usedLen = usedLen + 4case reflect.Int64:usedLen = usedLen + 8case reflect.Uint64:usedLen = usedLen + 8case reflect.Float32:usedLen = usedLen + 4case reflect.Float64:usedLen = usedLen + 8case reflect.String://待处理case reflect.Slice://待处理case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {l, err := refUnmarshal(data[usedLen:], v.Field(i), v.Type().Field(i), streLen)if err != nil {return 0, err}usedLen = usedLen + l}}return usedLen, nil
}

解析到这个地方我们发现，我们又遇到了另外的一个问题：我们没有办法单纯的通过类型来获取到string和struct的长度，而且我们还必须处理这两个类型，因为这两个类型在协议处理中是很常见的。既然单纯的通过类型无法判断长度，我们就要借助tag了。我们尝试着在string和slice上设定tag来解决这个问题。但是tag是属于结构体的，只有结构体内部元素才能拥有tag，而且我们不能通过元素本身获取tag，必须通过上层的struct的type才能获取到，所以此时我们入参还要加入一个通过结构体type获取到的对应字段reflect.StructField：

func refUnmarshal(data []byte, v reflect.Value, tag reflect.StructField) (int, error) {var usedLen int = 0if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:usedLen = usedLen + 1case reflect.Uint8:usedLen = usedLen + 1case reflect.Int16:usedLen = usedLen + 2case reflect.Uint16:usedLen = usedLen + 2case reflect.Int32:usedLen = usedLen + 4case reflect.Uint32:usedLen = usedLen + 4case reflect.Int64:usedLen = usedLen + 8case reflect.Uint64:usedLen = usedLen + 8case reflect.Float32:usedLen = usedLen + 4case reflect.Float64:usedLen = usedLen + 8case reflect.String:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {//} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}usedLen = usedLen + int(lens)case reflect.Slice:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {//} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}usedLen = usedLen + int(lens)case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {l, err := refUnmarshal(data[usedLen:], v.Field(i), v.Type().Field(i))if err != nil {return 0, err}usedLen = usedLen + l}}return usedLen, nil
}

这样我们就能过获取到所有的字段对应的长度了，这个很关键。然后我们只需要根据对应的长度，从data中填充对应的数据值即可

func refUnmarshal(data []byte, v reflect.Value, tag reflect.StructField) (int, error) {var usedLen int = 0if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:v.SetInt(int64(data[0]))usedLen = usedLen + 1case reflect.Uint8:v.SetUint(uint64(data[0]))usedLen = usedLen + 1case reflect.Int16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}v.SetInt(int64(Bytes2Word(data)))usedLen = usedLen + 2case reflect.Uint16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}v.SetUint(uint64(Bytes2Word(data)))usedLen = usedLen + 2case reflect.Int32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}v.SetInt(int64(Bytes2DWord(data)))usedLen = usedLen + 4case reflect.Uint32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}v.SetUint(uint64(Bytes2DWord(data)))usedLen = usedLen + 4case reflect.Int64:v.SetInt(64)usedLen = usedLen + 8case reflect.Uint64:v.SetUint(64)usedLen = usedLen + 8case reflect.Float32:v.SetFloat(32.23)usedLen = usedLen + 4case reflect.Float64:v.SetFloat(64.46)usedLen = usedLen + 8case reflect.String:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {//} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}if len(data) < int(lens) {return 0, fmt.Errorf("data to short")}v.SetString(string(data[:lens]))usedLen = usedLen + int(lens)case reflect.Slice:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {//} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}v.SetBytes(data[:lens])usedLen = usedLen + int(lens)case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {l, err := refUnmarshal(data[usedLen:], v.Field(i), v.Type().Field(i))if err != nil {return 0, err}usedLen = usedLen + l}}return usedLen, nil
}

一个基本的Unmarshal函数就完成了。但是这个处理是比较理想的，在实际中可能会存在这样的一种情况：在一个协议中有若干字段，其他的字段都是固定长度，只有一个字段是长度可变的，而这个可变长度的计算是由总体长度-固定长度来计算出来的。在这种情况下，我们需要提前计算出已知字段的固定长度，然后用data长度-固定长度，得到唯一的可变字段的长度。所以我现在要有一个获取这个结构的有效长度的函数。前面的Unmarshal内部已经可以获取到每个字段的长度了，我们只需要把这个函数简单改造一下就行了：

func RequireLen(v interface{}) (int, error) {rv := reflect.ValueOf(v)if rv.Kind() != reflect.Ptr || rv.IsNil() {return 0, fmt.Errorf("error")}return refRequireLen(reflect.ValueOf(v), reflect.StructField{})
}func refRequireLen(v reflect.Value, tag reflect.StructField) (int, error) {var usedLen int = 0if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:usedLen = usedLen + 1case reflect.Uint8:usedLen = usedLen + 1case reflect.Int16:usedLen = usedLen + 2case reflect.Uint16:usedLen = usedLen + 2case reflect.Int32:usedLen = usedLen + 4case reflect.Uint32:usedLen = usedLen + 4case reflect.Int64:usedLen = usedLen + 8case reflect.Uint64:usedLen = usedLen + 8case reflect.Float32:usedLen = usedLen + 4case reflect.Float64:usedLen = usedLen + 8case reflect.String:strLen := tag.Tag.Get("len")if strLen == "" {return 0, nil}lens, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}usedLen = usedLen + int(lens)case reflect.Slice:strLen := tag.Tag.Get("len")if strLen == "" {return 0, nil}lens, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}usedLen = usedLen + int(lens)case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {l, err := refRequireLen(v.Field(i), v.Type().Field(i))if err != nil {return 0, err}usedLen = usedLen + l}}return usedLen, nil
}

这样我们就可以实现一个完整的Unmarshal

func Unmarshal(data []byte, v interface{}) (int, error) {rv := reflect.ValueOf(v)if rv.Kind() != reflect.Ptr || rv.IsNil() {return 0, fmt.Errorf("error")}lens, err := RequireLen(v)if err != nil {return 0, err}if len(data) < lens {return 0, fmt.Errorf("data too short")}return refUnmarshal(data, reflect.ValueOf(v), reflect.StructField{}, len(data)-lens)
}func refUnmarshal(data []byte, v reflect.Value, tag reflect.StructField, streLen int) (int, error) {var usedLen int = 0if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:v.SetInt(int64(data[0]))usedLen = usedLen + 1case reflect.Uint8:v.SetUint(uint64(data[0]))usedLen = usedLen + 1case reflect.Int16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}v.SetInt(int64(Bytes2Word(data)))usedLen = usedLen + 2case reflect.Uint16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}v.SetUint(uint64(Bytes2Word(data)))usedLen = usedLen + 2case reflect.Int32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}v.SetInt(int64(Bytes2DWord(data)))usedLen = usedLen + 4case reflect.Uint32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}v.SetUint(uint64(Bytes2DWord(data)))usedLen = usedLen + 4case reflect.Int64:v.SetInt(64)usedLen = usedLen + 8case reflect.Uint64:v.SetUint(64)usedLen = usedLen + 8case reflect.Float32:v.SetFloat(32.23)usedLen = usedLen + 4case reflect.Float64:v.SetFloat(64.46)usedLen = usedLen + 8case reflect.String:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {lens = streLen} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}if len(data) < int(lens) {return 0, fmt.Errorf("data to short")}v.SetString(string(data[:lens]))usedLen = usedLen + int(lens)case reflect.Slice:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {lens = streLen} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}v.SetBytes(data[:lens])usedLen = usedLen + int(lens)case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {l, err := refUnmarshal(data[usedLen:], v.Field(i), v.Type().Field(i), streLen)if err != nil {return 0, err}usedLen = usedLen + l}}return usedLen, nil
}

理解了上面的流程，Marshal就就很好写了，只是复制过程反过来就行了。这其中还有一些小的转换逻辑将字节数组转换成多字节整形：Bytes2Word、Word2Bytes、Bytes2DWord、Dword2Bytes。这类转换都使用大端格式处理。完整代码如下：

package codecimport ("fmt""reflect""strconv"
)func RequireLen(v interface{}) (int, error) {rv := reflect.ValueOf(v)if rv.Kind() != reflect.Ptr || rv.IsNil() {return 0, fmt.Errorf("error")}return refRequireLen(reflect.ValueOf(v), reflect.StructField{})
}func Unmarshal(data []byte, v interface{}) (int, error) {rv := reflect.ValueOf(v)if rv.Kind() != reflect.Ptr || rv.IsNil() {return 0, fmt.Errorf("error")}lens, err := RequireLen(v)if err != nil {return 0, err}if len(data) < lens {return 0, fmt.Errorf("data too short")}return refUnmarshal(data, reflect.ValueOf(v), reflect.StructField{}, len(data)-lens)
}func Marshal(v interface{}) ([]byte, error) {rv := reflect.ValueOf(v)if rv.Kind() != reflect.Ptr || rv.IsNil() {return []byte{}, fmt.Errorf("error")}return refMarshal(reflect.ValueOf(v), reflect.StructField{})
}func refRequireLen(v reflect.Value, tag reflect.StructField) (int, error) {var usedLen int = 0if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:usedLen = usedLen + 1case reflect.Uint8:usedLen = usedLen + 1case reflect.Int16:usedLen = usedLen + 2case reflect.Uint16:usedLen = usedLen + 2case reflect.Int32:usedLen = usedLen + 4case reflect.Uint32:usedLen = usedLen + 4case reflect.Int64:usedLen = usedLen + 8case reflect.Uint64:usedLen = usedLen + 8case reflect.Float32:usedLen = usedLen + 4case reflect.Float64:usedLen = usedLen + 8case reflect.String:strLen := tag.Tag.Get("len")if strLen == "" {return 0, nil}lens, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}usedLen = usedLen + int(lens)case reflect.Slice:strLen := tag.Tag.Get("len")if strLen == "" {return 0, nil}lens, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}usedLen = usedLen + int(lens)case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {l, err := refRequireLen(v.Field(i), v.Type().Field(i))if err != nil {return 0, err}usedLen = usedLen + l}}return usedLen, nil
}func refUnmarshal(data []byte, v reflect.Value, tag reflect.StructField, streLen int) (int, error) {var usedLen int = 0if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:v.SetInt(int64(data[0]))usedLen = usedLen + 1case reflect.Uint8:v.SetUint(uint64(data[0]))usedLen = usedLen + 1case reflect.Int16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}v.SetInt(int64(Bytes2Word(data)))usedLen = usedLen + 2case reflect.Uint16:if len(data) < 2 {return 0, fmt.Errorf("data to short")}v.SetUint(uint64(Bytes2Word(data)))usedLen = usedLen + 2case reflect.Int32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}v.SetInt(int64(Bytes2DWord(data)))usedLen = usedLen + 4case reflect.Uint32:if len(data) < 4 {return 0, fmt.Errorf("data to short")}v.SetUint(uint64(Bytes2DWord(data)))usedLen = usedLen + 4case reflect.Int64:v.SetInt(64)usedLen = usedLen + 8case reflect.Uint64:v.SetUint(64)usedLen = usedLen + 8case reflect.Float32:v.SetFloat(32.23)usedLen = usedLen + 4case reflect.Float64:v.SetFloat(64.46)usedLen = usedLen + 8case reflect.String:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {lens = streLen} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}if len(data) < int(lens) {return 0, fmt.Errorf("data to short")}v.SetString(string(data[:lens]))usedLen = usedLen + int(lens)case reflect.Slice:strLen := tag.Tag.Get("len")var lens int = 0if strLen == "" {lens = streLen} else {lens64, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return 0, err}lens = int(lens64)}v.SetBytes(data[:lens])usedLen = usedLen + int(lens)case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {l, err := refUnmarshal(data[usedLen:], v.Field(i), v.Type().Field(i), streLen)if err != nil {return 0, err}usedLen = usedLen + l}}return usedLen, nil
}func refMarshal(v reflect.Value, tag reflect.StructField) ([]byte, error) {data := make([]byte, 0)if v.Kind() == reflect.Ptr {v = v.Elem()}switch v.Kind() {case reflect.Int8:data = append(data, byte(v.Int()))case reflect.Uint8:data = append(data, byte(v.Uint()))case reflect.Int16:temp := Word2Bytes(uint16(v.Int()))data = append(data, temp...)case reflect.Uint16:temp := Word2Bytes(uint16(v.Uint()))data = append(data, temp...)case reflect.Int32:temp := Dword2Bytes(uint32(v.Int()))data = append(data, temp...)case reflect.Uint32:temp := Dword2Bytes(uint32(v.Uint()))data = append(data, temp...)case reflect.String:strLen := tag.Tag.Get("len")lens, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return []byte{}, err}if int(lens) > v.Len() {zeroSlice := make([]byte, int(lens)-v.Len())data = append(data, zeroSlice...)}data = append(data, v.String()...)case reflect.Slice:strLen := tag.Tag.Get("len")lens, err := strconv.ParseInt(strLen, 10, 0)if err != nil {return []byte{}, err}if int(lens) > v.Len() {zeroSlice := make([]byte, int(lens)-v.Len())data = append(data, zeroSlice...)}data = append(data, v.Bytes()...)case reflect.Struct:fieldCount := v.NumField()for i := 0; i < fieldCount; i++ {fmt.Println(v.Field(i).Type().String())d, err := refMarshal(v.Field(i), v.Type().Field(i))if err != nil {return []byte{}, err}data = append(data, d...)}}return data, nil
}func Bytes2Word(data []byte) uint16 {if len(data) < 2 {return 0}return (uint16(data[0]) << 8) + uint16(data[1])
}func Word2Bytes(data uint16) []byte {buff := make([]byte, 2)buff[0] = byte(data >> 8)buff[1] = byte(data)return buff
}func Bytes2DWord(data []byte) uint32 {if len(data) < 4 {return 0}return (uint32(data[0]) << 24) + (uint32(data[1]) << 16) + (uint32(data[2]) << 8) + uint32(data[3])
}func Dword2Bytes(data uint32) []byte {buff := make([]byte, 4)buff[0] = byte(data >> 24)buff[1] = byte(data >> 16)buff[2] = byte(data >> 8)buff[3] = byte(data)return buff
}