Compare commits

...

18 Commits

Author SHA1 Message Date
fc340b9417 Merge pull request '已知问题修复' (#7) from fetature/fix_parent_tag into master
Reviewed-on: #7
2025-04-29 14:58:00 +08:00
141a376d19 修复嵌套对象json tag设置错误问题 2025-04-29 14:57:15 +08:00
d139b409fd 修复数组未判断长度导致索引越界的BUG 2025-04-29 11:37:13 +08:00
7e464374bb remove debug code 2025-04-28 21:11:18 +08:00
6b9942a175 Merge pull request '支持从json动态生成结构体,并转化为指定格式' (#6) from feature/support_json into master
Reviewed-on: #6
2025-04-28 21:05:25 +08:00
e4c0f9abf3 数组支持递归解析 2025-04-28 18:18:04 +08:00
b4932bf7cd 支持通过配置传入XmlName, 默认值 XmlData 2025-04-28 17:40:40 +08:00
f9024b9498 支持从json -> xml的转换 2025-04-28 17:34:23 +08:00
cd85d80da6 Merge pull request '未指定tag时, 默认支持 json/xml/yaml/toml/ini' (#5) from feature/upgrade_tag into master
Reviewed-on: #5
2025-04-28 10:42:09 +08:00
103f3b38ba 未指定tag时, 默认支持 json/xml/yaml/toml/ini 2025-04-28 10:41:42 +08:00
5d4788e0fd 更新README 2025-04-01 15:35:34 +08:00
c8d891c438 update go mod 2025-04-01 14:41:13 +08:00
1b217c0e4f Merge pull request '支持传入的跟结构就是数组' (#4) from feature/fix_root_array into master
Reviewed-on: #4
2025-03-23 20:55:06 +08:00
bc91937116 支持传入的跟结构就是数组 2025-03-23 20:54:35 +08:00
0b609e8e26 Merge pull request '修复嵌套结构体 + 嵌套数组的BUG' (#3) from feature/unit_test into master
Reviewed-on: #3
2025-03-23 18:00:55 +08:00
7507cf8fe1 增加单元测试 2025-03-23 17:51:13 +08:00
07fb74b68b 修复嵌套结构体 + 嵌套数组的BUG 2025-03-23 15:11:46 +08:00
9ef4123f38 整理代码结构, 增加普通结构体单元测试 2025-03-23 14:20:49 +08:00
12 changed files with 701 additions and 162 deletions

113
README.md
View File

@ -1,5 +1,7 @@
# Golang 运行时动态结构体
## 简介
主要提供在运行时动态生成结构体的能力, 同时支持合并多个已有结构体,生成一个新的结构体
主要功能如下:
@ -11,53 +13,82 @@
* 修改已存在字段的类型以及Tag标签
* 读取动态结构体字段的Helper
* 动态结构体的值, 解析到一个已定义的结构体中
* 动态结构体生成slice或者map实例
* 动态结构体生成slice或者map实例
* 支持嵌套的结构体
* 支持N维数组
* 支持嵌套的结构体数组
## 开发背景
开发个人网关项目时, 面临一个问题: **`如何验证网关请求参数的合法性?`** 在Golang生态中,比较知名的参数验证库是 [https://github.com/go-playground/validator](go-playground/validator/v10)。 其支持的参数验证方式如下:
- 单个值按照指定规则进行验证, 无上下文信息
- map数据验证, 无上下文信息, 可以看做是单个值的验证的批量版本
- 结构体数据验证, 支持上下文信息, 可以跨结构体字段进行关联验证
网关参数验证的需求背景如下:
- 基础的参数验证, 如取值范围、长度等
- 数据格式校验IP地址、URL、邮箱、手机号等
- 数据关联校验两个字段的值必须相等、A字段不存在,则B字段必须存在等
- ......
综合数据验证库提供的验证能力, 以及业务场景的复杂性, 需要使用结构体数据进行参数验证。那么, 此时则面临另外一个问题: 不同的网关接口是有不同的参数结构的, 如何在运行时动态生成结构体呢? 这就需要用到本文所介绍的动态结构体生成能力。
## 快速开始
```bash
# 安装
go get -v -u github.com/liuxiaobopro/golang-dynamic-struct
```
## 使用示例
```go
package main
import (
"encoding/json"
"fmt"
"log"
"git.zhangdeman.cn/zhangdeman/dynamic-struct"
dynamicstruct "git.zhangdeman.cn/zhangdeman/dynamic-struct"
"github.com/go-playground/validator/v10"
"fmt"
)
func main() {
instance := NewStruct().
AddField("Integer", "", 0, `json:"int"`, false).
AddField("Text", "", "", `json:"someText"`, false).
AddField("Float", "", 0.0, `json:"double"`, false).
AddField("Boolean", "", false, "", false).
AddField("Slice", "", []int{}, "", false).
AddField("Anonymous", "", "", `json:"-"`, false).
Build().
New()
data := []byte(`
{
"int": 123,
"someText": "example",
"double": 123.45,
"Boolean": true,
"Slice": [1, 2, 3],
"Anonymous": "avoid to read"
}
`)
err := json.Unmarshal(data, &instance)
if err != nil {
log.Fatal(err)
}
data, err = json.Marshal(instance)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(data))
// Out:
// {"int":123,"someText":"example","double":123.45,"Boolean":true,"Slice":[1,2,3]}
// 动态生成一个结构体
ds := dynamicstruct.NewStruct()
ds = ds.AddField("name", "", `json:"name" validate:"required"`)
ds = ds.AddField("age", 0, `json:"age" validate:"gte=0,lte=130"`)
ds = ds.AddField("email", "", `json:"email" validate:"required,email"`)
// 生成结构体实例
user := ds.Build().New()
userTestData := `{
"name": "张三",
"age": 180,
"email": "test@qq.com",
}`
validatorInstance = validator.New()
validatorInstance.SetTagName("validate")
err := validatorInstance.Struct(user)
fmt.Println(err)
}
```
## 实现原理
无论任何语言, 但凡出现 **`动态`** 这一类字眼, 实现原理一定脱离不了 **`反射`** , 且 **`反射是核心实现技术`** , 本库中的实现原理也不例外, 也是基于反射进行实现的。核心逻辑是利用 **`reflect.New`** 这一泛实例化数据的机制进行实现. 确定原理方向, **`抽象的需求运行时动态生成不定结构的结构体`** 已经转化为具体功能诉求, **`如何利用reflect.New这一机制, 动态实例化结构体`**
## 具体实现
核心就是一句代码:
```go
structFields := []reflect.StructField
reflect.New(reflect.StructOf(structFields))
```
其中, **`structFields`** 是一个 **`一个结构体字段配置列表,其可配置的属性具体查看即可`**, **`reflect.StructOf`** 则是根据结构体字段配置反射出结构体类型, 利用 **`reflect.New`** 这一机制, 可以动态实例化一个 **`结构体`**。
核心逻辑就是这么简单, 但是, 动态生成结构体的过程中, 还需要考虑到很多其他的问题, 如:
- 如何动态生成嵌套的结构体
- 如何动态生成嵌套的结构体数组
- 如何动态生成N维数组
- 如何动态生成结构体数组
其实上述逻辑无非是基于最原始的基础逻辑, **`递归进行实现`** , 所以, 具体实现细节, 可以查看源码。

48
abstract.go Normal file
View File

@ -0,0 +1,48 @@
// Package dynamicstruct ...
//
// Description : dynamicstruct ...
//
// Author : go_developer@163.com<白茶清欢>
//
// Date : 2025-03-23 13:34
package dynamicstruct
// IBuilder 运行时动态生成结构体的接口约束
type IBuilder interface {
// AddField 添加结构体字段
AddField(name string, pkg string, typ any, tag string, anonymous bool) IBuilder
// RemoveField 移除指定名称的结构体字段
RemoveField(name string) IBuilder
// HasField 检测指定名称的结构体字段是否存在
HasField(name string) bool
// GetField 根据名称获取结构体字段定义
GetField(name string) IFieldConfig
// Build 返回动态定义的结构体.
Build() IDynamicStruct
}
// IFieldConfig 结构体字段的定义.
type IFieldConfig interface {
// SetType 设置字段类型.
SetType(typ any) IFieldConfig
// SetTag 设置字段 tag.
SetTag(tag string) IFieldConfig
// GetType 获取类型
GetType() any
// GetTag 获取tag
GetTag() string
}
// IDynamicStruct contains defined dynamic struct.
// This definition can't be changed anymore, once is built.
// It provides a method for creating new instances of same defintion.
type IDynamicStruct interface {
// New 获取结构体实例, 所有字段值均为对应类型的初始零值
New() any
// NewSliceOfStructs slice实例化
NewSliceOfStructs() any
// NewMapOfStructs map 或者 struct实例化
NewMapOfStructs(key any) any
}

View File

@ -1,79 +1,29 @@
package dynamicstruct
import (
"fmt"
"git.zhangdeman.cn/zhangdeman/wrapper"
"reflect"
"strings"
)
type (
// Builder 运行时动态生成结构体的接口约束
Builder interface {
// AddField 添加结构体字段
AddField(name string, pkg string, typ any, tag string, anonymous bool) Builder
// RemoveField 移除指定名称的结构体字段
RemoveField(name string) Builder
// HasField 检测指定名称的结构体字段是否存在
HasField(name string) bool
// GetField 根据名称获取结构体字段定义
GetField(name string) FieldConfig
// Build 返回动态定义的结构体.
Build() DynamicStruct
}
const tagTpl = `json:"{TAG_NAME}" xml:"{TAG_NAME}" toml:"{TAG_NAME}" yaml:"{TAG_NAME}" ini:"{TAG_NAME}"`
// FieldConfig 结构体字段的定义.
FieldConfig interface {
// SetType 设置字段类型.
SetType(typ any) FieldConfig
// SetTag 设置字段 tag.
SetTag(tag string) FieldConfig
}
type nestedStruct struct {
Field string
Builder IBuilder
Tag string
}
// DynamicStruct contains defined dynamic struct.
// This definition can't be changed anymore, once is built.
// It provides a method for creating new instances of same defintion.
DynamicStruct interface {
// New 获取结构体实例, 所有字段值均为对应类型的初始零值
New() any
// NewSliceOfStructs slice实例化
NewSliceOfStructs() any
// NewMapOfStructs map 或者 struct实例化
NewMapOfStructs(key any) any
}
nestedStruct struct {
Field string
Builder Builder
Tag string
}
builderImpl struct {
fields []*fieldConfigImpl
nestedStructTable map[string]nestedStruct // 嵌套结构体的定义, 父级路径 => 父级路径下挂接的子路径
maxFieldDeep int // 字段嵌套最大深度
structTagTable map[string]string // 字段路径 => json tag最高优先级, 没传的时候会使用AddField的tag, 也为空使用手搓json tag
}
fieldConfigImpl struct {
name string
pkg string
typ any
tag string
anonymous bool
}
dynamicStructImpl struct {
structFields []reflect.StructField
definition reflect.Type
}
)
type builderImpl struct {
fields []*fieldConfigImpl
nestedStructTable map[string]nestedStruct // 嵌套结构体的定义, 父级路径 => 父级路径下挂接的子路径
maxFieldDeep int // 字段嵌套最大深度
structTagTable map[string]string // 字段路径 => json tag最高优先级, 没传的时候会使用AddField的tag, 也为空使用手搓json tag
}
// NewStruct 获取builder实例
// 传入的tag映射表: 字段路径 => json tag最高优先级, 没传的时候会使用AddField的tag, 也为空使用手搓json tag
func NewStruct(structTagTable map[string]string) Builder {
func NewStruct(structTagTable map[string]string) IBuilder {
return &builderImpl{
fields: []*fieldConfigImpl{},
nestedStructTable: make(map[string]nestedStruct),
@ -82,12 +32,12 @@ func NewStruct(structTagTable map[string]string) Builder {
}
// ExtendStruct 基于已有结构体, 生成动态结构体(相当于继承指定的结构体属性)
func ExtendStruct(value ...any) Builder {
func ExtendStruct(value ...any) IBuilder {
return MergeStructs(value...)
}
// MergeStructs 多个结构体合并成一个动态结构体
func MergeStructs(values ...any) Builder {
func MergeStructs(values ...any) IBuilder {
builder := NewStruct(map[string]string{})
for _, value := range values {
@ -111,14 +61,17 @@ func (b *builderImpl) parseNestedField(fieldName string) ([]string, bool) {
}
// AddField 添加结构体字段
func (b *builderImpl) AddField(name string, pkg string, typ any, tag string, anonymous bool) Builder {
// 瞎话线转驼峰, 传入的name实际对应序列化时的json tag
func (b *builderImpl) AddField(name string, pkg string, typ any, tag string, anonymous bool) IBuilder {
// 下划线转驼峰, 传入的name实际对应序列化时的json tag
// name = wrapper.String(name).SnakeCaseToCamel()
if cfgTag, exist := b.structTagTable[name]; exist && len(cfgTag) > 0 {
tag = cfgTag
} else {
if len(tag) == 0 {
tag = fmt.Sprintf(`json:"%s"`, name)
arr := strings.Split(strings.TrimSuffix(name, ".[]"), ".")
// 没指定tag, 字段名称作为tag名称
tag = strings.ReplaceAll(tagTpl, "{TAG_NAME}", arr[len(arr)-1])
// tag = fmt.Sprintf(`json:"%s"`, name)
}
}
// 判断是否嵌套结构体
@ -169,7 +122,8 @@ func (b *builderImpl) addNestedField(nameArr []string, pkg string, typ any, tag
parentName := strings.Join(nameArr[:i], ".")
parentJsonTag := nameArr[i-1]
parentFieldName := wrapper.String(parentJsonTag).SnakeCaseToCamel()
fieldTag := fmt.Sprintf(`json:"%s"`, parentJsonTag)
// fieldTag := fmt.Sprintf(`json:"%s"`, parentJsonTag)
fieldTag := strings.ReplaceAll(tagTpl, "{TAG_NAME}", parentJsonTag)
if tagCfg, exist := b.structTagTable[parentName]; exist && len(tagCfg) > 0 {
fieldTag = tagCfg
}
@ -191,7 +145,7 @@ func (b *builderImpl) addNestedField(nameArr []string, pkg string, typ any, tag
}
// RemoveField 根据名称移除结构体字段
func (b *builderImpl) RemoveField(name string) Builder {
func (b *builderImpl) RemoveField(name string) IBuilder {
newFieldList := make([]*fieldConfigImpl, 0)
for i := range b.fields {
if b.fields[i].name == name {
@ -214,7 +168,7 @@ func (b *builderImpl) HasField(name string) bool {
}
// GetField 根据名称获取字段配置, 不存在, 返回nil
func (b *builderImpl) GetField(name string) FieldConfig {
func (b *builderImpl) GetField(name string) IFieldConfig {
for i := range b.fields {
if b.fields[i].name == name {
return b.fields[i]
@ -224,11 +178,14 @@ func (b *builderImpl) GetField(name string) FieldConfig {
}
// Build 构建动态结构体
func (b *builderImpl) Build() DynamicStruct {
func (b *builderImpl) Build() IDynamicStruct {
// 按照嵌套深度, 进行一次回溯处理
for deep := b.maxFieldDeep - 1; deep > 0; deep-- {
// 嵌套数据结构
for parentIndex, builderCfg := range b.nestedStructTable {
if parentIndex == ArrayRootFlag {
continue
}
parentNameArr := strings.Split(parentIndex, ".")
if len(parentNameArr) != deep {
// 从深度最深处向上处理
@ -240,7 +197,7 @@ func (b *builderImpl) Build() DynamicStruct {
} else {
// 处理数组
arrDeep := 0
newParentNameArr := []string{}
var newParentNameArr []string
for i := len(parentNameArr) - 1; i >= 0; i-- {
if parentNameArr[i] != ArraySplit {
newParentNameArr = parentNameArr[:i+1]
@ -252,23 +209,40 @@ func (b *builderImpl) Build() DynamicStruct {
}
if arrDeep > 0 {
// 数组嵌套数组配置
val := reflect.ValueOf(builderCfg.Builder.Build().New()).Elem().Interface()
val := reflect.ValueOf(builderCfg.Builder.Build().New()).Interface()
for i := 0; i < arrDeep; i++ {
val = reflect.New(reflect.SliceOf(reflect.TypeOf(val))).Interface()
val = reflect.New(reflect.SliceOf(reflect.TypeOf(val).Elem())).Interface()
// 删除数组记录
arrPath := strings.Join(parentNameArr[:len(parentNameArr)-i], ".")
delete(b.nestedStructTable, arrPath)
}
// 解除指针引用
val = reflect.ValueOf(val).Elem().Interface()
newParamIndex := strings.Join(newParentNameArr, ".")
newParentIndex := strings.Join(newParentNameArr, ".")
isTopIndex := len(newParentNameArr) == 1
if isTopIndex {
// 顶层结构, 数组类型不存在还有其他属性情况, 直接追加字段, 并移除嵌套表里的定义
b.AddField(b.nestedStructTable[newParamIndex].Field, "", val, b.nestedStructTable[newParamIndex].Tag, false)
delete(b.nestedStructTable, newParamIndex)
b.AddField(b.nestedStructTable[newParentIndex].Field, "", val, b.nestedStructTable[newParentIndex].Tag, false)
// b.nestedStructTable[newParamIndex].Builder.AddField(b.nestedStructTable[newParamIndex].Field, "", val, b.nestedStructTable[newParamIndex].Tag, false)
} else {
// 非顶层结构, 再上探一级
b.nestedStructTable[strings.Join(newParentNameArr[:len(newParentNameArr)-1], ".")].Builder.AddField(b.nestedStructTable[newParamIndex].Field, "", val, b.nestedStructTable[newParamIndex].Tag, false)
newParentName := ""
fieldName := ""
if len(newParentNameArr) == 0 {
// 说明传入的直接就是数组
newParentName = ArrayRootFlag
fieldName = ArrayRootFlag
b.nestedStructTable[newParentName] = nestedStruct{
Field: fieldName,
Builder: NewStruct(b.structTagTable),
Tag: "",
}
b.nestedStructTable[newParentName].Builder.AddField(b.nestedStructTable[newParentName].Field, "", reflect.ValueOf(val).Elem().Interface(), b.nestedStructTable[newParentName].Tag, false)
} else {
newParentName = strings.Join(newParentNameArr[:len(newParentNameArr)-1], ".")
b.nestedStructTable[newParentName].Builder.AddField(b.nestedStructTable[newParentIndex].Field, "", reflect.ValueOf(val).Elem().Interface(), b.nestedStructTable[newParentIndex].Tag, false)
}
}
// 嵌套结构中删除数组字段
delete(b.nestedStructTable, newParentIndex)
} else {
// 非数组
// (非顶层) 父级结构存在, 将其追加到父级结构中即可, 向前看一步即为父级结构
@ -277,11 +251,20 @@ func (b *builderImpl) Build() DynamicStruct {
}
}
}
if nestedCfg, exist := b.nestedStructTable[ArrayRootFlag]; exist {
// 说明是传入的根路径就是数组
return &dynamicStructImpl{
structFields: nil,
definition: reflect.TypeOf(nestedCfg.Builder.GetField(ArrayRootFlag).GetType()),
}
}
// 一级字段属性
var structFields []reflect.StructField
for _, field := range b.fields {
if strings.Contains(field.name, ArraySplit) {
// TODO : 临时过滤, 后续需要在正确逻辑处正确移除
// 去除数组路径
continue
}
structFields = append(structFields, reflect.StructField{
@ -297,31 +280,3 @@ func (b *builderImpl) Build() DynamicStruct {
definition: reflect.StructOf(structFields),
}
}
// SetType 设置字段类型
func (f *fieldConfigImpl) SetType(typ any) FieldConfig {
f.typ = typ
return f
}
// SetTag 设置字段标签
func (f *fieldConfigImpl) SetTag(tag string) FieldConfig {
f.tag = tag
return f
}
// New 创建动态结构体实例
func (ds *dynamicStructImpl) New() any {
// 不要指针,全部解引用
return reflect.New(ds.definition).Interface()
}
// NewSliceOfStructs 创建动态结构体切片实例
func (ds *dynamicStructImpl) NewSliceOfStructs() any {
return reflect.New(reflect.SliceOf(ds.definition)).Interface()
}
// NewMapOfStructs 创建动态结构体map实例
func (ds *dynamicStructImpl) NewMapOfStructs(key any) any {
return reflect.New(reflect.MapOf(reflect.Indirect(reflect.ValueOf(key)).Type(), ds.definition)).Interface()
}

View File

@ -9,19 +9,14 @@ package dynamicstruct
import (
"encoding/json"
"fmt"
"reflect"
"testing"
. "github.com/smartystreets/goconvey/convey"
)
func Test_dynamicStructImpl_New(t *testing.T) {
/*func Test_dynamicStructImpl_New(t *testing.T) {
instance := NewStruct(map[string]string{}).
/*AddField("Integer", "", 0, `json:"int"`, false).
AddField("Text", "", "", `json:"someText"`, false).
AddField("Float", "", 0.0, `json:"double"`, false).
AddField("Boolean", "", false, "", false).
AddField("Slice", "", []int{}, "", false).
AddField("Anonymous", "", "", `json:"-"`, false).*/
AddField("user.base.age", "", 20, `json:"age"`, false).
AddField("user.base.name", "", "", `json:"name"`, false).
AddField("user.job.address", "", "", `json:"address"`, false).
@ -52,3 +47,221 @@ func Test_dynamicStructImpl_New(t *testing.T) {
fmt.Println(string(valByte))
// fmt.Println(reflect.ValueOf(val).Elem().FieldByName("Integer").Interface())
}
*/
// TestStruct : 测试结构体
func TestStruct(t *testing.T) {
Convey("普通结构体,无嵌套", t, func() {
data := []byte(`
{
"age": 123,
"name": "example",
"double": 123.45,
"tag_map": "ttt",
"bool": true,
"slice": [1, 2, 3]
}
`)
instance := NewStruct(map[string]string{
"test_tag_map": `json:"tag_map"`,
}).
AddField("age", "", 0, `json:"age"`, false).
AddField("name", "", "", `json:"name"`, false).
AddField("test_tag_map", "", "", "", false).
AddField("double", "", 0.0, `json:"double"`, false).
AddField("bool", "", false, "", false).
AddField("slice", "", []int{}, `json:"slice"`, false).Build()
val := instance.New()
err := json.Unmarshal(data, &val)
So(err, ShouldBeNil)
reflectType := reflect.TypeOf(val)
// 需要是结构体类型
So(reflectType.Elem().Kind(), ShouldEqual, reflect.Struct)
// 验证数据
// 数据序列化成功
targetByte, targetErr := json.Marshal(val)
So(targetErr, ShouldBeNil)
// 数据转map成功
var res map[string]any
targetUnmarshalErr := json.Unmarshal(targetByte, &res)
So(targetUnmarshalErr, ShouldBeNil)
// 验证数据
So(len(res), ShouldEqual, 6)
So(res["age"], ShouldEqual, float64(123))
So(res["name"], ShouldEqual, "example")
So(res["tag_map"], ShouldEqual, "ttt")
So(res["double"], ShouldEqual, 123.45)
So(res["bool"], ShouldEqual, true)
So(res["slice"], ShouldEqual, []any{float64(1), float64(2), float64(3)})
})
}
// TestNestedStruct : 测试嵌套结构体
func TestNestedStruct(t *testing.T) {
Convey("嵌套结构体", t, func() {
data := []byte(`
{
"user": {"info": {
"age": 123,
"name": "example",
"double": 123.45,
"bool": true,
"slice": [1, 2, 3]
}}}
`)
instance := NewStruct(map[string]string{
"test_tag_map": `json:"tag_map"`,
}).
AddField("user.info.age", "", 0, `json:"age"`, false).
AddField("user.info.name", "", "", `json:"name"`, false).
Build()
val := instance.New()
err := json.Unmarshal(data, &val)
So(err, ShouldBeNil)
reflectType := reflect.TypeOf(val)
// 需要是结构体类型
So(reflectType.Elem().Kind(), ShouldEqual, reflect.Struct)
// 验证数据
// 数据序列化成功
targetByte, targetErr := json.Marshal(val)
So(targetErr, ShouldBeNil)
// 数据转map成功
var res map[string]any
targetUnmarshalErr := json.Unmarshal(targetByte, &res)
So(targetUnmarshalErr, ShouldBeNil)
So(len(res), ShouldEqual, 1)
So(res["user"], ShouldNotBeEmpty)
userMap, ok := res["user"].(map[string]any)
So(ok, ShouldBeTrue)
So(len(userMap), ShouldEqual, 1)
userInfoMap, ok := userMap["info"].(map[string]any)
So(ok, ShouldBeTrue)
So(userInfoMap["age"], ShouldEqual, float64(123))
So(userInfoMap["name"], ShouldEqual, "example")
})
}
// TestStruct : 测试数组
func TestArray(t *testing.T) {
Convey("嵌套结构体带多级数组", t, func() {
data := []byte(`
{
"user": {"list":[[[{
"age": 123,
"name": "example",
"double": 123.45,
"bool": true,
"slice": [1, 2, 3]
}]]]}
}
`)
instance := NewStruct(map[string]string{
"test_tag_map": `json:"tag_map"`,
}).
AddField("user.list.[].[].[].age", "", 0, `json:"age"`, false).
AddField("user.list.[].[].[].name", "", "", `json:"name"`, false).
Build()
val := instance.New()
err := json.Unmarshal(data, &val)
So(err, ShouldBeNil)
reflectType := reflect.TypeOf(val)
// 需要是结构体类型
So(reflectType.Elem().Kind(), ShouldEqual, reflect.Struct)
// 验证数据
// 数据序列化成功
targetByte, targetErr := json.Marshal(val)
So(targetErr, ShouldBeNil)
// 数据转map成功
var res map[string]map[string][][][]any
targetUnmarshalErr := json.Unmarshal(targetByte, &res)
So(targetUnmarshalErr, ShouldBeNil)
So(len(res), ShouldEqual, 1)
So(res["user"], ShouldNotBeEmpty)
So(len(res["user"]), ShouldEqual, 1)
So(len(res["user"]["list"][0]), ShouldEqual, 1)
So(len(res["user"]["list"][0][0]), ShouldEqual, 1)
userMap, ok := res["user"]["list"][0][0][0].(map[string]any)
So(ok, ShouldBeTrue)
So(len(userMap), ShouldEqual, 2)
So(userMap["age"], ShouldEqual, float64(123))
So(userMap["name"], ShouldEqual, "example")
So(userMap["double"], ShouldBeNil)
})
}
func TestFieldManage(t *testing.T) {
Convey("结构体字段管理", t, func() {
instance := NewStruct(map[string]string{
"test_tag_map": `json:"tag_map"`,
}).AddField("age", "", 0, `json:"age"`, false).
AddField("name", "", "", `json:"name"`, false)
So(instance.HasField("Name"), ShouldBeTrue)
So(instance.GetField("Name"), ShouldNotBeNil)
instance.RemoveField("Name")
So(instance.HasField("Name"), ShouldBeFalse)
So(instance.GetField("Name"), ShouldBeNil)
So(instance.HasField("Age"), ShouldBeTrue)
})
}
func TestExtendStruct(t *testing.T) {
Convey("结构体字段继承", t, func() {
type User struct {
Name string `json:"name"`
Age int `json:"age"`
}
type Company struct {
Address string `json:"address"`
Logo string `json:"logo"`
Age float64 `json:"company_age"` // 应该覆盖掉 User.Age
}
instance := ExtendStruct(User{}, Company{})
So(instance.HasField("Name"), ShouldBeTrue)
So(instance.HasField("Age"), ShouldBeTrue)
So(instance.HasField("Address"), ShouldBeTrue)
So(instance.HasField("Logo"), ShouldBeTrue)
So(reflect.TypeOf(instance.GetField("Age").GetType()).Kind(), ShouldEqual, reflect.Float64)
So(instance.GetField("Age").GetTag(), ShouldEqual, `json:"company_age"`)
})
}
func TestArrayRoot(t *testing.T) {
Convey("嵌套结构体带多级数组", t, func() {
data := []byte(`
[[[{
"age": 123,
"name": "example",
"double": 123.45,
"bool": true,
"slice": [1, 2, 3]
}]]]
`)
instance := NewStruct(map[string]string{
"test_tag_map": `json:"tag_map"`,
}).
AddField("[].[].[].age", "", 0, `json:"age"`, false).
AddField("[].[].[].name", "", "", `json:"name"`, false).
Build()
val := instance.New()
err := json.Unmarshal(data, &val)
So(err, ShouldBeNil)
reflectType := reflect.TypeOf(val)
// 需要是数组类型
So(reflectType.Kind(), ShouldEqual, reflect.Ptr)
So(reflectType.Elem().Kind(), ShouldEqual, reflect.Slice)
// 验证数据
// 数据序列化成功
targetByte, targetErr := json.Marshal(val)
So(targetErr, ShouldBeNil)
// 数据转map成功
var res [][][]any
targetUnmarshalErr := json.Unmarshal(targetByte, &res)
So(targetUnmarshalErr, ShouldBeNil)
So(len(res), ShouldEqual, 1)
userMap, ok := res[0][0][0].(map[string]any)
So(ok, ShouldBeTrue)
So(len(userMap), ShouldEqual, 2)
So(userMap["age"], ShouldEqual, float64(123))
So(userMap["name"], ShouldEqual, "example")
So(userMap["double"], ShouldBeNil)
})
}

View File

@ -3,4 +3,6 @@ package dynamicstruct
const (
// ArraySplit is the string used to split array elements
ArraySplit = "[]"
// ArrayRootFlag 传入路径直接以数组开头
ArrayRootFlag = "__RootArrayFlag"
)

32
dynamic_struct.go Normal file
View File

@ -0,0 +1,32 @@
// Package dynamicstruct ...
//
// Description : dynamicstruct ...
//
// Author : go_developer@163.com<白茶清欢>
//
// Date : 2025-03-23 14:14
package dynamicstruct
import (
"reflect"
)
type dynamicStructImpl struct {
structFields []reflect.StructField
definition reflect.Type
}
// New 创建动态结构体实例
func (ds *dynamicStructImpl) New() any {
return reflect.New(ds.definition).Interface()
}
// NewSliceOfStructs 创建动态结构体切片实例
func (ds *dynamicStructImpl) NewSliceOfStructs() any {
return reflect.New(reflect.SliceOf(ds.definition)).Interface()
}
// NewMapOfStructs 创建动态结构体map实例
func (ds *dynamicStructImpl) NewMapOfStructs(key any) any {
return reflect.New(reflect.MapOf(reflect.Indirect(reflect.ValueOf(key)).Type(), ds.definition)).Interface()
}

36
field_config.go Normal file
View File

@ -0,0 +1,36 @@
// Package dynamicstruct ...
//
// Description : dynamicstruct ...
//
// Author : go_developer@163.com<白茶清欢>
//
// Date : 2025-03-23 14:07
package dynamicstruct
type fieldConfigImpl struct {
name string
pkg string
typ any
tag string
anonymous bool
}
// SetType 设置字段类型
func (f *fieldConfigImpl) SetType(typ any) IFieldConfig {
f.typ = typ
return f
}
func (f *fieldConfigImpl) GetType() any {
return f.typ
}
// SetTag 设置字段标签
func (f *fieldConfigImpl) SetTag(tag string) IFieldConfig {
f.tag = tag
return f
}
func (f *fieldConfigImpl) GetTag() string {
return f.tag
}

13
go.mod
View File

@ -3,14 +3,21 @@ module git.zhangdeman.cn/zhangdeman/dynamic-struct
go 1.24.1
require (
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250227040546-863c03f34bb8 // indirect
git.zhangdeman.cn/zhangdeman/wrapper v0.0.0-20250321102712-1cbfbe959740
github.com/smartystreets/goconvey v1.8.1
)
require (
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250425024726-cc17224cb995 // indirect
git.zhangdeman.cn/zhangdeman/op_type v0.0.0-20240122104027-4928421213c0 // indirect
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20241223084948-de2e49144fcd // indirect
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20250428041157-135850ee8a58 // indirect
git.zhangdeman.cn/zhangdeman/util v0.0.0-20240618042405-6ee2c904644e // indirect
git.zhangdeman.cn/zhangdeman/wrapper v0.0.0-20250302133417-c1588abcb436 // indirect
github.com/BurntSushi/toml v1.5.0 // indirect
github.com/axgle/mahonia v0.0.0-20180208002826-3358181d7394 // indirect
github.com/go-ini/ini v1.67.0 // indirect
github.com/gopherjs/gopherjs v1.17.2 // indirect
github.com/jtolds/gls v4.20.0+incompatible // indirect
github.com/smarty/assertions v1.15.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect

27
go.sum
View File

@ -1,21 +1,47 @@
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250227040546-863c03f34bb8 h1:VEifPc+vkpEQoX9rj7zxmT1m+IA81XjOxe7+Z1aqWNM=
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250227040546-863c03f34bb8/go.mod h1:IXXaZkb7vGzGnGM5RRWrASAuwrVSNxuoe0DmeXx5g6k=
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250328040304-7e4a6f9f148c h1:cl3gQGXQpJ8ugDs0C/hQLfcvF4lGBm5BeABLvROFDoM=
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250328040304-7e4a6f9f148c/go.mod h1:IXXaZkb7vGzGnGM5RRWrASAuwrVSNxuoe0DmeXx5g6k=
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250425024726-cc17224cb995 h1:LmPRAf0AsxRVFPibdpZR89ajlsz8hof2IvMMyTqiEq4=
git.zhangdeman.cn/zhangdeman/consts v0.0.0-20250425024726-cc17224cb995/go.mod h1:5p8CEKGBxi7qPtTXDI3HDmqKAfIm5i/aBWdrbkbdNjc=
git.zhangdeman.cn/zhangdeman/op_type v0.0.0-20240122104027-4928421213c0 h1:gUDlQMuJ4xNfP2Abl1Msmpa3fASLWYkNlqDFF/6GN0Y=
git.zhangdeman.cn/zhangdeman/op_type v0.0.0-20240122104027-4928421213c0/go.mod h1:VHb9qmhaPDAQDcS6vUiDCamYjZ4R5lD1XtVsh55KsMI=
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20241223084948-de2e49144fcd h1:q7GG14qgXKB4MEXQFOe7/UYebsqMfPaSX80TcPdOosI=
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20241223084948-de2e49144fcd/go.mod h1:+D6uPSljwHywjVY5WSBY4TRVMj26TN5f5cFGEYMldjs=
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20250426132259-73cf1be49c7f h1:7QgAcGnmVEVyIPeWH0ZkQN/jpzklYXsKCenTR2GpxbE=
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20250426132259-73cf1be49c7f/go.mod h1:Ig3GZC2hJDkQp7F8Tm53GvMWLh9bdbbauow/vxGO4YA=
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20250428041157-135850ee8a58 h1:fTkmucGaUoKocoX+ASM4AnwsAVJOtOOLUFSqA+uwVzg=
git.zhangdeman.cn/zhangdeman/serialize v0.0.0-20250428041157-135850ee8a58/go.mod h1:Ig3GZC2hJDkQp7F8Tm53GvMWLh9bdbbauow/vxGO4YA=
git.zhangdeman.cn/zhangdeman/util v0.0.0-20240618042405-6ee2c904644e h1:Q973S6CcWr1ICZhFI1STFOJ+KUImCl2BaIXm6YppBqI=
git.zhangdeman.cn/zhangdeman/util v0.0.0-20240618042405-6ee2c904644e/go.mod h1:VpPjBlwz8U+OxZuxzHQBv1aEEZ3pStH6bZvT21ADEbI=
git.zhangdeman.cn/zhangdeman/wrapper v0.0.0-20250302133417-c1588abcb436 h1:SM4zc54W2wmM72+4pMNQ8iS371H6lj4J8rj8KJKf7pw=
git.zhangdeman.cn/zhangdeman/wrapper v0.0.0-20250302133417-c1588abcb436/go.mod h1:YJ1FlvFgkfAHkxkt3l5rKKUqEpQkNMbCFDzDmgteEU8=
git.zhangdeman.cn/zhangdeman/wrapper v0.0.0-20250321102712-1cbfbe959740 h1:zPUoylfJTbc0EcxW+NEzOTBmoeFZ2I/rLFBnEzxb4Wk=
git.zhangdeman.cn/zhangdeman/wrapper v0.0.0-20250321102712-1cbfbe959740/go.mod h1:1ct92dbVc49pmXusA/iGfcQUJzcYmJ+cjAhgc3sDv1I=
github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/axgle/mahonia v0.0.0-20180208002826-3358181d7394 h1:OYA+5W64v3OgClL+IrOD63t4i/RW7RqrAVl9LTZ9UqQ=
github.com/axgle/mahonia v0.0.0-20180208002826-3358181d7394/go.mod h1:Q8n74mJTIgjX4RBBcHnJ05h//6/k6foqmgE45jTQtxg=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g=
github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k=
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY=
github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec=
github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=
github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=
github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI=
github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
@ -23,6 +49,7 @@ github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JT
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

153
wrapper/json.go Normal file
View File

@ -0,0 +1,153 @@
// Package wrapper ...
//
// Description : wrapper ...
//
// Author : go_developer@163.com<白茶清欢>
//
// Date : 2025-04-28 15:32
package wrapper
import (
"encoding/xml"
"errors"
"fmt"
dynamicstruct "git.zhangdeman.cn/zhangdeman/dynamic-struct"
"git.zhangdeman.cn/zhangdeman/serialize"
"github.com/tidwall/gjson"
)
func NewJson(sourceData string, o *Option) (*ownJson, error) {
gjsonRes := gjson.Parse(sourceData)
if gjsonRes.Value() == nil {
return nil, errors.New("source data parse result is nil")
}
// 仅支持map
if !gjsonRes.IsObject() {
return nil, errors.New("source result is not map or struct Marshal string")
}
if nil == o {
o = &Option{}
}
if o.XmlName == "" {
o.XmlName = "XmlData"
}
oj := &ownJson{
sourceData: sourceData,
gjsonResult: gjsonRes,
structBuilder: dynamicstruct.NewStruct(nil),
o: o,
}
if err := oj.GenerateStruct(); nil != err {
return nil, err
}
return oj, nil
}
type ownJson struct {
sourceData string // 数据源
gjsonResult gjson.Result // 数据源解析为gjson
structBuilder dynamicstruct.IBuilder // 结构体构造器
structRes any // 解析的结构体值
o *Option // 一些操作选项
}
// GenerateStruct 生成结构体字段列表
func (oj *ownJson) GenerateStruct() error {
oj.gjsonResult.ForEach(func(fieldName, fieldValue gjson.Result) bool {
oj.generateStructField("", fieldName.String(), fieldValue)
return true
})
// 追加xml的标签
oj.structBuilder.AddField("XMLName", "", xml.Name{}, fmt.Sprintf(`json:"-" toml:"-" yml:"-" init:"-" xml:"%v"`, oj.o.XmlName), false)
val := oj.structBuilder.Build().New()
if err := serialize.JSON.UnmarshalWithNumber([]byte(oj.sourceData), &val); nil != err {
return err
}
oj.structRes = val
return nil
}
// Marshal 结果序列化
func (oj *ownJson) Marshal(marshalType string) ([]byte, error) {
return serialize.Wrapper.Marshal(marshalType, oj.structRes)
}
// generateStructField 递归解析
func (oj *ownJson) generateStructField(rootPath string, currentName string, currentResult gjson.Result) {
structPath := oj.getPath(rootPath, currentName)
if currentResult.IsBool() {
// bool类型
oj.structBuilder.AddField(structPath, "", true, "", false)
return
}
if currentResult.Type == gjson.Number {
// 数字类型, 统一用float64
oj.structBuilder.AddField(structPath, "", float64(0), "", false)
return
}
if currentResult.Value() == nil {
// 空值, any类型
oj.structBuilder.AddField(structPath, "", (*any)(nil), "", false)
return
}
if currentResult.Type == gjson.String {
// 字符串类型
oj.structBuilder.AddField(structPath, "", "", "", false)
return
}
if currentResult.IsObject() {
// 还是一个嵌套对象, 递归处理
currentResult.ForEach(func(key, value gjson.Result) bool {
oj.generateStructField(structPath, key.String(), value)
return true
})
return
}
if currentResult.IsArray() {
// 数组, 递归处理
arrList := currentResult.Array()
if len(arrList) == 0 {
// 空数组,无法判断类型,使用any
oj.structBuilder.AddField(structPath, "", []any{}, "", false)
return
}
if arrList[0].Type == gjson.True || arrList[0].Type == gjson.False {
oj.structBuilder.AddField(structPath, "", []bool{}, "", false)
return
}
if arrList[0].Type == gjson.Number {
oj.structBuilder.AddField(structPath, "", []float64{}, "", false)
return
}
if arrList[0].Type == gjson.String {
oj.structBuilder.AddField(structPath, "", []string{}, "", false)
return
}
if arrList[0].Type == gjson.Null {
oj.structBuilder.AddField(structPath, "", []any{}, "", false)
return
}
if arrList[0].IsArray() {
// 数组递归处理
oj.generateStructField(structPath+".[]", "", arrList[0])
return
}
// 对象结构,递归处理
arrList[0].ForEach(func(key, value gjson.Result) bool {
oj.generateStructField(structPath+".[]", key.String(), value)
return true
})
return
}
}
// getPath 获取路径
func (oj *ownJson) getPath(root string, currentName string) string {
if root == "" {
return currentName
}
if currentName == "" {
return root
}
return root + "." + currentName
}

22
wrapper/json_test.go Normal file
View File

@ -0,0 +1,22 @@
// Package wrapper ...
//
// Description : wrapper ...
//
// Author : go_developer@163.com<白茶清欢>
//
// Date : 2025-04-28 16:59
package wrapper
import (
"fmt"
"testing"
)
func TestNewJson(t *testing.T) {
sourceData := `{"name": "test", "age":18,"company":{"address": "Beijing", "name":"lala"},"index":[1,2,3,4], "deep":[[{"name":"a","age":20},{"name":"c"}], [{"name":"b"},{"name":"d"}]]}`
instance, iErr := NewJson(sourceData, &Option{XmlName: "ResponseData"})
fmt.Println(iErr)
res, err := instance.Marshal("xml")
fmt.Println(err)
fmt.Println(string(res))
}

13
wrapper/option.go Normal file
View File

@ -0,0 +1,13 @@
// Package wrapper ...
//
// Description : wrapper ...
//
// Author : go_developer@163.com<白茶清欢>
//
// Date : 2025-04-28 17:35
package wrapper
// Option 生成结构体之后的可用选项
type Option struct {
XmlName string `json:"xml_name"` // 如果目标格式是 xml, 必须指定xmlName作为根节点
}