C# Json object serialization with dynamic custom object

Question

I have a C# class like the following:

public class FunctionCall
{
    public string x {get; set;}
    public int y {get; set;}; 

    public object obj1 {get; set;};
    public object obj2 {get; set;};
    ... 
    ...
    public object objN {get; set;};
}

with N >= 0. That means that there could be 0 or N > 0 objects objX So the problem is that I do not know at build time how many objects there will be and I need to serialize a complex structure that contains instances of the class.

The class FunctionCall is used to call OpenAI Assistant API passing the parameters of a function. ChatGPT expects the parameters to be embedded as JSon objects like the following. The Json is just a fragment that represents an objX instance embedded in a more complex document which has no dynamic fields so it's known at compile time and it's not a problem.

{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Determine weather in my location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The city and state e.g. San Francisco, CA"
            },
            "unit": {
              "type": "string",
              "enum": [
                "c",
                "f"
              ]
            }
          },
          "required": [
            "location"
          ]
        }
    }
}

"paramters" contains 2 objects: "location" and "unit", that are the 2 parameters of the function. Actually the JSon is a schema of the function's parameters. It should be serialized/deserialized using an instance of FunctionCall but the parameter objects are variable (depends on the functions that are specified during configuration at runtime, not at compile time) and cannot be passed as a collection, but as independent objects, like "location" and "unit".

For serialization/deserialization I use:

System.Text.Json.JsonSerializer.Serialize 
System.Text.Json.JsonSerializer.Deserialize

And I tried to define a custom serializer but it doesn't seem to solve my problem.

I also tried to use Newtonsoft dynamic serialization (using a dynamic object), but it returns a JObject instead of an anonymous class instance with the fields I need, so it doesn't work for me.

Any helpful hint would be much appreciated.

Answer 1

I'm afraid you want to have a similar feature like what we can do in JS to read json content directly without having a model binding to the json data. If so, then I had a test with code below which might meet your requirement.

    public void readJsonString() { 
        string json1 = "{\"x\":\"valuex\", \"y\":2, \"str1\":\"value1\"}";
        string json2 = "{\"x\":\"valuex\", \"y\":2, \"str1\":\"value1\", \"str2\":\"value2\", \"str3\":\"value3\"}";
        JObject jsonObj1 = JObject.Parse(json1);
        JObject jsonObj2 = JObject.Parse(json2);
        var a = jsonObj1.ContainsKey("str2") ? jsonObj1["str2"] : "null";
        var b = jsonObj2.ContainsKey("str2") ? jsonObj2["str2"] : "null";
    }

In addition, I don't think it's a good idea to handle data like this. It's not good for maintance at least... It's better for us to parse data format at first then create a model for it.

Answer 2

I'm not sure I understand the problem well but you could use the method GetFields() https://learn.microsoft.com/en-us/dotnet/api/system.type.getfields?view=net-8.0

you should be able to easily get the desired result with something like this:

Type myType = typeof(Test);
FieldInfo[] myField = myType.GetFields();

Answer 3

In the end I succeeded by creating a custom converter (System.Text.Json.Serialization.JsonConverter) like in the following example. So it can be done as the example below demonstrate it (it has been successfully tested with a .NET 8 project):

....
....
using System.Text.Json;
using System.Text.Json.Serialization;



    internal class FunctionCallConverter : JsonConverter<FunctionCall>
    {
        public override FunctionCall Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) 
        {         
            if (reader.TokenType != JsonTokenType.StartObject)
            {
                throw new JsonException();
            }

            reader.Read();
            if (reader.TokenType != JsonTokenType.PropertyName)
            {
                throw new FunctionCallDeserializationException("type");
            }

            reader.Read();//Type
            string tool = reader.GetString()!;
            if (tool != "function")
            {
                throw new InvalidToolDeserializationException(tool);
            }

            reader.Read();//Read function object
            reader.Read();//Open object token

            if (reader.TokenType != JsonTokenType.StartObject)
            {
                throw new FunctionCallUnexpectedTokenException();
            }

            reader.Read();//function
            if (reader.TokenType != JsonTokenType.PropertyName)
            {
                throw new FunctionCallDeserializationException("name");
            }
            reader.Read();//function name
            string? functionName = reader.GetString();

            FunctionCall fcall = new(functionName!);

            reader.Read();//Read description
            reader.Read();//Read description value
            string? description = reader.GetString();            
            fcall.Description = description;

            reader.Read();
            string? parameters = reader.GetString();
            if (parameters != "parameters")
            {
                throw new FunctionCallDeserializationException("parameters");
            }

            reader.Read();
            if (reader.TokenType != JsonTokenType.StartObject)
            {
                throw new FunctionCallDeserializationException("parameters");
            }

            while (reader.TokenType != JsonTokenType.EndObject)
            {
                reader.Read();//type
                reader.Read();//type value
                string objName = reader.GetString()!;
                fcall.Params.ParameterObjectType = objName;
                                
                reader.Read();//Start object
                reader.Read();//Param name

                reader.Read();

                while (reader.TokenType != JsonTokenType.EndObject)
                {
                    string pname = reader.GetString()!;
                    FunctionCallTool.Parameters.Parameter p = new(pname);
                    if (fcall.Params.ParamsList == null) { fcall.Params.ParamsList = []; }
                    fcall.Params.ParamsList.Add(p);

                    reader.Read();//Open object
                    reader.Read();//Type
                    reader.Read();//Type value

                    p.ParameterType = reader.GetString();

                    reader.Read();//Description or enum (if present)
                    if (reader.TokenType == JsonTokenType.PropertyName && reader.GetString() == "description") {
                        
                        reader.Read();//Description value
                        string? paramDescription = reader.GetString();
                        if (paramDescription != null)
                        {
                            p.Description = paramDescription;
                        }
                        reader.Read();//Enum (if present)
                    }                    

                    if (reader.TokenType == JsonTokenType.PropertyName)
                    {
                        string? enumName = reader.GetString();
                        if (enumName != null && enumName == "enum")
                        {
                            reader.Read();//Enum start token
                            reader.Read();//Enum value
                            
                            if (p.Enum == null) { p.Enum = new(); }

                            do
                            {
                                
                                string enumValue = reader.GetString()!;
                                p.Enum.Add(enumValue);
                                reader.Read();//Next enum value or end enum token

                            } while (reader.TokenType != JsonTokenType.EndArray);
                        }
                    }
                    reader.Read();//End single parameter object
                }
                reader.Read();//End parameters object
            }

            reader.Read();//Skip end object token

            if (reader.TokenType == JsonTokenType.PropertyName)
            {
                string? requiredParams = reader.GetString();
                if (requiredParams != null && requiredParams == "required")
                {
                    if (fcall.Params.RequiredParams == null) { fcall.Params.RequiredParams = []; }
                    reader.Read();//Skip current token
                    reader.Read();//Skip open array token

                    while (reader.TokenType != JsonTokenType.EndArray)
                    {
                        string? pname = reader.GetString();//Param name
                        if (pname != null)
                        {
                            var param = fcall.Params.GetParameter(pname);
                            fcall.Params.RequiredParams.Add(param);
                            reader.Read();//Next parameter or end array token
                        }                        
                    }
                    reader.Read();//Skip end of array token
                }
            }

            reader.Read();//Skip end of parameters object
            reader.Read();//Skip end of function object            

            return fcall;
        }


        public override void Write(Utf8JsonWriter writer, FunctionCall fcall, JsonSerializerOptions options)
        {
            writer.WriteStartObject();//Function object start
            
            writer.WriteString("type", "function");
            
            writer.WriteStartObject("function");
            
            writer.WriteString("name", fcall.Name);
            writer.WriteString("description", fcall.Description);

            writer.WriteStartObject("parameters");

            writer.WriteString("type", fcall.Params.ParameterObjectType);

            if (fcall.Params.ParamsList != null)
            {
                writer.WriteStartObject("properties");
                foreach (var p in fcall.Params.ParamsList)
                {
                    writer.WriteStartObject(p.Name);
                    writer.WriteString("type", p.ParameterType);

                    if (p.Description != null)
                    {
                        writer.WriteString("description", p.Description);
                    }
                    if (p.Enum != null)
                    {
                        writer.WriteStartArray("enum");

                        foreach (var e in p.Enum)
                        {
                            writer.WriteStringValue(e);
                        }

                        writer.WriteEndArray();
                    }

                    writer.WriteEndObject();//Parameter
                }
                writer.WriteEndObject();//properties
            }

            if (fcall.Params.RequiredParams != null)
            {
                writer.WriteStartArray("required");

                foreach (var r in fcall.Params.RequiredParams) 
                { 
                    writer.WriteStringValue(r.Name);
                }

                writer.WriteEndArray();
            }
            
            writer.WriteEndObject();//parameters

            writer.WriteEndObject();//Function object end

            writer.WriteEndObject();//Enclosing object end
        }
        
    }



....
....
string j = "{\r\n      \"type\": \"function\",\r\n      \"function\": {\r\n        \"name\": \"get_weather\",\r\n        \"description\": \"Determine weather in my location\",\r\n        \"parameters\": {\r\n          \"type\": \"object\",\r\n          \"properties\": {\r\n            \"location\": {\r\n              \"type\": \"string\",\r\n              \"description\": \"The city and state e.g. San Francisco, CA\"\r\n            },\r\n            \"unit\": {\r\n              \"type\": \"string\",\r\n              \"enum\": [\r\n                \"c\",\r\n                \"f\"\r\n              ]\r\n            }\r\n          },\r\n          \"required\": [\r\n            \"location\"\r\n          ]\r\n        }\r\n      }\r\n    }";


FunctionCall i = System.Text.Json.JsonSerializer.Deserialize<FunctionCall>(j)!;
string o = System.Text.Json.JsonSerializer.Serialize<FunctionCall>(i)!;

The code above does deserialize the following JSon into the FunctionCall class of the question that has 2 properties child objects (location and unit), but it can do it with any number of properties objects:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Determine weather in my location",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "The city and state e.g. San Francisco, CA"
        },
        "unit": {
          "type": "string",
          "enum": [ "c", "f" ]
        }
      },
      "required": [ "location" ]
    }
  }
}

The full FunctionCall class declaration is:

[JsonConverter(typeof(FunctionCallConverter))]
public class FunctionCall(string n)
{
    public string? Name { get; set; } = n;

    public string? Description { get; set; }

    public class Parameters
    {

        public string ParameterObjectType { get; set; } = "object";
        
        public class Parameter(string n)
        {
            public string Name { get; set; } = n;

            public string? ParameterType { get; set; }

            public string? Description { get; set; }

            public List<string>? Enum { get; set; } = null;
        }

        public List<Parameter>? ParamsList { get; set; }
 
        public List<Parameter>? RequiredParams { get; set; }

        public Parameter GetParameter(string name)
        {
            return ParamsList!.Where(x => x.Name == name).First();
        }
    }
 
    public Parameters Params { get; set; } = new Parameters();
}