How to use (pack) Google.Protobuf.WellknownTypes.Any in protobuf-net code first gRPC

Question

I am creating an gRPC service and we decided to choose the code first approach with protobuf-net. Now I am running into a scenario where we have a couple of classes that need to be wrapped. We do not want to define KnownTypes in the MyMessage class (just a sample name to illustrate the problem). So I am trying to use the Any type which currently gives me some struggle with packing.

The sample code has the MyMessage which defines some header values and has to possiblity to deliver any type as payload.

[ProtoContract]
public class MyMessage 
{
  [ProtoMember(1)] public int HeaderValue1 { get; set; }
  [ProtoMember(2)] public string HeaderValue2 { get; set; }
  [ProtoMember(3)] public Google.Protobuf.WellknownTypes.Any Payload { get; set; }
}

[ProtoContract]
public class Payload1 
{
  [ProtoMember(1)] public bool Data1   { get; set; }
  [ProtoMember(2)] public string Data2 { get; set; }
}

[ProtoContract]
public class Payload2 
{
  [ProtoMember(1)] public string Data1 { get; set; }
  [ProtoMember(2)] public string Data2 { get; set; }  
}

Somewhere in the code I construct my message with a payload ...

  Payload2 payload = new Payload2 {
    Data1 = "abc",
    Data2 = "def"
  };
  
  MyMessage msg = new MyMessage 
  {
    HeaderValue1 = 123,
    HeaderValue2 = "iAmHeaderValue2",
    Payload = Google.Protobuf.WellknownTypes.Any.Pack(payload)
  };

Which doesn't work because Payload1 and Payload2 need to implement Google.Protobuf.IMessage. Since I can't figure out how and do not find a lot information how to do it at all I am wondering if I am going a wrong path.

• How is it intedend to use Any in protobuf-net?
• Is there a simple (yet compatible) way to pack a C# code first class into Google.Protobuf.WellknownTypes.Any?
• Do I really need to implement Google.Protobuf.IMessage?

Answer 1

My Any implementation.

[ProtoContract(Name = "type.googleapis.com/google.protobuf.Any")]
public class Any
{
    /// <summary>Pack <paramref name="value"/></summary>
    public static Any Pack(object? value)
    {
        // Handle null
        if (value == null) return new Any { TypeUrl = null!, Value = Array.Empty<byte>() };
        // Get type
        System.Type type = value.GetType();
        // Write here
        MemoryStream ms = new MemoryStream();
        // Serialize
        RuntimeTypeModel.Default.Serialize(ms, value);
        // Create any
        Any any = new Any
        {
            TypeUrl = $"{type.Assembly.GetName().Name}/{type.FullName}",
            Value = ms.ToArray()
        };
        // Return
        return any;
    }

    /// <summary>Unpack any record</summary>
    public object? Unpack()
    {
        // Handle null
        if (TypeUrl == null || Value == null || Value.Length == 0) return null;
        // Find '/'
        int slashIx = TypeUrl.IndexOf('/');
        // Convert to C# type name
        string typename = slashIx >= 0 ? $"{TypeUrl.Substring(slashIx + 1)}, {TypeUrl.Substring(0, slashIx)}" : TypeUrl;
        // Get type (Note security issue here!)
        System.Type type = System.Type.GetType(typename, true)!;
        // Deserialize
        object value = RuntimeTypeModel.Default.Deserialize(type, Value.AsMemory());
        // Return
        return value;
    }

    /// <summary>Test type</summary>
    public bool Is(System.Type type) => $"{type.Assembly.GetName().Name}/{type.FullName}" == TypeUrl;
      
    /// <summary>Type url (using C# type names)</summary>
    [ProtoMember(1)]
    public string TypeUrl = null!;
    /// <summary>Data serialization</summary>
    [ProtoMember(2)]
    public byte[] Value = null!;

    /// <summary></summary>
    public static implicit operator Container(Any value) => new Container(value.Unpack()! );
    /// <summary></summary>
    public static implicit operator Any(Container value) => Any.Pack(value.Value);

    /// <summary></summary>
    public struct Container
    {
        /// <summary></summary>
        public object? Value;
        /// <summary></summary>
        public Container()
        {
            this.Value = null;
        }

        /// <summary></summary>
        public Container(object? value)
        {
            this.Value = value;
        }
    }
}

'System.Object' can be used as a field or property in a surrounding Container record:

[DataContract]
public class Container
{
    /// <summary></summary>
    [DataMember(Order = 1, Name = nameof(Value))]
    public Any.Container Any { get => new Any.Container(Value); set => Value = value.Value; }
    /// <summary>Object</summary>
    public object? Value;
}

Serialization

    RuntimeTypeModel.Default.Add(typeof(Any.Container), false).SetSurrogate(typeof(Any));
    var ms = new MemoryStream();
    RuntimeTypeModel.Default.Serialize(ms, new Container { Value = "Hello world" });
    Container dummy = RuntimeTypeModel.Default.Deserialize(typeof(Container), ms.ToArray().AsMemory()) as Container;

Answer 2

Firstly, since you say "where we have a couple of classes that need to be wrapped" (emphasis mine), I wonder if what you actually want here is oneof rather than Any. protobuf-net has support for the oneof concept, although it isn't obvious from a code-first perspective. But imagine we had (in a contract-first sense):

syntax = "proto3";
message SomeType {
    oneof Content {
       Foo foo = 1;
       Bar bar = 2;
       Blap blap = 3;
    }
}

message Foo {}
message Bar {}
message Blap {}

This would be implemented (via the protobuf-net schema tools) as:

private global::ProtoBuf.DiscriminatedUnionObject __pbn__Content;

[global::ProtoBuf.ProtoMember(1, Name = @"foo")]
public Foo Foo
{
    get => __pbn__Content.Is(1) ? ((Foo)__pbn__Content.Object) : default;
    set => __pbn__Content = new global::ProtoBuf.DiscriminatedUnionObject(1, value);
}
public bool ShouldSerializeFoo() => __pbn__Content.Is(1);
public void ResetFoo() => global::ProtoBuf.DiscriminatedUnionObject.Reset(ref __pbn__Content, 1);

[global::ProtoBuf.ProtoMember(2, Name = @"bar")]
public Bar Bar
{
    get => __pbn__Content.Is(2) ? ((Bar)__pbn__Content.Object) : default;
    set => __pbn__Content = new global::ProtoBuf.DiscriminatedUnionObject(2, value);
}
public bool ShouldSerializeBar() => __pbn__Content.Is(2);
public void ResetBar() => global::ProtoBuf.DiscriminatedUnionObject.Reset(ref __pbn__Content, 2);

[global::ProtoBuf.ProtoMember(3, Name = @"blap")]
public Blap Blap
{
    get => __pbn__Content.Is(3) ? ((Blap)__pbn__Content.Object) : default;
    set => __pbn__Content = new global::ProtoBuf.DiscriminatedUnionObject(3, value);
}
public bool ShouldSerializeBlap() => __pbn__Content.Is(3);
public void ResetBlap() => global::ProtoBuf.DiscriminatedUnionObject.Reset(ref __pbn__Content, 3);

optionally with an enum to help:

public ContentOneofCase ContentCase => (ContentOneofCase)__pbn__Content.Discriminator;

public enum ContentOneofCase
{
    None = 0,
    Foo = 1,
    Bar = 2,
    Blap = 3,
}

This approach may be easier and preferable to Any.

---

On Any:

Short version: protobuf-net has not, to date, had any particular request to implement Any. It probably isn't a huge amount of work - simply: it hasn't yet happened. It looks like you're referencing both protobuf-net and the Google libs here, and using the Google implementation of Any. That's fine, but protobuf-net isn't going to use it at all - it doesn't know about the Google APIs in this context, so: implementing IMessage won't actually help you.

I'd be more than happy to look at Any with you, from the protobuf-net side. Ultimately, time/availability is always the limiting factor, so I prioritise features that are seeing demand. I think you may actually be the first person asking me about Any in protobuf-net.