Save document with the member number instead the name with protobuf-net and MongoDB

Question

I saw somewhere that with the Go MongoDB driver it is possible to save a document with the order number instead of the field name.
They end up with this in the database:

{
   "3": "foo",
   "10": 1,
   "33": 123456
   "107": {
    "2": "bar",
    "1": "foo"
   }
}

I like the idea! So, I tried to find a way to do the same with the MongoDB C# driver.
I have the code below but I am not sure what I should bring from the protobut-net to get the member order number.

var pack = new ConventionPack();
pack.AddMemberMapConvention("numbered", m => m.SetElementName( WHAT TO PUT HERE ));
ConventionRegistry.Register("numbered", pack, type => true);       

The SetElementName takes a string parameter.
How can I grab the order number of a member from protobuf-net?
Something like ...Member.Order.ToString()
I don't know if this whole thing is a great idea but I want to test it.

Thanks

-- UPDATE --

Just to add more information. I am using inheritance for my models to use generics.

[BsonDiscriminator("Base", RootClass = true)]
[DataContract]
public abstract class Base
{
    [BsonId]
    [BsonRepresentation(BsonType.ObjectId)]
    [ProtoMember(1)]
    public string Id { get; set; }
    
    [BsonDateTimeOptions]
    [ProtoMember(2)]
    public DateTime CreatedDate { get; private set; } = DateTime.UtcNow;

    [BsonDateTimeOptions]
    [ProtoMember(3)]
    public DateTime UpdatedDate { get; set; } = DateTime.UtcNow;
}       
        
[ProtoContract]
public class Todo : Base
{
    [ProtoMember(10)]
    public string Title { get; set; }
    [ProtoMember(20)]
    public string Content { get; set; }
    [ProtoMember(30)]
    public string Category { get; set; }
}      
 

And I added this line as shown in the protobuf-net documentation:

RuntimeTypeModel.Default[typeof(Base)].AddSubType(42, typeof(Todo));

So with that and what Marc showed to get the member's number, I end up having a custom Convention Class in MongoDB with <T> so I can use it for other objects:

public class NumberedElementNameConvention<T> : ConventionBase, IMemberMapConvention where T : Base
{
    public void Apply(BsonMemberMap memberMap) 
    {
        var members = RuntimeTypeModel.Default[typeof(T)].GetFields();
        foreach (var member in members)
        {
            memberMap.SetElementName(member.FieldNumber.ToString());
        }
    }
}          

And the registration of this Convention is done like so:

var pack = new ConventionPack { new NumberedElementNameConvention<Todo>() };
ConventionRegistry.Register("NumberedName", pack, type => true);

After running this I get this error:

Grpc.AspNetCore.Server.ServerCallHandler[6] Error when executing service method 'CreateOne'. MongoDB.Bson.BsonSerializationException: The property 'UpdatedDate' of type 'Nnet.Models.Base' cannot use element name '30' because it is already being used by property 'CreatedDate'...

Also, when I run the code below I am expecting to get all members of the Todo object.

var members = RuntimeTypeModel.Default[typeof(Todo)].GetFields();
foreach (var member in members)
{
   Console.WriteLine($"{member.FieldNumber}: {member.Member.Name}");
}       

However, I am not getting those inherited from the Base object:

❯ dotnet run
10: Title
20: Content
30: Category

Answer 1

Re the edit; in this region:

var members = RuntimeTypeModel.Default[typeof(T)].GetFields();
foreach (var member in members)
{
    memberMap.SetElementName(member.FieldNumber.ToString());
}

I believe you're meant to identify the relevant field from memberMap - i.e. in this context you're only talking about one field at the time; I suspect what is happening is that for each member in turn you're changing the element name multiple times, leaving it at the last protobuf field defined.

Separately, there's a complication of inheritance; protobuf-net doesn't implement inheritance in a flat way - instead, the base type is also expected to be a [ProtoContract] and is meant to define a [ProtoInclude(...)] for each derived type; the field numbers are type-specific, meaning: both the base type and the derived type can legally have a field 1. If you need to describe inheritance, and you are determined to use protobuf-net's model, then you would need to handle this; for example, you could use the [ProtoInclude(...)] number as a prefix on each, so Base.Id is "1", and if we imagine that Todo has field 5 in the [ProtoInclude(...)], then Todo.Title could be "5.10".

Alternatively: if you're not actively using protobuf-net: maybe just use your own attribute for the numbers? or there's usually an inbuilt attribute that the serializer you've chosen would use directly.

Answer 2

The field metadata for protobuf-net is available from the RuntimeTypeModel API, for example:

var members = RuntimeTypeModel.Default[yourType].GetFields();
foreach (var member in members)
{
    Console.WriteLine($"{member.FieldNumber}: {member.Member.Name}");
}

The .FieldNumber gives the protobuf field-number, and .Member gives the MemberInfo of the corresponding field or property. You may want to do some level of caching if the m => m.SetElementName( WHAT TO PUT HERE ) is evaluated lots of times for the same m, so you don't perform unnecessary work - but: before you do, just add some logging to the lambda first, and see how often it gets called: if it isn't too often, maybe don't worry about it.

Note that there is also a lookup on MetaType that allows query by MemberInfo:

var member = RuntimeTypeModel.Default[yourType][memberInfo];

Answer 3

Okay now! So after a some investigation I end up with this simple way to do it with Marc's help. In MongoDB instead of using attributes to decorate models and its properties, it is possible to use code within BsonClassMap. Within that class I add the foreach loop that Marc provided and the right parameters, we can now have numbers instead names.

On the Client side and Server side it is this same code:

//Base Model ClassMap
BsonClassMap.RegisterClassMap<Base>(cm => 
{
    cm.AutoMap();
    foreach (var member in RuntimeTypeModel.Default[typeof(Base)].GetFields())
    {
        cm.MapMember(typeof(Base).GetMember(member.Member.Name)[0])
            .SetElementName(member.FieldNumber.ToString())
            .SetOrder(member.FieldNumber);
    }
});

//Todo Model ClassMap
BsonClassMap.RegisterClassMap<Todo>(cm => 
{
    cm.AutoMap();
    foreach (var member in RuntimeTypeModel.Default[typeof(Todo)].GetFields())
    {
        cm.MapMember(typeof(Todo).GetMember(member.Member.Name)[0])
             .SetElementName(member.FieldNumber.ToString())
             .SetOrder(member.FieldNumber);
    }
});        

it's a little ugly but you can rework it.

One thing to note is that MongoDB has the control over the Id. In the database anything that represent the object id become _id. Same thing when you insert a new document in the database a _t field is added if you use Discriminator (I am not sure if it's full related). Basically, every member beginning with a underscore is reserved. See the image below after running de code:

You can refer to the question above in the update section to see if this result represent the models with the given orders (it does).

Here is the code I use for insertion and queries:

// INSERT
var client = channel.CreateGrpcService<IBaseService<Todo>>();
var reply = await client.CreateOneAsync(
   new Todo
   {
      Title = "Some Title"
   }
);        
  
// FIND BY ID
var todoId = new UniqueIdentification { Id = "613c110a073055f0d87a0e27"};
var res = await client.GetById(todoId);
     
    
// FIND ONE BY QUERY FILTER REQUEST 
    ...
var filter = Builders<Todo>.Filter.Eq("10", "Some Title");
var filterString = filter.Render(documentSerializer, serializerRegistry);
    ...         

The last one above it's a query with the number ("10") of the property Title. But it's possible in the same way to query with the property name, like so:

// FIND ONE BY QUERY FILTER REQUEST 
     ...
var filter = Builders<Todo>.Filter.Eq(e => e.Title, "Some Title");
var filterString = filter.Render(documentSerializer, serializerRegistry);
    ...      
     

What is great with this approach is that these BsonClassMap are called once on the Client or/and Server when they are initiated.

I just realize that this might not be a good idea because it is going to be painful to prevent collision between numbers. The order numbers in the code below is possible:

[BsonDiscriminator("Base", RootClass = true)]
[DataContract]
public abstract class Base
{
   [BsonId]
   [BsonRepresentation(BsonType.ObjectId)]
   [ProtoMember(1)]
   public string Id { get; set; }

   [BsonDateTimeOptions]
   [ProtoMember(2)]
   public DateTime CreatedDate { get; private set; } = DateTime.UtcNow;

   [BsonDateTimeOptions]
   [ProtoMember(3)]
   public DateTime UpdatedDate { get; set; } = DateTime.UtcNow;
}       
    
[ProtoContract]
public class Todo : Base
{
   [ProtoMember(1)]
   public string Title { get; set; }
   [ProtoMember(2)]
   public string Content { get; set; }
   [ProtoMember(3)]
   public string Category { get; set; }
}          

but there is going to be three collisions if the foreach loop runs.
Yeah... :/
This is where Marc's second solution comes in, where you put a prefix... I am going to keep the name convention by default.

Cheers!