Azure StorageException when using emulated storage (within documented constraints)

Question

Our application performs several batches of TableBatchOperation. We ensure that each of these table batch operations has

• 100 or fewer table operations
• table operations for one entity partition key only

Along the lines of the following:

        foreach (var batch in batches)
        {
            var operation = new TableBatchOperation();
            operation.AddRange(batch.Select(x => TableOperation.InsertOrReplace(x)));
            await table.ExecuteBatchAsync(operation);
        }

• When we use emulated storage we 're hitting a Microsoft.WindowsAzure.Storage.StorageException - "Element 99 in the batch returned an unexpected response code."
• When we use production Azure, everything works fine.

Emulated storage is configured as follows:

<add key="StorageConnectionString" value="UseDevelopmentStorage=true;" />

I'm concerned that although everything is working OK in production (where we use real Azure), the fact that it's blowing up with emulated storage may be symptomatic of us doing something we shouldn't be.

I've run it with a debugger (before it blows up) and verified that (as per API):

• The entire operation is only only 492093 characters when serialized to JSON (984186 bytes as UTF-16)
• There are exactly 100 operations
• All entities have the same partition key

See https://learn.microsoft.com/en-us/dotnet/api/microsoft.windowsazure.storage.table.tablebatchoperation?view=azurestorage-8.1.3

EDIT: It looks like one of the items (#71/100) is causing this to fail. Structurally it is no different to the other items, however it does have some rather long string properties - so perhaps there is an undocumented limitation / bug?

EDIT: The following sequence of Unicode UTF-16 bytes (on a string property) is sufficent to cause the exception:

r     e     n     U+0019         space
114 0 101 0 110 0 25 0 115 0 32 0

(it's the bytes 25 0 115 0 i.e. unicode end-of-medium U+0019 which is causing the exception).

EDIT: Complete example of failing entity:

JSON:

{"SomeProperty":"ren\u0019s ","PartitionKey":"SomePartitionKey","RowKey":"SomeRowKey","Timestamp":"0001-01-01T00:00:00+00:00","ETag":null}

Entity class:

public class TestEntity : TableEntity
{
    public string SomeProperty { get; set; }
}

Entity object construction:

var entity = new TestEntity
{
    SomeProperty = Encoding.Unicode.GetString(new byte[]
        {114, 0, 101, 0, 110, 0, 25, 0, 115, 0, 32, 0}),
    PartitionKey = "SomePartitionKey",
    RowKey = "SomeRowKey"
};

Answer 1

According to your description, I also can reproduce the issue that you mentioned. After I tested I found that the special Unicode Character 'END OF MEDIUM' (U+0019) seems that not supported by Azure Storage Emulator. If replace to other unicode is possible, please try to use another unicode to instead of it.

we also could give our feedback to Azure storage team.