How to properly recreate a file from an MTOM-Message in C#

Question

I have the following situation. I am trying to write a program in C# that is able to read the binary data from an MTOM-Message and write it into a new file. Here is an example of an MTOM-Message for an image the server I am working with sent to me (the actual binary data is shortened):

--uuid:42069568-2fd1-4f66-874c-000cd2e36fae
Content-Type: text/xml; charset=UTF-8
Content-Transfer-Encoding: binary
Content-ID: <[email protected]>

<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"><wsu:Timestamp wsu:Id="6cb7c01e-372a-40f8-9ab3-ae5735d68a37" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><wsu:Created>2023-04-05T12:14:40Z</wsu:Created><wsu:Expires>2023-04-05T12:19:40Z</wsu:Expires></wsu:Timestamp><saml2:Assertion ID="155d9fa0-bd04-404f-bf12-b84d9347c02b" IssueInstant="2023-04-05T12:14:40Z" Version="2.0" xmlns:vwsu="http://xmldefs.volkswagenag.com/Technical/Security/UsernameToken/V1" xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion"><saml2:Issuer>https://ws-gateway.volkswagenag.com</saml2:Issuer><saml2:Subject><saml2:NameID Format="urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified" NameQualifier="http://xmldefs.volkswagenag.com/Technical/Security/NameQualifier/V1#Anonymous">UNKNOWN</saml2:NameID><saml2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:sender-vouches"/></saml2:Subject><saml2:Conditions NotBefore="2023-04-05T12:13:40Z" NotOnOrAfter="2023-04-05T12:19:40Z"/><saml2:Advice><saml2:Assertion ID="8ccded29-1327-419a-b51a-b84d93471792" IssueInstant="2023-04-05T12:14:40Z" Version="2.0"><saml2:Issuer>https://ws-gateway.volkswagenag.com</saml2:Issuer><saml2:Subject><saml2:NameID Format="urn:oasis:names:tc:SAML:2.0:ac:classes:unspecified" NameQualifier="http://xmldefs.volkswagenag.com/Technical/Security/NameQualifier/V1#UMSGlobalUserID">DUONZQM</saml2:NameID><saml2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:sender-vouches"/></saml2:Subject><saml2:Conditions NotBefore="2023-04-05T12:13:40Z" NotOnOrAfter="2023-04-05T12:19:40Z"/><saml2:AuthnStatement AuthnInstant="2023-04-05T12:14:40Z"><saml2:AuthnContext><saml2:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:TLSClient</saml2:AuthnContextClassRef></saml2:AuthnContext></saml2:AuthnStatement><saml2:AttributeStatement><saml2:Attribute Name="ValidationType" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><saml2:AttributeValue>sc:no-credential-validation-type</saml2:AttributeValue></saml2:Attribute></saml2:AttributeStatement></saml2:Assertion></saml2:Advice></saml2:Assertion></wsse:Security><To wsu:Id="98eebfae-b071-4ad8-ba4f-ae5735d69f52" xmlns="http://www.w3.org/2005/08/addressing" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">ws://volkswagenag.com/PP/QM/GroupProblemManagementService/V3</To><Action wsu:Id="8ecd6f90-d506-4597-90e3-ae5735d68a80" xmlns="http://www.w3.org/2005/08/addressing" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">http://xmldefs.volkswagenag.com/PP/QM/GroupProblemManagementService/V3/KpmService/GetDocumentResponse</Action><MessageID wsu:Id="3b62de53-13fd-46be-a2b2-ae5735d6eefa" xmlns="http://www.w3.org/2005/08/addressing" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">${= "urn:uuid:" + UUID.randomUUID()}</MessageID><wsa:RelatesTo xmlns:wsa="http://www.w3.org/2005/08/addressing">${= "urn:uuid:" + UUID.randomUUID()}</wsa:RelatesTo></soap:Header><soap:Body><ns2:GetDocumentResponse xmlns:ns2="http://xmldefs.volkswagenag.com/PP/QM/GroupProblemManagementService/V3"><GetDocumentResponseInternal><ResponseMessage><MessageId>INFO_001</MessageId><MessageType>MT_INFO</MessageType><MessageText>Method completed successfully</MessageText><SessionKey>de.volkswagen.kpm.backend.command.KPMSessionImpl@806d9365</SessionKey><VersionId>release_17.3.2</VersionId><VersionDate>Fri Mar 17 11:33:58 CET 2023</VersionDate></ResponseMessage><Document><Name>Pound_layer_cake</Name><Suffix>jpg</Suffix><Description>Uploaded 20230216_10-13-37</Description><AccessRight>0</AccessRight><FileType>01</FileType><Data><xop:Include href="cid:[email protected]" xmlns:xop="http://www.w3.org/2004/08/xop/include"/></Data></Document></GetDocumentResponseInternal></ns2:GetDocumentResponse></soap:Body></soap:Envelope>
--uuid:42069568-2fd1-4f66-874c-000cd2e36fae
Content-Type: image/jpeg
Content-Transfer-Encoding: binary
Content-ID: <[email protected]>

ÿØÿáExif  II*                b       j   (       1     r   2    ’   i‡    ¨   à   ÀÆ- '  ÀÆ- '  Adobe Photoshop CS6 (Macintosh) 2021:07:06 16:31:56         0221     ÿÿ       <       ƒ                    .      6  (           >      Å      H      H      ÿØÿí Adobe_CM ÿî Adobe d€   ÿÛ „        



ÿÀ  |  " ÿÝ  
ÿÄ?            
........

I am receiving the raw data from the server as a byte array, using WebClient.UploadData. So far, I have tried using several methods to properly read and write the data into a new file, including XmlDictionaryReader.CreateMtomReader, MimeEntity and MtomMessageEncodingBindingElement.CreateMessageEncoderFactory. Here is a example of how I managed to do it with XmlDictionaryReader:

                // Create an MTOM-Reader and save the data inside it
                XmlDictionaryReader reader = XmlDictionaryReader.CreateMtomReader(result, 0, result.Length, Encoding.UTF8, XmlDictionaryReaderQuotas.Max);

                // Reading the Data out of the mtomReader and saving it in a file
                while (reader.Read())
                {
                    if (reader.Name != "Data")
                    {
                        reader.Read();
                        continue;
                    }

                    byte[] val = reader.ReadElementContentAs(typeof(byte[]), null) as byte[];
                    File.WriteAllBytes(DestinationFile, val);
                    break;
                }

While most of those methods were able to read the Mtom-Message and were able to extract the binary part into a file, the created file only worked when it was a text file. More complex binary data (such as the image above) always ended up being corrupted.

I managed to get my hands on the originals files. I tried opening the original, working file in a text editor to compare what was written there with the data from the server in a text comparer. Both files had exactly the same text written into it.

Assuming that there would be more to it, I wrote a program that imports both files as a byte array and compares each byte to determine if there is a difference. The result was there were quite a few differences. It seems to me that there is either some additional data missing, which is needed to create a working file, or that I am doing something wrong and am not writing the data properly into the new file.

I cannot figure out what I am doing wrong or what data is additionally needed.

Answer 1

I tried not using a method for reading MTOM-Messages at all. Instead I took the server response and cut off the header and the soap envelope manually:

                // Find the end of the soap envelope and cut header and soap envelope off the array
                int endEnvelopeIndex = FindBytes(result, Encoding.UTF8.GetBytes("</soap:Envelope>")) + 16;
                result = result.Skip(endEnvelopeIndex).Take(result.Length - endEnvelopeIndex).ToArray();
                // Find the start of the Content-ID header and cut everything except the binary data off the array
                int startIndexBinary = FindBytes(result, Encoding.UTF8.GetBytes("apache.org>")) + 15;
                result = result.Skip(startIndexBinary).Take(result.Length - startIndexBinary).ToArray();
                // Find the UUID at the end of the binary data and cut it off the array
                int uuidIndex = FindBytes(result, Encoding.UTF8.GetBytes("--uuid:"));
                result = result.Take(uuidIndex).ToArray();
                // Saving the binary data into a new file
                File.WriteAllBytes(DestinationFile, result);

All Files turned out well and working. Sometimes the solution is so simple....