An error has occurred while opening external DTD - 400 Error

Question

I'm having an issue where a third party dtd is intermittently throwing a HTTP status 400 error when I am trying to validate xml.

The error thrown is:

An error has occurred while opening external DTD 'http://example.com/fake/path/MyDTD.dtd': Response status code does not indicate success: 400 (Bad Request).
at System.Xml.XmlTextReaderImpl.Throw(Exception e)
at System.Xml.XmlTextReaderImpl.PushExternalEntityOrSubset(String publicId, String systemId, Uri baseUri, String entityName)
at System.Xml.XmlTextReaderImpl.DtdParserProxy_PushExternalSubset(String systemId, String publicId)
at System.Xml.XmlTextReaderImpl.DtdParserProxy.System.Xml.IDtdParserAdapter.PushExternalSubset(String systemId, String publicId)
at System.Xml.DtdParser.ParseExternalSubset()
at System.Xml.DtdParser.ParseInDocumentDtd(Boolean saveInternalSubset)
at System.Xml.DtdParser.Parse(Boolean saveInternalSubset)
at System.Xml.DtdParser.System.Xml.IDtdParser.ParseInternalDtd(IDtdParserAdapter adapter, Boolean saveInternalSubset)
at System.Xml.XmlTextReaderImpl.ParseDtd()
at System.Xml.XmlTextReaderImpl.ParseDoctypeDecl()
at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
at System.Xml.XmlTextReaderImpl.Read()
at System.Xml.XmlValidatingReaderImpl.Read()
at MyApi.Infrastructure.Serializers.MyCustomXmlSerializer.ValidateAgainstDtd(String xml, Boolean throwOnFailure)
in D:\a\1\s\src\Infrastructure\Serializers\MyCustomXmlSerializer.cs:line 198

And here is my method for validating xml:

public bool ValidateAgainstDtd(string xml, bool throwOnFailure = false)
{
    var messages = new List<string>();

    try
    {
        var settings = new XmlReaderSettings
        {
            ValidationType = ValidationType.DTD,
            DtdProcessing = DtdProcessing.Parse,
            XmlResolver = new XmlPreloadedResolver(new XmlUrlResolver())
        };

        settings.ValidationEventHandler += (sender, args) => messages.Add(args.Message);

        using var stream = new MemoryStream(Encoding.UTF8.GetBytes(xml));
        using var reader = XmlReader.Create(stream, settings);

        while (reader.Read()) { }  //this is line 198 in the exception
    }
    catch (Exception e)
    {
        messages.Add($"DTD Validation Exception: {e.Message}{Environment.NewLine}{e.StackTrace}");
    }

    if (messages.Count == 0)
    {
        return true;
    }

    if (throwOnFailure)
    {
        throw new CustomXmlSerializationException(messages.ToArray());
    }

    return false;
}

I'm not getting any other validation errors, and the model we are serializing is common between several different calls, but it only fails for one call and only intermittently - that same call works in some other instances.

The same code works fine in integration tests run as part of an Nunit suite on a local dev machine, but when we deploy the code into our stage environment we suddenly get intermittent issues with pulling back the DTD.