Service Fabric application Multi-Instance randomly return 503 error

Question

We have a service fabric application which comprises of a single service. This service exposes HTTP endpoints via Web Api on port 8910

we deploy this application to different instances and use the following code to prevent port clashes between our service instances

    protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
    {
        var settingsResolver = SettingsResolver.GetResolver();

        return new ServiceInstanceListener[]
        {
            new ServiceInstanceListener(serviceContext => 
                new OwinCommunicationListener(
                    startup => new OwinBuilder(settingsResolver, baseLogger)
                                .Configure(startup), 
                    serviceContext, 
                    ServiceEventSource.Current, 
                    "ServiceEndpoint",
                    serviceContext.ServiceName.Segments[1]))
        };
    }

serviceContext.ServiceName.Segments[1] resolves to the name of the application instance

The service manifest has the port configuration

<Endpoint Protocol="http" Name="ServiceEndpoint" Type="Input" Port="8910" />

However when we run two instances within the same Azure cluster we randomly get 503 errors connecting to our endpoints. These resolve themselves eventually - but I am wondering if there are any additional steps I need to setup in order to handle multi-instance applications with port sharing?