Ktor: HttpRequestTimeoutException when downloading file (but Chrome downloads successfully)

43 Views Asked by At

I'm encountering io.ktor.client.plugins.HttpRequestTimeoutException errors when using Ktor to download a file from the public API endpoint https://public.bybit.com/trading/10000000AIDOGEUSDT/10000000AIDOGEUSDT2024-03-05.csv.gz.

Interestingly, I can download the same file successfully using a web browser like Chrome.

Code Snippet:

suspend fun downloadFile(
url: String,
destination: File
): File {
    val response = httpClient.get(url).bodyAsChannel()
    val sink = destination.writeChannel()
    response.copyTo(sink)
    sink.close()
    return destination
}

HttpClient(CIO) {
    this.install(ContentNegotiation) {
        json(
            Json {
                ignoreUnknownKeys = true
                prettyPrint = true
                isLenient = true
            }
        )
    }
    this.install(HttpTimeout) {
        requestTimeoutMillis = 20000
    }  
} 

Error:

Exception in thread "DefaultDispatcher-worker-7" io.ktor.client.plugins.HttpRequestTimeoutException: Request timeout has expired [url=https://public.bybit.com/trading/10000000AIDOGEUSDT/10000000AIDOGEUSDT2024-03-05.csv.gz, request_timeout=20000 ms]
at io.ktor.client.plugins.HttpTimeout$Plugin$install$1$1$killer$1.invokeSuspend(HttpTimeout.kt:165)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@75171877, Dispatchers.IO]

Problem:

  • The download consistently fails with HttpRequestTimeoutException despite having a 20-second timeout set.
  • Downloading the same file from the same URL works using a web browser.

Question:

  • What could be causing the HttpRequestTimeoutException when using Ktor to download the file, even though a web browser can download it successfully?
  • Are there specific considerations or configurations I need to be aware of when downloading data using Ktor?
  • Are there alternative approaches or troubleshooting steps I can try to overcome this timeout issue?
0

There are 0 best solutions below