Compose HorizontalPager swipe conflict with swipe back event

Question

I currently have an issue with HorizontalPager swipe behavior, when perform the Android swipe back gesture from either of the screen edges, the HorizontalPager also swipes to the next image. Here is the GIF of it. I'm using compose UI version 1.3.3

HorizontalPager(
        pageCount = pageCount,
        state = pagerState,
        modifier = Modifier.fillMaxSize()
      ) { page ->
        DefaultAsyncImage(
          modifier = Modifier.fillMaxSize(),
          imageUrl = urls[page],
          aspectRatio = PRODUCT_LAYOUT_ASPECT_RATIO
        )
      }

How can I prevent the HorizontalPager from consuming the swipe gesture when the swipe should only close the Activity?