Android RecyclerView item reorder infinite loop issue

Question

I'm having a RecyclerView with GridLayoutManager. I've implemented item reordering so user can reorder the items using ItemTouchHelper & SimpleCallback.

But there is a problem of infinite loop when reordering. To better understand the issue, see the attached video:

The code for reordering is as follows:

override fun onMove(
    recyclerView: RecyclerView,
    viewHolder: RecyclerView.ViewHolder,
    target: RecyclerView.ViewHolder,
): Boolean {
    val from = viewHolder.adapterPosition
    val to = target.adapterPosition
    val newSwitches = switches.toMutableList()
    // 1
    if (from < to) for (i in from until to) Collections.swap(newSwitches, i, i + 1)
    else for (i in from downTo to + 1) Collections.swap(newSwitches, i, i - 1)
    // Or 2
    Collections.swap(newList, from, to)
    setSwitches(newSwitches)
    return true
}

In code above, approach 1 as well as 2 results in an infinite loop issue as shown in video.

When I print from & to values to console they're like:

from: 10, to: 11
from: 11, to: 10
from: 10, to: 11
from: 11, to: 10
and so on...

PS: I'm using edge-to-edge to draw behind system bars & applying extra padding to the RecyclerView with clipChildren="false" & clipToPadding="false". Could the issued be due to this?

Edit 1: setSwitches(newSwitches) uses DiffUtil.