ScrollBy and scrollTo conflicting with browser scrolling

Question

I have created a simple carousel based on a native browser scrolling behaviour with the use of scroll-behavior: smooth; and scroll-snap technique. Works perfectly and is progressively enhanced. I trigger the carousel with js with scrollBy and scrollTo so it would act like a typical carousel. However when a user tries to navigate to a section on a page with a href="#section-name and the carousel runs, the user initiated vertical scroll stops before reaching its destination. Am I being dumb here and this a natural browser behaviour - you cannot have two scrolls at the same time? If so, how would you go about it? I want the carousel to run in a loop but if this brakes the navigation it does not make sense. Here's a demo: https://codepen.io/rwaniek/pen/jOzNQpz