How can I keep the group of clusters that are inside the lane?

Question

I am using DBSCAN to do line clustering with respect to the output of a neural network. The problem is that I would like to keep the points of the cluster group inside the lane. I have calculated the centroids of each cluster grouping but sometimes if more cluster groupings appear, the distance from that centroid to the middle of my image which in this case would be at x = 160, I would choose it.

What do you think? The lane paths can be straight lines or curves.

I leave you 2 images: The first image is the clustering with its centroids, and the second image is the output of the neural network

I leave you the code where I make DBSCAN and I keep the clusters that are on the right and left, but I would like to keep the clusters of interest that are in my lane:

def clustering(self,img,cv_image):

        """
        Calculate clusters with DBSCAN algorithm and we save each cluster with his points and his centroids

        Args: 
               img:  normalise image ll_seg_out (out yolop)

        Return: 
               dict_clusters: dictionary with all clusters {id_cluster,points,centroid}
        """
        #--Convert image in points. Img is out network
        points_lane = np.column_stack(np.where(img > 0))
        dbscan = DBSCAN(eps=22, min_samples=15,metric="euclidean")
        left_clusters = []
        right_clusters = []
        

        if points_lane.size > 0:
            dbscan.fit(points_lane)
            labels = dbscan.labels_

            # Ignore noise if present
            clusters = set(labels)
            if -1 in clusters:
                clusters.remove(-1)
            #n_clusters_ = len(clusters)
            print("Clusters: " + str(len(clusters)))
          

            for cluster in clusters:
                points_cluster = points_lane[labels==cluster,:]
                color = self.colors[cluster % len(self.colors)]
                centroid = points_cluster.mean(axis=0).astype(int)
               
              

                cv2.circle(cv_image, tuple(centroid[::-1]), 3, (0, 0, 0), -1) 
                cv_image[points_cluster[:,0],points_cluster[:,1]] = color

               
                if (centroid[1] < img.shape[1] / 2):
                   
                    left_clusters.append(points_cluster)
                    
                    
                else:
                    
                  
                    right_clusters.append(points_cluster)
                
            return left_clusters,right_clusters,cv_image
        
        else:
            return None,None