tSNE cuML taking a long time to run/ not running but returning no error

Question

I recently installed RAPIDs in an conda environment using WSL2 on my Windows laptop. I used the following command to install it:

conda create --solver=libmamba -n rapids-24.02 -c rapidsai -c conda-forge -c nvidia  \
    rapids=24.02 python=3.10 cuda-version=12.0 \
    jupyterlab tensorflow pytorch

I am using a laptop with a GTX 4050 laptop GPU and an AMD Ryzen 7 7840HS processor.

Using sample code (shown below) everything is running fine:

# Reduce Dimensionality with TSNE
import numpy as np
import cudf
from cuml.manifold import TSNE
# Generate fake data
np.random.seed(42)
images = np.random.rand(10000, 10000)  # 100 samples, each with 784 features (fake image data)

def tsne_2D (df):
    
    print('Starting t-SNE')
    tsne = TSNE(n_components=2, method='barnes_hut', random_state=42)
    tsne_result = tsne.fit_transform(df)

    # Perform t-SNE
    #tsne = TSNE(n_components=2, random_state=42, perplexity = 30, n_iter = 1000, learning_rate = 'auto')
    #tsne_result = tsne.fit_transform(df)
    print("Shape of tSNE result:", tsne_result.shape)

    return tsne_result

test = tsne_2D(images)

However, when I try to use this as part of a batch processing code, it seems to not work but also doesnt return any errors. I feel I'm likely missing something simple but other processing methods (i.e. PCA based) are running without issues

Answer 1

What I have learned, and am putting here in case others find it helpful, is that the reason this wasn't working is because the algorithm couldn't converge on a solution. changing some of the parameters of the tSNE algorithm now has the code running.

Therefore, the error wasn't with the implementation of the code, but rather its application to my dataset