Memory problem with a data frame of 200k records

158 Views Asked by At

I got 2 matrices each with 200k records (one is a large get_sentences - review_phrases, the other is review_scores). Binded them in to a data frame and need to write it on a csv but get a memory error. What should i do? Do the packages bigmemory or ff help?

I'm using Windows 10 64bit 8GB RAM. memory limits give the following output:

memory.limit()
[1] 1.759219e+13
memory.size()
[1] 418.85

Here's the code:

  sentiment_matrix_final = data.frame (Comment = review_phrases)

 df_scores = data.frame(Scores = review_scores)

 sentiment_matrix_final = cbind(sentiment_matrix_final, df_scores)

 sentiment_matrix_final = data.frame (Comment = review_phrases, Score = review_scores)

 str(sentiment_matrix_final)


  # Write CSV in R
  write.csv(sentiment_matrix_final, file = "Score_sentiment_test.csv")

I get the output:

>sentiment_matrix_final = data.frame (Comment = review_phrases)

> df_scores = data.frame(Scores = review_scores)

> sentiment_matrix_final = cbind(sentiment_matrix_final, df_scores)
Error: cannot allocate vector of size 750 Kb
0

There are 0 best solutions below