Attempting to set up a local spark connection in R with spark_connect fails.
sc <- sparklyr::spark_connect(master = "local")
The error message:
Error in prepare_windows_environment(spark_home, environment) :
FindFileOwnerAndPermission error (1789): The trust relationship between this workstation and the primary domain failed.
In addition: Warning messages:
1: In system2(winutils, c("ls", shQuote(hivePath)), stdout = TRUE) :
running command '"C:\Users\*******\AppData\Local\spark\spark-3.3.1-bin-hadoop2\tmp\hadoop\bin\winutils.exe" ls "C:\Users\*******\AppData\Local\spark\spark-3.3.1-bin-hadoop2\tmp\hive"' had status 1
2: In !is.null(output) && grepl("error", output) :
'length(x) = 2 > 1' in coercion to 'logical(1)'
I have the most recent versions of spark and hadoop installed using spark_install.
sparklyr::spark_install(version = 3.3, hadoop_version = 3)
I've tried uninstalling and reinstalling sparklyr from Github:
devtools::install_github("rstudio/sparklyr")
Manually setting spark version:
sc <- spark_connect(master = "local", version = "3.3")
Setting spark_home to the path of my spark installation:
sc <- spark_connect(master = "local", spark_home = "C:/Users/*******/AppData/Local/spark/spark-3.3.1-bin-hadoop3")
Unsetting SPARK_HOME environment before establishing a connection:
Sys.unsetenv("SPARK_HOME")
How would I go about setting up a local spark connection?