Sparklyr spark_connect error in prepare_windows_environment: FindFileOwnerAndPermission error (1789)

10 Views Asked by At

Attempting to set up a local spark connection in R with spark_connect fails.

sc <- sparklyr::spark_connect(master = "local")

The error message:

Error in prepare_windows_environment(spark_home, environment) : 
  FindFileOwnerAndPermission error (1789): The trust relationship between this workstation and the primary domain failed.
In addition: Warning messages:
1: In system2(winutils, c("ls", shQuote(hivePath)), stdout = TRUE) :
  running command '"C:\Users\*******\AppData\Local\spark\spark-3.3.1-bin-hadoop2\tmp\hadoop\bin\winutils.exe" ls "C:\Users\*******\AppData\Local\spark\spark-3.3.1-bin-hadoop2\tmp\hive"' had status 1
2: In !is.null(output) && grepl("error", output) :
  'length(x) = 2 > 1' in coercion to 'logical(1)'

I have the most recent versions of spark and hadoop installed using spark_install.
sparklyr::spark_install(version = 3.3, hadoop_version = 3)

I've tried uninstalling and reinstalling sparklyr from Github:

devtools::install_github("rstudio/sparklyr")

Manually setting spark version:

sc <- spark_connect(master = "local", version = "3.3")

Setting spark_home to the path of my spark installation:

sc <- spark_connect(master = "local", spark_home = "C:/Users/*******/AppData/Local/spark/spark-3.3.1-bin-hadoop3")

Unsetting SPARK_HOME environment before establishing a connection:

Sys.unsetenv("SPARK_HOME")

How would I go about setting up a local spark connection?

0

There are 0 best solutions below