You need to build Spark before running this program error when running bin/pyspark

Question

I am getting started with Spark.

I am getting an issue when starting spark.

I downloaded from spark official website, I am trying to quickstart from this https://spark.apache.org/docs/0.9.0/quick-start.html

I have downloaded, setup sparkhome., and when I tried running ./pyspark from bin directory I get this error :

soundarya@soundarya:~/Downloads/spark-1.5.0/bin$ sudo ./pyspark
[sudo] password for soundarya: 
ls: cannot access /home/soundarya/Downloads/spark-1.5.0/assembly/target/scala-2.10: No such file or directory
Failed to find Spark assembly in /home/soundarya/Downloads/spark-1.5.0/assembly/target/scala-2.10.
You need to build Spark before running this program.
soundarya@soundarya:~/Downloads/spark-1.5.0/bin$ 

Can anyone help me to figure out this?

Answer 1

If you want to use source distribution you have to build Spark first. You can find details in the Building Spark.

Otherwise just download a binary distribution (select pre-built) from downloads:

Also don't use outdated documentation. You can use either the latest:

http://spark.apache.org/docs/latest

or version specific

http://spark.apache.org/docs/1.5.0

Answer 2

Download the pre-build version of spark like zero323 suggested. Also, It is not a good practice to run spark in sudo mode.

Answer 3

Check space in your spark folder name like spark 2.4 then remove it and make spark-2.4 will fix issue