Spark app returns IOException, "Failed to open native connection to Cassandra at {127.0.0.1}:9042"

Question

My Versions:

• spark-2.1.1-bin-hadoop2.7,
• kafka_2.11-0.9.0.0,
• apache-cassandra-3.9.

val sparkConf = new SparkConf().setAppName("KafkaSparkStreaming").set("spark.cassandra.connection.host", "127.0.0.1")

val ssc = new StreamingContext(sparkConf, Seconds(5))

val topicpMap = "mytopic".split(",").map((_, 1.toInt)).toMap

val lines = KafkaUtils.createStream(ssc, "localhost:2181", "sparkgroup", topicpMap).map(_._2)

lines.map(line => { val arr = line.split(","); (arr(0),arr(1),arr(2),arr(3),arr(4)) }).saveToCassandra("sparkdata", "cust_data", SomeColumns("fname", "lname","url","product","cnt"))

I tried to run this command but i got the error :

lines.map(line => { val arr = line.split(","); (arr(0),arr(1),arr(2),arr(3),arr(4)) }).saveToCassandra("sparkdata", "cust_data", SomeColumns("fname", "lname","url","product","cnt"))

the error i am getting:

java.io.IOException: Failed to open native connection to Cassandra at {127.0.0.1}:9042
  at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:168)
  at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$8.apply(CassandraConnector.scala:154)
  at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$8.apply(CassandraConnector.scala:154)
  at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:32)
  at com.datastax.spark.connector.cql.RefCountedCache.syncAcquire(RefCountedCache.scala:69)
  at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:57)
  at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:79)
  at com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:111)
  at com.datastax.spark.connector.cql.CassandraConnector.withClusterDo(CassandraConnector.scala:122)
  at com.datastax.spark.connector.cql.Schema$.fromCassandra(Schema.scala:330)
  at com.datastax.spark.connector.cql.Schema$.tableFromCassandra(Schema.scala:350)
  at com.datastax.spark.connector.writer.TableWriter$.apply(TableWriter.scala:336)
  at com.datastax.spark.connector.streaming.DStreamFunctions.saveToCassandra(DStreamFunctions.scala:53)
  ... 58 elided
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [/127.0.0.1:9042] Cannot connect))
  at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:233)
  at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:79)
  at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1483)
  at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:399)
  at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala:161)
  ... 70 more

please anybody pointout my error to resolve?

Answer 1

A NoHostAvailableException means the connection cannot be established. That's a lower layer, so before trying any authentication. Thus, your problem is connectivity. Here are some things to check:

1. Are you sure 9042 is the correct port? Did you change the default native port?

2. Is there a firewall? This seems highly unlikely because you're trying to connect to 127.0.0.1:9042.

3. Did you verify that Cassandra is listening on port 9042 (netstat) and that the port is open t

Either way, it's a connectivity issue.

Answer 2

Based on the other question you posted, it looks like you've kicked off your Spark application while Cassandra is still down leading to the NoHostAvailableException.

You need to make sure that Cassandra is operational before you run any applications or clients that connect to it.

Check the Cassandra system.log to monitor the progress of the startup sequence. When you see a log entry that looks like this:

... Starting listening for clients ...

then you know that Cassandra started up successfully and is ready to accept client requests. Cheers!