Caching in Apache Spark stores RDDs (Resilient Distributed Datasets) in mem

What is the difference between Caching and Persistence in Apache Spark?

Question Posted / Ravi

1 Answers
347 Views
I also Faced
E-Mail Answers

Answer Posted / Ravi

Caching in Apache Spark stores RDDs (Resilient Distributed Datasets) in memory for faster access during subsequent tasks. Once cached, the data remains in memory until explicitly cleared or the application terminates. Persistence, on the other hand, saves DataFrames or Datasets to an external storage system like HDFS or Cassandra for long-term storage.

Is This Answer Correct ?

0 Yes

0 No

Post New Answer View All Answers

Please Help Members By Posting Answers For Below Questions

What is the latest version of spark?

287

Explain how RDDs work with Scala in Spark

355

List the advantage of Parquet file in Apache Spark?

473

What is meant by Transformation? Give some examples.

328