Interested to Buy Any Domain ? << Click Here >> for more details...
How does Apache Spark handles accumulated Metadata?
- 1 Answers
- 367 Views
- I also Faced
- E-Mail Answers
Answer / Peeyush Tripathi
Apache Spark manages accumulated metadata using a structure called the lineage. Each RDD in the computation has a lineage, which is a record of all its ancestors. This allows Spark to trace back the history of data and recalculate any RDD that is needed again if it or one of its dependencies fails. Additionally, Spark periodically prunes the lineage graph to reduce memory usage.
| Is This Answer Correct ? | 0 Yes | 0 No |
What are the various advantages of DataFrame over RDD in Apache Spark?
Explain about trformations and actions in the context of rdds?
What is spark database?
What is difference between map and flatmap?
Is rdd type safe?
How can you compare Hadoop and Spark in terms of ease of use?
Can we run Apache Spark without Hadoop?
What is lineage graph in spark?
How do I download adobe spark?
How can you implement machine learning in Spark?
Why apache spark is faster than hadoop?
What is executor memory and driver memory in spark?
-
Apache Hadoop (394) - MapReduce (354)
- Apache Hive (345)
- Apache Pig (225)
- Apache Spark (991)
- Apache HBase (164)
- Apache Flume (95)
- Apache Impala (72)
- Apache Cassandra (392)
- Apache Mahout (35)
- Apache Sqoop (82)
- Apache ZooKeeper (65)
- Apache Ambari (93)
- Apache HCatalog (34)
- Apache HDFS Hadoop Distributed File System (214)
- Apache Kafka (189)
- Apache Avro (26)
- Apache Presto (15)
- Apache Tajo (26)
- Hadoop General (407)
