How can you manually partition the rdd?
Answer / Mohit Kumar Sharma
Manual partitioning of RDDs can be achieved by using the glom() and mapPartitions() transformation functions. For example: rdd.glom().mapPartitions(iter => iter.toList.splitAll(partitionSize)).flatMap(_.iterator).collect()
| Is This Answer Correct ? | 0 Yes | 0 No |
What is spark parallelize?
Where is spark used?
How is dag created in spark?
Why is the spark so fast?
Name types of Cluster Managers in Spark.
How can we launch Spark application on YARN?
Explain different transformation on DStream?
What is spark application?
Name the Spark Library which allows reliable file sharing at memory speed across different cluster frameworks.
What is the difference between spark and scala?
What is amazon spark?
What is pagerank?
Apache Hadoop (394)
MapReduce (354)
Apache Hive (345)
Apache Pig (225)
Apache Spark (991)
Apache HBase (164)
Apache Flume (95)
Apache Impala (72)
Apache Cassandra (392)
Apache Mahout (35)
Apache Sqoop (82)
Apache ZooKeeper (65)
Apache Ambari (93)
Apache HCatalog (34)
Apache HDFS Hadoop Distributed File System (214)
Apache Kafka (189)
Apache Avro (26)
Apache Presto (15)
Apache Tajo (26)
Hadoop General (407)