R: Apply a function to each partition of an RDD

lapplyPartition {SparkR}

R Documentation

Apply a function to each partition of an RDD

Description

Return a new RDD by applying a function to each partition of this RDD.

mapPartitions is the same as lapplyPartition.

Usage

lapplyPartition(X, FUN)

## S4 method for signature 'RDD,'function''
lapplyPartition(X, FUN)

mapPartitions(X, FUN)

## S4 method for signature 'RDD,'function''
mapPartitions(X, FUN)

Arguments

`X`	The RDD to apply the transformation.
`FUN`	the transformation to apply on each partition.

Value

a new RDD created by the transformation.

Examples

## Not run: 
##D sc <- sparkR.init()
##D rdd <- parallelize(sc, 1:10)
##D partitionSum <- lapplyPartition(rdd, function(part) { Reduce("+", part) })
##D collect(partitionSum) # 15, 40
## End(Not run)

[Package SparkR version 0.1 Index]