R: Aggregate an RDD using the given combine functions and a...

aggregateRDD {SparkR}

R Documentation

Aggregate an RDD using the given combine functions and a neutral "zero value".

Description

Aggregate the elements of each partition, and then the results for all the partitions, using given combine functions and a neutral "zero value".

Usage

aggregateRDD(rdd, zeroValue, seqOp, combOp)

## S4 method for signature 'RDD'
aggregateRDD(rdd, zeroValue, seqOp, combOp)

Arguments

`rdd`	An RDD.
`zeroValue`	A neutral "zero value".
`seqOp`	A function to aggregate the RDD elements. It may return a different result type from the type of the RDD elements.
`combOp`	A function to aggregate results of seqOp.

Value

The aggregation result.

Examples

## Not run: 
##D sc <- sparkR.init()
##D rdd <- parallelize(sc, list(1, 2, 3, 4))
##D zeroValue <- list(0, 0)
##D seqOp <- function(x, y) { list(x[[1]] + y, x[[2]] + 1) }
##D combOp <- function(x, y) { list(x[[1]] + y[[1]], x[[2]] + y[[2]]) }
##D aggregateRDD(rdd, zeroValue, seqOp, combOp) # list(10, 4)
## End(Not run)