Transparent efficiency for in-memory execution of map reduce job sequences
Executing a map reduce sequence may comprise executing all jobs in the sequence by a collection of a plurality of processes with each process running one or more mappers, combiners, partitioners and reducers for each job, and transparently sharing heap state between the jobs to improve metrics assoc...
Saved in:
Main Authors | , , , |
---|---|
Format | Patent |
Language | English |
Published |
29.09.2015
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | Executing a map reduce sequence may comprise executing all jobs in the sequence by a collection of a plurality of processes with each process running one or more mappers, combiners, partitioners and reducers for each job, and transparently sharing heap state between the jobs to improve metrics associated with the job. Processes may communicate among themselves to coordinate completion of map, shuffle and reduce phases, and completion of said all jobs in the sequence. |
---|---|
Bibliography: | Application Number: US201213594788 |