MapReduce is Good Enough? If All You Have is a Hammer, Throw Away Everything That's Not a Nail!
Jimmy Lin

TL;DR
This paper argues that MapReduce is sufficiently versatile for big data analysis, advocating for simplifying problem approaches rather than developing new models for non-nail algorithms.
Contribution
The paper challenges the need for alternative models by emphasizing problem reformulation over new programming paradigms for non-iterative algorithms.
Findings
Iterative algorithms are poorly suited for MapReduce.
Replacing algorithms with non-iterative solutions can be effective.
Simplifying problem approaches can reduce complexity in big data processing.
Abstract
Hadoop is currently the large-scale data analysis "hammer" of choice, but there exist classes of algorithms that aren't "nails", in the sense that they are not particularly amenable to the MapReduce programming model. To address this, researchers have proposed MapReduce extensions or alternative programming models in which these algorithms can be elegantly expressed. This essay espouses a very different position: that MapReduce is "good enough", and that instead of trying to invent screwdrivers, we should simply get rid of everything that's not a nail. To be more specific, much discussion in the literature surrounds the fact that iterative algorithms are a poor fit for MapReduce: the simple solution is to find alternative non-iterative algorithms that solve the same problem. This essay captures my personal experiences as an academic researcher as well as a software engineer in a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBig Data and Business Intelligence · Data Quality and Management
