RankMap: A framework for distributed learning from dense data sets

Azalia Mirhoseini, Eva L. Dyer, Ebrahim M. Songhori, Richard Baraniuk, Farinaz Koushanfar

Research output: Contribution to journalArticle

1 Scopus citations

Abstract

This paper introduces RankMap, a platform-aware end-to-end framework for efficient execution of a broad class of iterative learning algorithms for massive and dense data sets. Our framework exploits data structure to scalably factorize it into an ensemble of lower rank subspaces. The factorization creates sparse low-dimensional representations of the data, a property which is leveraged to devise effective mapping and scheduling of iterative learning algorithms on the distributed computing machines. We provide two APIs, one matrix-based and one graph-based, which facilitate automated adoption of the framework for performing several contemporary learning applications. To demonstrate the utility of RankMap, we solve sparse recovery and power iteration problems on various real-world data sets with up to 1.8 billion nonzeros. Our evaluations are performed on Amazon EC2 and IBM iDataPlex servers using up to 244 cores. The results demonstrate up to two orders of magnitude improvements in memory usage, execution speed, and bandwidth compared with the best reported prior work, while achieving the same level of learning accuracy.

Original languageEnglish (US)
Pages (from-to)2717-2730
Number of pages14
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume29
Issue number7
DOIs
StatePublished - Jul 2018

Keywords

  • Dense and big data
  • iterative machine learning (ML)
  • large-scale distributed computing
  • low rank approximation
  • sparse matrix factorization
  • union of subspaces

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'RankMap: A framework for distributed learning from dense data sets'. Together they form a unique fingerprint.

Cite this