Learning big (image) data via coresets for dictionaries

Dan Feldman, Micha Feigin, Nir Sochen

Research output: Contribution to journalArticlepeer-review

Abstract

Signal and image processing have seen an explosion of interest in the last few years in a new form of signal/image characterization via the concept of sparsity with respect to a dictionary. An active field of research is dictionary learning: the representation of a given large set of vectors (e.g. signals or images) as linear combinations of only few vectors (patterns). To further reduce the size of the representation, the combinations are usually required to be sparse, i.e., each signal is a linear combination of only a small number of patterns. This paper suggests a new computational approach to the problem of dictionary learning, known in computational geometry as coresets. A coreset for dictionary learning is a small smart non-uniform sample from the input signals such that the quality of any given dictionary with respect to the input can be approximated via the coreset. In particular, the optimal dictionary for the input can be approximated by learning the coreset. Since the coreset is small, the learning is faster. Moreover, using merge-and-reduce, the coreset can be constructed for streaming signals that do not fit in memory and can also be computed in parallel. We apply our coresets for dictionary learning of images using the K-SVD algorithm and bound their size and approximation error analytically. Our simulations demonstrate gain factors of up to 60 in computational time with the same, and even better, performance. We also demonstrate our ability to perform computations on larger patches and high-definition images, where the traditional approach breaks down.

Original languageEnglish
Pages (from-to)276-291
Number of pages16
JournalJournal of Mathematical Imaging and Vision
Volume46
Issue number3
DOIs
StatePublished - Jul 2013
Externally publishedYes

Keywords

  • Coresets
  • Dictionary learning
  • K-SVD
  • Sparsity

ASJC Scopus subject areas

  • Statistics and Probability
  • Modeling and Simulation
  • Condensed Matter Physics
  • Computer Vision and Pattern Recognition
  • Geometry and Topology
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Learning big (image) data via coresets for dictionaries'. Together they form a unique fingerprint.
  • From high definition image to low space optimization

    Feigin, M., Feldman, D. & Sochen, N., 2012, Scale Space and Variational Methods in Computer Vision - Third International Conference, SSVM 2011, Revised Selected Papers. p. 459-470 12 p. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); vol. 6667 LNCS).

    Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

    Open Access

Cite this