## Abstract

How many bits of information are revealed by a learning algorithm for a concept class of VC-dimension d? Previous works have shown that even for d = 1 the amount of information may be unbounded (tend to ∞ with the universe size). Can it be that all concepts in the class require leaking a large amount of information? We show that typically concepts do not require leakage. There exists a proper learning algorithm that reveals O(d) bits of information for most concepts in the class. This result is a special case of a more general phenomenon we explore. If there is a low information learner when the algorithm knows the underlying distribution on inputs, then there is a learner that reveals little information on an average concept without knowing the distribution on inputs.

Original language | English |
---|---|

Pages (from-to) | 633-646 |

Number of pages | 14 |

Journal | Proceedings of Machine Learning Research |

Volume | 98 |

State | Published - 2019 |

Externally published | Yes |

Event | 30th International Conference on Algorithmic Learning Theory, ALT 2019 - Chicago, United States Duration: 22 Mar 2019 → 24 Mar 2019 |

### Bibliographical note

Publisher Copyright:© 2019 Proceedings of Machine Learning Research. All rights reserved.

## Keywords

- Learning Theory, Information Theory

## ASJC Scopus subject areas

- Artificial Intelligence
- Software
- Control and Systems Engineering
- Statistics and Probability