Improving and Extending the Testing of Distributions for Shape-Restricted Properties

• Published in: Algorithmica Vol. 81; no. 9; pp. 3765 - 3802
• Main Authors: Fischer, E., Lachish, O., Vasudev, Y.
• Format: Journal Article
• Language: English
• Published: New York Springer US 16.09.2019; Springer Nature B.V
• Subjects: Algorithm Analysis and Problem Complexity; Algorithms; Computer Science; Computer Systems Organization and Communication Networks; Data Structures and Information Theory; Decomposition; Mathematics of Computing; Partitions; Properties (attributes); Theory of Computation; Conditional samples; Decomposability; Testing; Distributions
• ISSN: 0178-4617; 1432-0541
• DOI: 10.1007/s00453-019-00598-1

Distribution testing deals with what information can be deduced about an unknown distribution over { 1 , … , n } , where the algorithm is only allowed to obtain a relatively small number of independent samples from the distribution. In the extended conditional sampling model, the algorithm is also allowed to obtain samples from the restriction of the original distribution on subsets of { 1 , … , n } . In 2015, Canonne, Diakonikolas, Gouleakis and Rubinfeld unified several previous results, and showed that for any property of distributions satisfying a “decomposability” criterion, there exists an algorithm (in the basic model) that can distinguish with high probability distributions satisfying the property from distributions that are far from it in the variation distance. We present here a more efficient yet simpler algorithm for the basic model, as well as very efficient algorithms for the conditional model, which until now was not investigated under the umbrella of decomposable properties. Additionally, we provide an algorithm for the conditional model that handles a much larger class of properties. Our core mechanism is an algorithm for efficiently producing an interval-partition of { 1 , … , n } that satisfies a “fine-grain” quality. We show that with such a partition at hand we can avoid the search for the “correct” partition of { 1 , … , n } .