Compression by replication

Abstract

A recently introduced inference method based on system replication and an online message passing algorithm is employed to complete a previously suggested compression scheme based on a nonlinear perceptron. The algorithm is shown to approach the information theoretical bounds for compression as the number of replicated systems increases, offering superior performance compared to basic message passing algorithms. In addition, the suggested method does not require fine-tuning of parameters or other complementing heuristic techniques, such as the introduction of inertia terms, to improve convergence rates to nontrivial results.