Abstract
For animals to navigate an uncertain world, their brains need to estimate uncertainty at the timescales of sensations and actions. Sampling-based algorithms afford a theoretically-grounded framework for probabilistic inference in neural circuits, but it remains unknown how one can implement fast sampling algorithms in biologically-plausible spiking networks. Here, we propose to leverage the population geometry, controlled by the neural code and the neural dynamics, to implement fast samplers in spiking neural networks. We first show that two classes of spiking samplers—efficient balanced spiking networks that simulate Langevin sampling, and networks with probabilistic spike rules that implement Metropolis-Hastings sampling—can be unified within a common framework. We then show that careful choice of population geometry, corresponding to the natural space of parameters, enables rapid inference of parameters drawn from strongly-correlated high-dimensional distributions in both networks. Our results suggest design prin-ciples for algorithms for sampling-based probabilistic inference in spiking neural networks, yielding potential inspiration for neuromorphic computing and testable predictions for neurobiology.
Competing Interest Statement
The authors have declared no competing interest.
Footnotes
paul_masset{at}fas.harvard.edu, jzavatoneveth{at}g.harvard.edu, cpehlevan{at}seas.harvard.edu
↵† VNM and CP jointly supervised this work.
NeurIPS 2022 Proceedings Version
3 In Appendix B.3, we show how the elastic net prior on firing rates used by Boerlin et al. [45] can be incorporated into this model. As it only modifies the definitions of the recurrent weights and spiking threshold, this extension does not add new conceptual difficulties, hence we do not discuss it further in the main text. Additionally, we provide a pedagogical introduction to the dynamics of the EBN in Appendix C.
4 In Appendix C, we give a concrete interpretation of this task in terms of the Gaussian linear models used in previous work by Hennequin et al. [9] and Savin and Denève [20].
5 Here, we chose to include the regularizing term α as a fixed offset in the voltage potential but it can equivalently be included as an offset in the spiking threshold Ti, as discussed in Appendix B.3.