Reverse Derivative Ascent: A Categorical Approach to Learning Boolean Circuits

Paul Wilson
(University of Southampton)
Fabio Zanasi
(University College London)

We introduce Reverse Derivative Ascent: a categorical analogue of gradient based methods for machine learning. Our algorithm is defined at the level of so-called reverse differential categories. It can be used to learn the parameters of models which are expressed as morphisms of such categories. Our motivating example is boolean circuits: we show how our algorithm can be applied to such circuits by using the theory of reverse differential categories. Note our methodology allows us to learn the parameters of boolean circuits directly, in contrast to existing binarised neural network approaches. Moreover, we demonstrate its empirical value by giving experimental results on benchmark machine learning datasets.

In David I. Spivak and Jamie Vicary: Proceedings of the 3rd Annual International Applied Category Theory Conference 2020 (ACT 2020), Cambridge, USA, 6-10th July 2020, Electronic Proceedings in Theoretical Computer Science 333, pp. 247–260.
Published: 8th February 2021.

ArXived at: https://dx.doi.org/10.4204/EPTCS.333.17 bibtex PDF
References in reconstructed bibtex, XML and HTML format (approximated).
Comments and questions to: eptcs@eptcs.org
For website issues: webmaster@eptcs.org