Applications of Spiking Neural Networks in Visual Place Recognition

Somayeh Hussaini1, Michael Milford1, Tobias Fischer1,
1Queensland University of Technology (QUT) Centre for Robotics
Preprint Video Code
Nerfies teaser image.

Our approach builds on independent Spiking Neural Network (SNN) modules, each trained on small subsets of the reference database. During inference, these modules' place predictions are fused in parallel within a "Standalone Modular SNN," enabling scalability across numerous locations. We leverage the parallel processing potential of neuromorphic processors by using ensembles, where multiple Modular SNNs represent the same place, showing that SNNs are highly responsive to ensembling. Lastly, we highlight the effectiveness of these Ensembles of Modular SNNs in sequence matching.

Abstract

In robotics, Spiking Neural Networks (SNNs) are increasingly recognized for their largely-unrealized potential energy efficiency and low latency particularly when implemented on neuromorphic hardware. Our paper highlights three advancements for SNNs in Visual Place Recognition (VPR).

Firstly, we propose Modular SNNs, where each SNN represents a set of non-overlapping geographically distinct places, enabling scalable networks for large environments. Secondly, we present Ensembles of Modular SNNs, where multiple networks represent the same place, significantly enhancing accuracy compared to single-network models. Each of our Modular SNN modules is compact, comprising only 1500 neurons and 474k synapses, making them ideally suited for ensembling due to their small size. Lastly, we investigate the role of sequence matching in SNN-based VPR, a technique where consecutive images are used to refine place recognition.

We demonstrate competitive performance of our method on a range of datasets, including higher responsiveness to ensembling compared to conventional VPR techniques and higher R@1 improvements with sequence matching than VPR techniques with comparable baseline performance. Our contributions highlight the viability of SNNs for VPR, offering scalable and robust solutions, and paving the way for their application in various energy-sensitive robotic tasks.

Video

Proof-of-concept Robot Deployment

We deployed our Modular SNN on an AgileX Scout Mini robot for real-time place recognition in a 100-meter path, running on a CPU. Our system successfully achieved 75% place recognition accuracy, with processing times of 1.1 to 2 seconds per image. This highlights its potential for future neuromorphic hardware deployment for further enhanced performance.

BibTeX

@article{hussaini2023applications,
      title={Applications of Spiking Neural Networks in Visual Place Recognition},
      author={Hussaini, Somayeh and Milford, Michael and Fischer, Tobias},
      journal={arXiv preprint arXiv:2311.13186},
      year={2023} 
}