Nerva is a collection of modular C++ and Python libraries for building and experimenting with neural networks. Originally developed for research on truly sparse neural networks, the Nerva libraries have evolved into a set of well-documented, flexible components that emphasize mathematical clarity and transparent implementation.
Note:
The original development history can be found on themain_oldbranch. This includes all sparse training experiments reported in [1]. The functionality has since been split into independent repositories, each with focused documentation and clean interfaces.
The Nerva project is organized into a set of focused libraries, each targeting a specific language or data layout:
| Repository | Description |
|---|---|
| nerva-rowwise | C++ backend with row-major layout (like PyTorch) |
| nerva-colwise | C++ backend with column-major layout (like MATLAB) |
| nerva-jax | Python implementation using JAX |
| nerva-numpy | Python implementation using NumPy |
| nerva-tensorflow | Python implementation using TensorFlow |
| nerva-torch | Python implementation using PyTorch |
| nerva-sympy | Symbolic Python backend for specification and validation |
The Nerva libraries share a common design:
- ✅ Support for common layers, activation functions, and loss functions
- ✅ Matrix-form equations for all operations, including explicit backpropagation
- ✅ Mini-batch training via matrix algebra
- ✅ Symbolic validation of equations using SymPy
- ✅ Uniform implementations grounded in a small set of matrix primitives
C++ Libraries Only (nerva-rowwise, nerva-colwise):
- ✅ Native CPU performance using Intel MKL
- ✅ Support for truly sparse layers (using CSR representation)
- ✅ Python bindings available for easy integration
⚠️ Currently limited to multilayer perceptrons (MLPs)⚠️ GPU support is not yet available⚠️ No convolutional or transformer layers (planned)
-
Nerva: a Truly Sparse Implementation of Neural Networks
arXiv:2407.17437 Introduces the library and reports sparse training experiments.
-
Batch Matrix-form Equations and Implementation of Multilayer Perceptrons
arXiv:2511.11918 Includes mathematical specifications and derivations.
Questions or contributions are welcome!
Contact: Wieger Wesselink (j.w.wesselink@tue.nl)