Backpropagation-free CNN training edges toward hardware-friendly, Dale-compliant learning

Sakana AI has published a paper, Diffusing Blame, that demonstrates the most capable Dale-compliant, backpropagation-free learning to date, training convolutional neural networks and reinforcement learning agents without backpropagation. The method achieves 96.7% on MNIST and 61.7% on CIFAR-10, using an algorithm called Error Diffusion with three key innovations. For AI builders, this work opens a path to training networks on neuromorphic and photonic hardware that cannot run standard backpropagation, and it produces implicit model compression of up to 68.8% sparsity in some layers.

What happened

The paper, posted to arXiv on July 17, extends Error Diffusion, a 25-year-old algorithm, to convolutional networks and reinforcement learning for the first time. Error Diffusion updates weights using only three local signals: the activity of the sending neuron, the derivative of the activation at the receiving neuron, and a single global error sign. This avoids the biologically implausible weight transport required by backpropagation.

Sakana AI's team added three innovations to make Error Diffusion work on CNNs and multi-class problems:

• Dual-stream architecture: Each layer is split into excitatory and inhibitory streams, keeping all trainable weights non-negative. Cross-stream connections are fixed as inhibitory via hardcoded negation signs.
• Modulo error routing: Each hidden unit is deterministically linked to a fixed output channel (unit i gets error from channel i modulo C), enabling multi-class classification without random feedback matrices.
• Classification-specific corrections: Layer-specific sigmoid widths prevent error signal decay (measured at 25x from output to first hidden layer), batch-centered class error corrects a suppression bias from 9-to-1 positive-negative target imbalance, and asymmetric initialization gives the excitatory stream a 3-to-1 starting advantage.

The ablation study reveals that the importance of these corrections is task-dependent: on MNIST, removing sigmoid widths collapses accuracy by 71.4 points, while on CIFAR-10, removing batch-centering causes a 47.9-point collapse and four of five training runs fail.

Why AI builders should care

The primary practical value is hardware compatibility. Neuromorphic chips like Intel's Loihi 2 and photonic systems encode synaptic weights as non-negative physical quantities. Standard backpropagation requires arbitrary-sign weights, but Error Diffusion's non-negative weight constraint and fixed-sign inhibitory routing map directly onto these substrates. This means models trained with Error Diffusion could run natively on hardware that conventional deep learning cannot use.

A secondary benefit is implicit model compression. After training, [37.3% of all weights converge to the minimum allowed value and are effectively pruned](https