Watch a modular network learn to read digits

An interactive companion to Yash Arya and Sang Hoon Lee, “Effects of relational graph modularity and depth on the learning performance of neural networks,” Journal of the Korean Physical Society (2026), doi:10.1007/s40042-026-01730-5. A small graph with communities decides which weights exist in every hidden layer of a multilayer perceptron. Shape the graph, see exactly which weights each node and link turns into, then train the network on handwritten digits in your browser.

Training data: UCI Optical Recognition of Handwritten Digits (8×8 pixels), not MNIST. Details and references are in the notes at the end of the page.

Demo created by Claude Opus 5.5.

Relational graph

Node i stands for neuron i of the hidden layers. A link means two neurons exchange messages; the small loop is a node passing its own state forward.

Weighted matrix of hidden layer 1 to hidden layer 2

Cell (i, j) is wij, the weight from neuron j to neuron i. Blank cells are links the graph doesn’t have, so this matrix is the graph’s weighted adjacency matrix with self-loops on the diagonal.

Layer

From the relational graph to the whole network

positive weight negative weight active neuron dense, not shaped by the graph
Hover a neuron, node or link anywhere to trace it through every layer. Click between two layers to show their weight matrix.

Input and prediction

Pick a test digit or draw your own.

Draw a digit

modular graph fully connected

Test error while training

The fully connected network, the complete graph, is the baseline. Within a run both start from the same weights and see the same batches; only the wiring differs. Bold lines are the run in progress, faint lines are finished runs.

modular graph fully connected baseline

Many runs, same graph

A single training run depends on its random starting weights. Each dot is the final test error of one run; a line joins the modular and fully connected results of the same run, which share starting weights and batches.

modular graph fully connected mean ± 1 standard deviation

Does mixing matter?

Retrain the current setup at six values of μ, five graphs each. This mirrors Fig. 7 of the paper at toy scale.

Each dot is one trained network (12 passes). The line joins the means, the dashed line is the fully connected baseline, and the dotted vertical line is μ = 1 − 1/c, where modularity Q drops to about zero.

Reading the translation

In layer r, each neuron of hidden layer r + 1 sums what its graph neighbours in hidden layer r send, itself included (Eq. 1):

xi(r+1) = ReLU( Σj ∈ N(i) wij(r) xj(r) )

A node i is neuron i, present in every hidden layer; its self-loop links neuron i in one layer to neuron i in the next. A link i–j becomes two weights per layer, wij and wji. A self-loop becomes wii, so a neuron’s own state isn’t discarded. A missing link fixes both weights at zero.

Depth is the number of hidden layers. This demo uses 5, as in the paper’s main results, so there are 4 graph-shaped weight layers between them, each with its own weights but the same wiring. The 64 pixels reach hidden layer 1 densely, and hidden layer 5 reaches the 10 digits densely.

Communities make each weight matrix nearly block-diagonal. Each community is built as an ER or static scale-free graph and trimmed to its largest connected component (the paper’s footnote 2). Then a fraction μ of all links is rewired to join different communities, keeping every node’s degree, so the number of trainable weights doesn’t change. Expected modularity is Q ≈ (1 − μ) − 1/c.

The paper found that on CIFAR-10, sparse modular graphs beat the baseline at depth 5, error fell as μ rose, and the advantage reversed at depth 8. This demo uses a much smaller dataset and network, so its results are noisy and needn’t match the paper’s.

Training and test data

The demo trains on the Optical Recognition of Handwritten Digits dataset from the UCI Machine Learning Repository, in the preprocessed version bundled with scikit-learn as load_digits.

The set has 1,797 handwritten digits from 0 to 9, written by 43 people. Each 32×32 bitmap was reduced to an 8×8 image by counting the inked pixels in each 4×4 block, giving 64 inputs with values from 0 to 16, which the demo scales to the range 0 to 1.

The images are shuffled once with a fixed seed. The first 1,497 are used for training and the last 300 for testing, and every error rate on this page is top-1 error on those 300 test digits. The whole set is embedded in the page, so nothing is downloaded. Digits you draw are converted to the same 8×8 format.

References. E. Alpaydin and C. Kaynak, “Optical Recognition of Handwritten Digits,” UCI Machine Learning Repository (1998), doi:10.24432/C50P49, licensed CC BY 4.0. F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Journal of Machine Learning Research 12, 2825–2830 (2011).

Disclaimer

This demo is a visual illustration of the method, not a replication of the paper’s results. The paper used CIFAR-10, 128-node graphs and 200 epochs on a GPU; this demo uses the much smaller UCI digits set and a smaller network, so its numbers are noisy and needn’t match the paper’s. In informal runs with the default settings, the modular and fully connected networks were within noise of each other.

References. CIFAR-10: A. Krizhevsky, “Learning Multiple Layers of Features from Tiny Images,” Technical Report, University of Toronto (2009), www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf. MNIST, mentioned at the top of this page: Y. LeCun, L. Bottou, Y. Bengio and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE 86, 2278–2324 (1998), doi:10.1109/5.726791.