Hebbian learning ("neurons that fire together, wire together") strengthens the connection between two neurons proportionally to their simultaneous activity. Applied to a competitive winner-take-all output layer it produces unsupervised clustering: the winning neuron's weights drift toward whichever input pattern triggered it, eventually specializing for one cluster. But pure Hebbian updating is unstable — one neuron monopolizes all patterns unless a stabilizing constraint is added. Two routes to that fix are compared below. The 2D explorer uses the same three modes as the 9-pixel demo in Tab ②.
For the output neuron j with the largest dot product between its weights and the inputs (the "winning" neuron), update each weight wij from input i to neuron j by the Hebbian rule:
where η is the learning rate and output Nj[k] will be 1 if the winning dot product is positive.
To implement lateral inhibition (LI), no other output neurons will be updated and each "loser" neuron ℓ will have Nℓ=0.
No constraint on weight growth — collapse: the first neuron to win a slight majority starts winning everything. In the explorer, vectors are soft-capped at magnitude 1.6 to keep them visible.
As in Mode ①, the winning output neuron will be updated (and the others will not), but the result will then be normalized. That is:
Dividing by the L2 norm ensures a unit magnitude after every update, placing the winner on the unit sphere. Strengthening weights toward one cluster's direction forces other directions to shrink — a fixed angular budget, not arbitrary growth. Arrows converge to point at distinct cluster centers.
As with the implementation of LI in Mode ①, each loser neuron ℓ will have Nℓ=0.
As in Mode ①, the winning output neuron will be updated (and the others will not), but losers will also be updated using an anti-Hebbian rule. That is, for the winner neuron j:
and for each "loser" neuron ℓ:
where α is the anti-Hebbian rate (0 < α ≤ 1; here α=0.4), which controls how strongly losers are depressed relative to the winner's potentiation. Note that the depression is applied to the loser regardless of whether it would have been activated without the LI. As in Modes ① and ②, each loser neuron ℓ will have Nℓ=0.
Where Mode ② limits growth by rescaling, Mode ③ provides direct competitive depression — losers are weakened proportionally to how strongly each input activated them, as described by Rumelhart & Zipser (1985). This method is less likely to result in output specialization on input clusters than Mode ② normalization because depression can eliminate a loser's ability to compete before it specializes.
The same 9-pixel patterns used in the memristor widget are clustered here with a conventional ANN and Hebbian weight updates — no spike timing, no ferroelectric physics. Selecting Mode ① demonstrates the collapse that motivates the other two modes. Modes ② and ③ both converge, through different mechanisms. Switching between modes and comparing to the memristor widget illustrates that the same clustering behavior can arise from very different physical substrates.