Personality in weight space 134 trait adapters · Qwen3.5-4B

← Methods

Geometry

The exact Gram, the factor analysis, and the controls

The geometric object is 134 LoRA weight updates compared through an exact Gram: the inner products are computed on the factored form, never by materialising a dense delta-W. From that Gram comes a correlation matrix, in two versions — plain cosine, and double-centred (ipsatised) to remove the grand mean. The two agree on off-diagonals at r =  0.9556.

Principal axis factoring with oblimin rotation was run on that matrix at four settings (centred and uncentred, k = 5 and k = 9). The solution used throughout is centred, k = 5, oblimin. Its factor intercorrelations are small (largest |Φ| 0.282), it converged in 9 iterations with 0 Heywood cases, and the file's own verification section reports all seven numerical checks passing.

The controls matter more than the result. Two null arms were trained at matched objective: adapters on shuffled preference labels and on permuted trait-to-corpus assignments. 11 principal components of the real cloud sit above the structureless (shuffled) null; 0 sits above the permuted null. The permuted arm is the hard one: it keeps the training dynamics and destroys only the trait labels. The project reports both and does not claim the elbow is usable.

Reproducibility across initialisations was checked directly: the same trait trained twice at different LoRA seeds agrees at cosine +0.018, which is not absence of signal but the expected overlap of two random rank-64 subspaces, and 40 of 40 traits still identify themselves. The arrangement reproduces even though the coordinates do not — cross-seed cosine tracks within-run cosine at Pearson 0.9947.

On the wiki

The full record