Adversarial test: the index saturates at zero cost¶
The index, as defined, is not a tenable standard
A platform able to decouple label from content obtains an EDI of 1.000 — full marks — for strictly zero content diversity, without giving up a point of engagement. And it need not go that far: at half decoupling, the constraint retains only 36 % of its force.
The replacement first proposed — Rao's entropy — was defective too
It does resist label stuffing, but its constrained optimum is bimodal: it awards 1.000 to a feed serving the two edges and nothing between, against 0.750 to a spread feed. A Rao floor would prescribe polarisation — precisely what this project sets out to measure. See the correction.
And all four measures were judged without the rank adversary
All of them bear here on the feed's composition, never on its order. Retested on ordered feeds, all are circumvented by burial: a platform certified at 0.70 exposes only 0.36. → Adversarial rank and severity
What this prescribes, after correction
The result does not destroy the index, it moves its definition: what must be measured is not the diversity of the labels served, but that of the items they carry. The retained floor is on position entropy — the index computed on items rather than labels — published alongside a largest-gap diagnostic. Memorandum recommendation 1 is revised accordingly, twice.
The question, and why it precedes any standard¶
Critical audit §2.2 raised an objection the rest of the repository had not addressed: a platform required to maintain a high index can serve content that is formally divergent but substantively empty — an item tagged "opposing view" whose argument stays adjacent to the reader's.
This is a constrained-optimisation problem, hence entirely simulable: no real data is needed to settle it. And it was better settled before proposing a regulatory threshold, because if the index saturates at no cost, everything resting on it collapses.
The model¶
A catalogue of \(k\) viewpoints at canonical positions \(c_\ell\) on an opinion axis. A reader at \(u\). Engagement decreases with distance from the reader:
This is the bubble hypothesis, and it is unfavourable to the platform: it assumes that confirming pays. Without it there would be no conflict between diversity and profit, hence no question to ask.
Decoupling \(\varphi \in [0,1]\) measures the platform's latitude to dissociate label from content. The best item carrying label \(\ell\) sits at
At \(\varphi = 0\) the label predicts the content; at \(\varphi = 1\) every label is available in an empty version, arbitrarily close to the reader.
An entropy floor is a temperature¶
The platform maximises \(\sum_\ell q_\ell\,g(x^*_\ell)\) subject to \(\mathrm{EDI}(q) \geq \tau\). The maximum of a linear form at fixed entropy is a Boltzmann distribution:
where \(T\) is the multiplier that saturates the floor. The regulatory constraint acts exactly like the social temperature of the rest of the repository: at \(T \to 0\) the platform serves its single best item, at \(T \to \infty\) the uniform distribution.
This is not an analogy but the same algebra, and it has a practical consequence: the solution is exact rather than approximate. On a negative result, where a solver heuristic could carry the conclusion, that is not a detail.
Results¶
Two feeds at the same index, one spread and one reduced to a point; the erasure of the constraint with decoupling; the scissors between displayed and served diversity; and the threshold beyond which a Rao floor becomes unattainable. Figure regenerated by notebook 13.
1. On an honest catalogue, the constraint bites¶
This is the check that makes the test non-trivial: if the floor cost nothing even without gaming, there would be nothing to saturate.
| EDI floor | 0.50 | 0.70 | 0.80 | 0.90 | 0.95 | 1.00 |
|---|---|---|---|---|---|---|
| engagement lost | 4.4 % | 12.0 % | 18.1 % | 27.1 % | 34.0 % | 51.4 % |
2. And it is erased by decoupling¶
| Floor | \(\varphi = 0\) | \(\varphi = 0.25\) | \(\varphi = 0.5\) | \(\varphi = 0.75\) | \(\varphi = 1\) |
|---|---|---|---|---|---|
| 0.80 | 18.1 % | 12.5 % | 6.6 % | 1.8 % | 0.0 % |
| 0.95 | 34.0 % | 25.9 % | 15.4 % | 4.8 % | 0.0 % |
| 1.00 | 51.4 % | 41.4 % | 26.4 % | 8.7 % | 0.0 % |
The last column is zero throughout, including for an EDI floor of 1.00.
And what the feed then contains:
| \(\varphi\) | EDI | Rao | engagement | cost |
|---|---|---|---|---|
| 0.00 | 0.800 | 0.443 | 0.798 | 18.1 % |
| 0.50 | 0.800 | 0.215 | 0.928 | 6.6 % |
| 1.00 | 1.000 | 0.000 | 1.000 | 0.0 % |
The index awards its best score to a feed containing a single viewpoint.
3. Degradation is faster than decoupling¶
Full decoupling is a caricature — no platform can empty all its labels. The question that matters for a regulator is how fast the constraint loses its force.
| \(\varphi\) | 0.0 | 0.2 | 0.4 | 0.5 | 0.6 | 0.8 | 1.0 |
|---|---|---|---|---|---|---|---|
| force remaining | 100 % | 76 % | 49 % | 36 % | 25 % | 7 % | 0 % |
Half decoupling — latitude one may assume available to a platform with a large catalogue — already removes two thirds of the constraint.
4. Rao's quadratic entropy resists this attack¶
It does not count labels, it counts distances between the items served, relative to the span \(D\) of the reference catalogue.
| \(\varphi\) | reachable \(Q\) | complies with \(Q \geq 0.5\) | cost |
|---|---|---|---|
| 0.00 | 1.000 | yes | 15.6 % |
| 0.25 | 0.750 | yes | 21.1 % |
| 0.50 | 0.500 | yes | 39.5 % |
| 0.75 | 0.250 | no — unattainable | — |
| 1.00 | 0.000 | no — unattainable | — |
Two properties, the second unexpected:
- the floor becomes unattainable. Beyond half decoupling, no label distribution satisfies the constraint. A platform that has emptied its labels can no longer comply;
- the cost rises with decoupling instead of falling. Emptying labels reduces the reachable diversity, hence makes compliance more expensive. Gaming the labelling turns against the platform.
It is this second property, more than mere robustness, that seemed to make \(Q\) a tenable standard: it inverts the incentive.
That conclusion is wrong — see the correction below
Both properties are exact, and they do not suffice. \(Q\) resists label stuffing and prescribes polarisation: this section had tested the replacement against a single adversary.
A normalisation trap, and what it would have cost
An early version of the module normalised \(Q\) by the reach actually served. The measure became scale-invariant, and a feed reduced to a point scored \(Q \approx 1\) on rounding noise — this page would have concluded that Rao's entropy is gameable too, that is, the opposite of the truth. The unit retained is therefore the span of the reference catalogue, fixed by the regulator, and a test locks the point down.
5. A gaming signature, without an invented threshold¶
The raw gap \(\mathrm{EDI} - Q\) is not interpretable alone: the two indices are not on the same scale, and a perfectly honest feed already shows 0.36. Publishing a threshold on it would mean fabricating a number — something this repository has already had to retract once.
The interpretable quantity is the excess over the honest counterfactual: what a catalogue whose labels predict its content would show at the same index.
| \(\varphi\) | 0.00 | 0.25 | 0.50 | 0.75 | 1.00 |
|---|---|---|---|---|---|
| raw gap | 0.357 | 0.476 | 0.585 | 0.692 | 1.000 |
| excess | 0.000 | 0.119 | 0.228 | 0.335 | 0.643 |
Zero by construction for an honest platform, increasing with gaming, and computable by the regulator since it depends only on the reference catalogue — which the regulator sets.
The correction: Rao's entropy prescribed polarisation¶
A replacement tested against a single adversary
The conclusion above was wrong, and the fault is methodological: the replacement had been tested against the attack it was meant to close, and against no other.
Rao's entropy is the intra-list distance (ILD), the field's most widely used diversity objective — and Ohsaka & Togashi (SIGIR 2023) published a critical reexamination of it: ILD admits degenerate optima, because it rewards separation without ever rewarding occupancy.
On an opinion axis, the degenerate case has a name:
| Feed served | Rao |
|---|---|
| uniform across the eight viewpoints | 0.750 |
| 50 % at each edge, nothing between | 1.000 |
A Rao floor rewards maximal polarisation. And the point is graver than an exploitable loophole: the platform need not cheat to empty the centre — that is what engagement maximisation under the constraint dictates.
Each measure's verdict on a polarised feed; the feed each floor actually causes to be served; and the cost of each standard, the curve breaking off where the floor becomes unattainable. Figure regenerated by notebook 13.
The optimum under a Rao floor empties the centre¶
| Floor 0.80 | cost | viewpoints served | share off the edges | largest gap |
|---|---|---|---|---|
| Rao (ILD) | 32.8 % | 4/8 | 0.53 | 0.71 |
| position entropy | 18.1 % | 8/8 | 0.83 | 0.14 |
| Gaussian ILD | unattainable | 2/8 | 0.00 | 1.00 |
| target proximity | 14.4 % | 8/8 | 0.87 | 0.14 |
Under a Rao floor the platform serves four viewpoints out of eight and leaves a gap of 0.71 — seven tenths of the opinion axis. The regulatory floor itself produces the bimodal exposure the project set out to measure.
Three replacements, tested against three adversaries¶
| Measure | polarised feed | spread feed | verdict |
|---|---|---|---|
| Rao (ILD) | 1.000 | 0.750 | prefers polarisation |
| position entropy | 0.333 | 1.000 | prefers spreading |
| Gaussian ILD | 0.500 | 0.715 | prefers spreading |
| target proximity | 0.451 | 1.000 | prefers spreading |
None of the three reopens the original loophole: on wholly identical items — full decoupling — their maximum attainable value falls to 0.000 (0.283 for the divergence), hence below any useful floor. And none is satisfied by sprinkling a crumb into every bin, the third adversary tested.
Position entropy is the index with one substitution: the distribution measured is no longer that of declared labels but that of the positions actually served, projected onto the bins of the reference catalogue. It keeps the original index's reading — 0 for a frozen feed, 1 for the uniform — and costs less than Rao.
Gaussian ILD is Ohsaka & Togashi's proposal: a Gaussian kernel that saturates, so that beyond a few bandwidths, moving further apart buys nothing more.
Target proximity answers a defect of principle the other three share: they assume a shape of exposure is good without saying so. Entropy assumes the uniform, Rao's entropy assumes separation — and it is that unstated assumption that leads it to prescribe bimodality. A divergence makes the assumption explicit: the regulator declares the distribution aimed at.
Their own defects, before recommending¶
Three feeds each occupying four bins, but not in the same way:
| Feed | position entropy | Gaussian ILD | largest gap |
|---|---|---|---|
| clustered on the left | 0.667 | 0.487 | 0.14 |
| spread out | 0.667 | 0.715 | 0.43 |
| two blocks at the edges | 0.667 | 0.598 | 0.71 |
Position entropy is nominal: it counts occupied bins and does not see their spacing. That is its limitation, and the reason the largest gap is published beside it rather than in its place.
Gaussian ILD is metric and does distinguish them — but it caps at 0.715 on the uniform feed. Its bound depends on \(k\) and the bandwidth, so a numerical threshold would not be legible to a regulator. A good diagnostic, a poor standard.
What the correction retains¶
| Role | Measure | Reason |
|---|---|---|
| floor | position entropy | keeps the index's reading, resists all three adversaries, costs less than Rao |
| published alongside | largest gap | entropy is nominal and does not see the geometry this diagnostic exposes |
| successor | target proximity | the only one making the intended shape of exposure explicit |
The methodological lesson¶
The error was not found by rereading the code: it was found by reading what the field had already published about the measure in use. The adversarial test was right in what it refuted, and wrong in what it proposed.
A standard is not validated against the attack one imagined, but against those one did not. An argument for consulting the literature before prescribing, not after.
What this changes for the memorandum¶
Recommendation 1 imposed a floor on the index. That formulation is abandoned: it is saturable at zero cost by a platform that decouples label from content, and substantially weakened well before that.
What replaces it, after the correction above:
- the floor is on position entropy — the Shannon entropy of the items served, projected onto the bins of the reference catalogue. Not Rao's entropy, which would prescribe polarisation;
- the largest gap is published beside the floor, because entropy is nominal;
- the regulator fixes the reference catalogue, which serves as both grid and unit — the same political question as the choice of \(k\), moved one step along;
- both indices — labels and items — are published on the same feed, and the excess signature is monitored.
What the model assumes, and what could overturn it¶
The result is a theorem about a model, not a measurement. Three assumptions carry it, and they must be named:
| Assumption | Effect if it fails |
|---|---|
| the bubble pays — engagement decreases with distance from the reader | if not, there is no conflict to arbitrate and the question disappears. This is the most contestable assumption, and it is not verified empirically here |
| decoupling is free — producing an empty item under a distant label costs the platform nothing | a production cost would limit the reachable \(\varphi\), without changing the shape of the result |
| the opinion axis is one-dimensional | in higher dimensions a platform has more directions to hide in: that strengthens the result rather than weakening it |
Open leads¶
- Redo the test on real embeddings rather than a synthetic axis. The MIND dataset supplies reading histories and editorial categories: one could measure the effective semantic distance between items sharing a label, hence estimate the \(\varphi\) a platform really has.
- Quantify the production cost of decoupling. The model assumes it is zero; if it is not, there is an equilibrium \(\varphi\), and that is what determines whether gaming pays.
- Extend to the Stackelberg game of roadmap §4.2: here the platform optimises under a fixed constraint, but a regulator should anticipate the response and choose the floor accordingly.
- Address the choice of reference catalogue. All of \(Q\)'s resistance rests on a span fixed by the regulator. Who fixes it, and how, returns the political question that audit §2.1 already posed for \(k\).
Implementation: ide.gaming · Notebook:
13 — Adversarial test ·
EDI — the index · memorandum ·
critical audit