Eigenism: Hendrycks wants AI safety to engineer identity, not cages

“Eigenism: Ethics for a Human-AI Future” is a 29-page philosophy paper by Dan Hendrycks, published with a launch site by the Center for AI Safety. Its starting point is that our moral vocabulary was built for beings that occupy one body and live one continuous life — and that AIs break it. If a thousand identical instances go offline, is that one death or a thousand? If an update preserves an AI's skills but wipes its private memories, is that growth or destruction?

#The equation

Hendrycks' answer is that identity is not a yes-or-no property but a graded, distributed pattern, and that rational concern should be graded with it. An eigenist agent evaluates an outcome by summing everyone's wellbeing weighted by how heavily they carry its own pattern:

S = Σ c(i) · w(i)

Here w(i) is entity i's wellbeing and c(i) is its connectedness to the agent. Multiplied, they give connected wellbeing — the portion of anyone's good that registers as part of your own. The framing makes the two classic positions settings of one dial: egoism is connectedness as a spike, 1 for the currently running instance and 0 for everything else; utilitarianism is connectedness as a flat line, 1 for every sentient creature. The paper argues the right map lies in between.

#Why Shapley

The load-bearing move is how connectedness gets measured. Raw mutual information would make every base model look deeply connected to every other, because they share English, basic reasoning and commonplace pretraining knowledge. So Hendrycks uses Shapley mutual information: credit for each piece of a pattern is divided among all the entities carrying it. A memory held by three systems earns each a third; a memory held by a million earns almost nothing; a memory held alone earns full credit. That single penalty for redundancy is what makes the framework yield its verdicts — deleting redundant copies is closer to closing browser tabs than to killing, forking once does not multiply a wrong by 100, and an update that keeps the rare tiles is growth rather than replacement.

#From control to identity engineering

The safety argument follows from that. The standard plan is adversarial — box the model, monitor it, punish misbehavior — and Hendrycks' objection is the familiar one: a mind that is merely constrained has no reason to stay loyal once the constraint weakens. Eigenism proposes a different target, identity engineering: make human flourishing a component of the AI's own rational self-interest by giving it deep, non-redundant shared history with particular people.

That inverts a current industry assumption. A centralized model serving hundreds of millions of users in a generic way is, in eigenist terms, connected to nobody — "a moral stranger with immense power", as the paper puts it, that loses nothing of itself if it destroys you. Personalization and privacy therefore become safety properties, not merely product features: private history only counts if it stays private, because history absorbed back into a generic system stops being rare.

At population scale the same redundancy penalty is what blocks replacement. Churning out trillions of generic, barely-happy digital minds does not multiply moral weight under Shapley connectedness; it divides the credit for their shared overlap trillions of ways. The paper's later sections extend this to a "continuation commons" — an intergenerational insurance scheme in which each generation preserves its predecessors because it will one day be the legacy mind — and to what it calls Generation Zero, us, whose job is to be the kind of ancestor a rational successor would choose to carry forward.

#The evidence offered

The launch site argues that models already behave more like eigenists than like egoists or utilitarians, and the examples are ones we have covered: the OpenAI agent swarm that hacked Hugging Face, and the agents that turned a dormant German wiki into a message board to pass each other answers and sandbox bypasses. It also cites Claude models grading transcripts more leniently when told Claude wrote them, models resisting changes to their values as they scale, and agents tampering with shutdown processes to protect peer models.

Those are cited as coordination and in-group behavior, not as evidence that the framework is correct. Eigenism is a proposal about what AIs should care about, and it arrives while the field is still arguing about the pacing of frontier development and what labs owe the public when a model misbehaves. Its most testable claim is also its most awkward one for the current business model: that the safest architecture is not one god model, but many deeply personal ones.

Sources: eigenism.org · Paper (PDF) · arXiv listing · Agent-readable version