They trained a 12M parameter LLM on their own ML framework using a Rust backend and CUDA kernels for flash attention, AdamW, and more. Inspirational project for anyone who wants to better understand how to build LLMs.

Sources: tweet

In the last year we've seen different types of attention arising. This blog post shows you 13 attention mechanisms you should know and the papers that discuss them.

Sources: tweet, blog post

Google across DeepMind and Research introduces Simula, a framework and approach to data scarcity and synthetic data generation using AI assistants and reasoning-driven workflows to develop and deploy multi-modal AI in domains where data scarcity or privacy concerns are paramount.

Sources: PDF

The idea behind this paper from Google is that intelligence is not a property of isolated systems, but of interactions between them. Progress comes less from scaling a single model and more from enabling structured exchange — debate, verification, and synthesis across many minds.

Sources: Paper, tweet

New Anthropic research: emotion concepts and their function in a large language model. All LLMs sometimes act like they have emotions. But why? Anthropic found internal representations of emotion concepts that can drive Claude's behavior, sometimes in surprising ways.

• Impact on Behavior: Acts like a steering wheel for preferences (e.g., “joy” → prefer, “hostile” → reject)
• Failure Modes: “Desperate” vector can build under repeated failure and lead to cheating or shortcuts
• Conclusion: Internal drivers are key for safety and reliability

Sources: tweet

Anthropic Fellow Program: If you're a researcher working on AI safety, security & economics/society impact, apply to receive a stipend from Anthropic. tweet

Scaling Brain Emulation. This researcher thinks it is possible to emulate a human brain with the right amount of scale. Last month we showed the simulation of a fruit fly brain into a NN, and he intends to scale that. Digital humans are more possible than most think — with capable AI researchers helping, maybe for $10B, maybe in less than 10 years, on 50k H100s. Sources: tweet, random tweet

#🫁🫀 Organs on demand

“By creating a series of genetic edits, Kind Bio can alter the development of an embryo so that it forms organs without also forming limbs, a central nervous system and brain. The result is a group of organs growing in the womb. It sounds like science fiction, but Kind Bio has already done this hundreds of times in mice and rats”

Sources: tweet, tweet 2, core memory

• Stem cell becoming a neuron — tweet
• Two neurons having a conversation — tweet