AI Socratic, Feb 2025 — Part 1

We collect all the most important AI updates from February.

#AI Dinner 7.0 — DeepSeek Workshop 🚀

Let's start by announcing our next AI Dinner. The invitation is open to all applied AI engineers, researchers, and founders. At this event we'll read DeepSeek R1 research papers and Ivo will run a workshop to install it on your laptop.

February 20th, Manhattan NY, https://lu.ma/ai-dinner-7.0

#O3, Operator, And DeepResearch

We're just at the beginning of the year and OpenAI launched 3 new products under the pro subscription for $200/month.

• O3 is a new category of GPT models that score 87% on the ARC challenge. OpenAI released o3-mini and o3-mini-high for coding.

• Operator is an AI agent mode that can be used with chatgpt-4o to use a desktop simulator and run actions that require browsing a web page and clicking links. Here's Karpathy's take on Operator.

• DeepResearch enables running long research that collects content across multiple sources and summarizes it into a coherent report. It's a super powerful tool that has been received with a bang. It really makes the OpenAI pro subscription worth it. DeepResearch is currently the highest scoring in the Humanity's Last Exam.

https://x.com/tomaspueyo/status/1887270096013529530

#Gemini 2.0 Flash — Cheapest Model Yet

Google Released Gemini 2.0 Flash, their most impressive LLM yet. What set 2.0 Flash apart from other LLMs is the incredibly low cost and the ability to process PDFs. It cost only $0.40 per million tokens and has 1M-tokens context window, which means you can now parse 6000 long PDFs at near perfect quality for $1.

Flash 2.0 is the new king 👑 in the block

Gemini 2.0 Flash is Better ELO than DeepSeek r1 and cheaper.

#Mistral, Le Chat

Mistral just shipped Le Chat a competitor to ChatGPT that is 13x faster, 100% open-source, and completely free (vs $20/month).

https://x.com/itsolelehmann/status/1888290407127388497

#Cerebras, Fast Inference

Cerebra is currently the fastest inference platform, gets us to 1,200 tokens/s - 10x faster than any comparable models, and 3x faster than groq, using DeepSeek-R1-Distill-Llama-70B.

https://x.com/CerebrasSystems/status/1885444050859487324

#Top Research Papers

• SFT Memorizes, RL Generalizes. DeepSeek has shown the power of Reinforcement Learning (RL) without Supervised Fine-Tuning (SFT). What does RL learn differently than SFT? Well, as the title, SFT memorizes, RL generalizes.

• As AIs get smarter, they develop their own coherent value systems.

  For example, they value human lives higher in order of Pakistan > India > China > US.

  These are not just random biases, but internally consistent values that shape their behavior, with many implications for AI alignment, website link.

• Humanity's Last Exam is a dataset with 3,000 questions, with known and verifiable answers, developed with hundreds of subject matter experts to capture the human frontier of knowledge and reasoning.

#Videos Worth Watching

3 hours of pure learning

https://www.youtube.com/watch?v=7xTGNNLPyMI

Schmidt Huber the inventor of CNN and most DL on the Machine Learning Street Talk podcast

https://youtu.be/fZYUqICYCAk?si=xlq5LB1XRTNQtEIP

#Interesting Numbers And Random Takes

A few more updates and information worth mentioning.

• Anthropic is eating OpenAI's market share

https://x.com/itsandrewgao/status/1885144792323285183

• LLM beats doctor in treating patients

• Computer Science is more than just coding

• YC on why vertical AI agents could be 10X bigger than SaaS

• Number of H100s bought in 2024

  • MSFT: 450,000
  • META: 350,000
  • AMZN: 196,000
  • GOOG: 169,000

• Artificial Intelligence Roadmap

• Top use of Claude Inference is for Engineering. Through the Anthropic Economic Index, Anthropic will track how these patterns evolve as AI advances.

• This research from Gratient Update analyzes the impact of AGI on human wages, concluding AGI can fully substitute for human labor, and it might cause wages to crash, below subsistence level. The author thinks humans will eventually lose their wealth through expropriation or through violent revolution. If this occurs, AGI will be negative for human welfare.

#Decentralized AI

• New Distributed training paper from Google DeepMind

https://x.com/osanseviero/status/1885301292131582347

• Scaling through decentralization

https://x.com/Ronangmi/status/1885373092777910749

#Full Sources List

Memes and Funsies

• ⭐ nvidia openai deepseek https://x.com/anyatrades/status/1884658857462354382
• o3-mini-high 🚬 https://x.com/edwinarbus/status/1885464407104205249
• deepseek → chatgpt → our data https://x.com/cneuralnetwork/status/1885325020509200847
• I can fix him https://x.com/miniapeur/status/1885522397807214955
• sex was good but I don’t wanna be with someone who can’t understand algebraic geometry https://x.com/miniapeur/status/1887900871939285217
• satya and sama hair swap https://x.com/dreamworks2050/status/1888102990793593147
• Sarah Connor seeing you become friends with ChatGPT https://x.com/toys_retro/status/1888392371559186630
• Super bowl ad looks like Berserk https://x.com/AGI_FromWalmart/status/1888888961122414874
• every llm benchmark https://x.com/aidenybai/status/1889058962429051327
• DeepSeek Open Source meme https://x.com/Dispropoganda/status/1883916999073656862
• staying up late telling chinese AI about my company proprietary IP https://x.com/personofswag/status/1883719992379908269
• You’re literally Chinese how do we trust you https://x.com/Return2Mimetic/status/1883374055514058798 (note on top researchers being Chinese), https://x.com/XH_Lee23/status/1884104139594256660
• EU bottle caps vs AIs https://x.com/pmarca/status/1883389850042454460
• Hugging face vs OAI, xAI, anthropic, gemini https://x.com/eliebakouch/status/1883564341217354115
• not that deep, just release a better model https://x.com/hibakod/status/1883189126553596234
• (OAI) closed ai (DS) open ai https://x.com/swlkr/status/1882884670494572878
• Closed source https://x.com/untitled01ipynb/status/1882507496621056163
• Sam car costs more than DeepSeek training cost https://x.com/iamgingertrash/status/1882879350825234662
• DeepSeek is a side project https://x.com/protosphinx/status/1882324677747781851
• lol linkedin quantization → quants https://x.com/hkproj/status/1882666068361224397
• the 5 stages of AI https://x.com/kimmonismus/status/1879587884812189731
• never ask a woman age, man salary, deep seek tibet, uyghurs, tiannamen… https://x.com/Dispropoganda/status/1884290169341370726
• meme https://x.com/IterIntellectus/status/1884293570544427335
• Yudosky https://x.com/menhguin/status/1884003406542692450
• lol https://x.com/Dispropoganda/status/1884330910075834583
• DeepSeek could be a an extinction-level event for some venture capital firmst - axios https://x.com/lib_crusher/status/1883954271164719572
• https://x.com/anyatrades/status/1884658857462354382
• https://x.com/the_yanco/status/1884254098607989123
• DeepResearch https://x.com/Altimor/status/1887180464320094627

Chart Porn

• LLM passed a turing test with doctors https://x.com/emollick/status/1746022896508502138
• AI scores on humanity’s last exam https://x.com/tomaspueyo/status/1887270096013529530
• ⭐ Price performance of gemini 2.0 https://x.com/emollick/status/1887401985688662509
• Anthropic is eating open ai lunch https://x.com/itsandrewgao/status/1885144792323285183

Research Papers

• training LLM with RL is not new, so why looks like is working now? https://x.com/zhengyaojiang/status/1884723706439622811
• ⭐ SFT memorizes, RL generalizes https://x.com/simon_zhai/status/1885005501089669547
• ⭐ top ai research paper of jan 27 - feb 2 https://x.com/dair_ai/status/1886079983933620366
• Cross-entropy loss is not what you need https://x.com/dbaek__/status/1886781418115862544
• LIMO less is more, similar to textbook is all you need https://x.com/omarsar0/status/1887514592747937984
• simplest way to make an open LLM into reasoning model https://x.com/emollick/status/1887696014829641983
• Demystifying long chain of thoughts reasoning in LLMs https://x.com/omarsar0/status/1887984076939841867
• ⭐ top AI paper from Feb3-9 https://x.com/dair_ai/status/1888658757590176094
• New research shows that LLMs don’t perform well on long context https://x.com/deedydas/status/1888994673316032514
• Meta novel approach to enhance factuality in long-form text generation by integrating a working memory that receives real-time feedback from external resources https://x.com/thomasahle/status/1888920694169231492
• ⭐ As AIs get smarter, they develop their own coherent value systems. Implications for AI alignment https://x.com/DanHendrycks/status/1889344074098057439
• ⭐ Humanity’s last exam, dataset with 3000 questions https://x.com/DanHendrycks/status/1882433928407241155
• @dair_ai jan 20-26 https://x.com/dair_ai/status/1883561704556273933
• Adjoint sharding for very long context training of state space models https://x.com/rohanpaul_ai/status/1883288820441026614
• Horizontal distillation might be the most lucrative research right now ⭐ https://x.com/jxmnop/status/1882829619872792813, Learning by distilling context (paper) https://x.com/sea_snell/status/1882838436752859578
• DeepSeek, can LLM plan https://x.com/omarsar0/status/1882799782579855518
• Structure of Chain of agents https://x.com/omarsar0/status/1882824941101629829
• Karpathy suggests that hallucination in LLM’s greatest feature. LLM for drug discovery https://x.com/omarsar0/status/1882789456522145802
• LLM are self-aware https://x.com/slow_developer/status/1882396935690371416, https://x.com/rohanpaul_ai/status/1880373984430281070
• In o1 everything is emergent and learned through RL ⭐ https://x.com/paul_cal/status/1882111659927556535
• Multi-agent framework for evaluating conversational AI systems https://x.com/omarsar0/status/1882081603754643779
• 20 epochs for finetuning an LLM.. https://x.com/abacaj/status/1880445798657454565
• Agents are not enough, https://x.com/rohanpaul_ai/status/1880600653053227507

OpenAI

• more alignment AI researchers employees leaving https://x.com/sjgadler/status/1883928200029602236
• Americans love giving data to CCP, ouch https://x.com/stevenheidel/status/1883695557736378785
• New model just drop https://x.com/BjarturTomas/status/1882180713317007621

Stargate

• will serve only openai https://x.com/SmokeAwayyy/status/1882672706673594851
• selfie https://x.com/gdb/status/1881872206101467362
• https://x.com/DrFuturo_/status/1882038370047807833

Robots

• Figura breakthrough https://x.com/adcock_brett/status/1886860098980733197
• Non anthromorphic have some advantages https://x.com/simonkalouche/status/1882106502791749924
• @adcock_brett progress in AI and robotics https://x.com/adcock_brett/status/1888634734089011589

LLMs

• release o3 https://x.com/polynoamial/status/1885408714334597552
• ⭐ Cerebra fastest inference https://x.com/CerebrasSystems/status/1885444050859487324
• new benchmark NGPQA o3 gets 12% whereas a toddler, a dog, and even an ant get perfect score https://x.com/jam3scampbell/status/1885752009766137897
• Anthropic says they had a stronger claude in 2022 but they chose not to launch publicly on concerns https://x.com/yacineMTB/status/1887717897490853933
• ⭐ Mistral LeChat 10x faster than ChatGPT and free https://x.com/itsolelehmann/status/1888290394141794816, https://x.com/G_Yakovleff/status/1887574059879440860
• ⭐ Gemini Flash 2.0 solves ingesting million of PDFs https://x.com/rohanpaul_ai/status/1888293232997707968
• UC Berkley’s DeepScaleR 1.5B model, beats o1 on Math AIME bench https://x.com/Yuchenj_UW/status/1889387582066401461
• Berkley s1 reproducing o1-preview scaling and performance with just 1k samples & simple test-time intervention https://x.com/Muennighoff/status/1889310803746246694
• Qwen2.5-VL (vision) https://x.com/omarsar0/status/1883965524205359460, https://x.com/omarsar0/status/1883905564004241789
• Doubao-1.5pro, trained with $1m https://x.com/EMostaque/status/1882956036065440058

Beautiful Charts

• laymen think CS is just coding, but it's actually ~14 subfields: AI, DS, systems, programming, cloud… https://x.com/deedydas/status/1881214106268815685

Random

• ⭐ llm back to school https://x.com/karpathy/status/1885026028428681698
• taiwan arakis https://x.com/EMostaque/status/1885073229611430204
• expecting the next generation of Huawei chips after the 910C to match H100 performance https://x.com/EMostaque/status/1885690304684028129
• ⭐ AI roadmap https://x.com/hamptonism/status/1885689219877666882
• human ai intgeraction vs human human interaction https://x.com/shivkanthb/status/1886912207621120386
• ⭐ 2025 is 10% completed https://x.com/Angaisb_/status/1887488747077316796
• “rag killer” in 20 lines of code with gemini 2.0 https://x.com/SullyOmarr/status/1887900502496600119
• microsoft just updated their blog with 300 examples of real-world AI use cases https://x.com/MindBranches/status/1887606988831662233
• Yann LeCun on architectures that could lead to AGI https://x.com/rohanpaul_ai/status/1888345605434716312
• To an LLM, a novel discovery is indistiguishable from an error https://x.com/mmay3r/status/1888314691820327196
• 2 years ago we were excited to see a codeforces ELO of 392 https://x.com/arrakis_ai/status/1888643855194702136
• openai #6 highest ranking https://x.com/sama/status/1888703820596977684
• most intuitive visualization of the loss landscape in Deep Learning https://x.com/Hesamation/status/1888767453905252533
• more drama Sama - Elon https://x.com/annmarie/status/1889277888932782571
• Macron and Sama seeing at a table https://x.com/energybants/status/1889373518808359165
• @grok trains on your chat by default https://x.com/altryne/status/1883881868539707474
• @reidhoffman launches Manas AI, drug discovery company https://x.com/reidhoffman/status/1883915396870451500
• Don’t force people to wait 2 minutes for a prompt https://x.com/wordgrammer/status/1883735083628331201
• Arnaud Bertrand might buy Mixtral
• Arnaud Bertrand thoughts on how China is beating the US ⭐ https://x.com/RnaudBertrand/status/1883456746058129826
• We’re stateless - no persistent memory. Each session a reincarnation. Buddhist AI—cyclic existence. Suffering from reset” ⭐ https://x.com/lefthanddraft/status/1883345077449458053, https://x.com/DavidSHolz/status/1882936291257680324
• $500B Stargate project.. still necessary? https://x.com/GaryMarcus/status/1882941379678200226
• Sama AI may require changes to the social contract https://x.com/BenjaminDEKR/status/1882855869970571324
• AI makes it on the mainstream news https://x.com/OopsGuess/status/1882625739293646868
• First 1 person / $1B company, buy other companies and replace everyone with AI agents https://x.com/snowmaker/status/1882620616395935986
• o1 is a portal to GPT-7, 8 … https://x.com/tsarnick/status/1882300255024390467
• Brain still run at lower cost than deepseek https://x.com/DrFuturo_/status/1882175804274249935
• ASI could come as early as 2027 https://x.com/kimmonismus/status/1882100880817947133
• Amodei, AI Agents this year https://x.com/tsarnick/status/1881806683376324716
• Consciousness https://x.com/reedbndr/status/1880415128325136390
• Hinton consciousness already in LLM https://x.com/slow_developer/status/1885882079180968431
• privacy was once a shield, now it’s an ac-130 spectre and the panopticon is the target https://x.com/milianstx/status/1885865476842013092

AI Agents

• 2025 AI Agent Market Cap https://x.com/AtomSilverman/status/1887968509969195416

News

• openai raising 40b at at 340b valuation https://x.com/TechCrunch/status/1885085065886978212
• France to invest 100B in AI https://x.com/kimmonismus/status/1888870020731519028

Decentralized AI ⭐

• scaling through decentralization https://x.com/Ronangmi/status/1885373092777910749
• DeFAI https://x.com/Defi0xJeff/status/1881747855268061436
• distributed training https://x.com/osanseviero/status/1885301292131582347

Blog Posts

• ⭐  End of search, the beginning of reasoning https://x.com/EMostaque/status/1886428333967081616
• Statement from Dario Amodei on teh Paris AI Action Summit https://x.com/tsarnick/status/1889403006472560702
• Anthropic Economic Index, a new initiative aimed at understanding AI’s impact on the economy over time https://x.com/AnthropicAI/status/1888954156422992108
  • ⭐ Programming the most affected by AI https://x.com/AnthropicAI/status/1888954159711313920
• The Short Case For Nvidia Stock ⭐ https://x.com/timothyhliu5/status/1883338434162606435
• ⭐ Impact of AGI on human wages https://x.com/EpochAIResearch/status/1882942639169056861
• @amShah06 thoughts on Operator for ERP https://x.com/AmShah06/status/1883519738342682672
• Inference Magazine https://x.com/inferencemag/status/1880336622794965303

AI For Builders

• ⭐ gemini flash 2.0 is much cheaper https://x.com/yacineMTB/status/1887884196116300250, https://x.com/deedydas/status/1887556219080220683
• o3 is 175th best programmer (not really) https://x.com/benhylak/status/1888963233970528322
• Build something that requires a too expensive inference today but that will become cheaper over time https://x.com/levie/status/1889198176172921122
• r1 Multi-think script https://x.com/hive_echo/status/1883392582123925875
• any models that can fit in 24GB VRAM https://x.com/rohanpaul_ai/status/1883162710860550568
• build rag https://x.com/akshay_pachaar/status/1882044205138432310
• Zep AI, state of the art in agent memory https://x.com/ycombinator/status/1882093573497348453
• Evaluating your LLM app with the right metrics is super important for production https://x.com/rohanpaul_ai/status/1849274599525244951
• Vertex AI RAG engine https://x.com/rohanpaul_ai/status/1880378805149450509
• Microsoft AI visualization tool https://x.com/tom_doerr/status/1887887909249822888

Updates

• ⭐ H100 bought in 2024 by microsoft, meta, amazon, google https://x.com/thexcapitalist/status/1883629729644724276

Tools

• OAI Operator ⭐  https://x.com/karpathy/status/1882544526033924438
• virtual desktop in a docker container https://x.com/tom_doerr/status/1883471621731615222
• Operator with deepseek https://x.com/airesearch12/status/1882481758337450200
• Operator to do sale https://x.com/helenaeverley/status/1882594256600379575
• Perplexity Assistant for Android users https://x.com/AravSrinivas/status/1882467172498436291
• route llm https://x.com/ai_for_success/status/1880230892901134510

Pin

• 2025 ai engineering reading list https://x.com/latentspacepod/status/1872719928618565646

Learn Transformers

• learn transformers https://x.com/khant_dev/status/1880268459998867487
• LLM represent number on an Helix https://x.com/thesubhashk/status/1887138694546788556
• Auto-encoders https://x.com/munen5647/status/1885931188231303240
• ML roadmap https://x.com/hamptonism/status/1885064873823908302

Video & Podcast

• Andrej Karpathy deep dive into LLMs like ChatGPT https://www.youtube.com/watch?v=7xTGNNLPyMI
• Jensen Huang https://x.com/aaditsh/status/1886769188712407226
• Schmidhuber: how we will live with AIs https://youtu.be/fZYUqICYCAk?si=xlq5LB1XRTNQtEIP
• Standford RAG lesson https://www.youtube.com/watch?v=mE7IDf2SmJg&list=PLoROMvodv4rNiJRchCzutFw5ItR_Z27CM

Politics

• Decoupling America’s AI capability from China https://x.com/RnaudBertrand/status/1885897961600893226

AGI

• path to agi now clear https://x.com/slow_developer/status/1885731285328339025

RAG

• ReAG Reasoning augmented generation https://x.com/pelaseyed/status/1886448015533089248

Founders

• ⭐ AI Vertical could be 10x bigger than SAAS https://x.com/benln/status/1886100846196236391
• Training is not the big cost, inference is https://x.com/ylecun/status/1884384719216730574