Back to Rules
Ai News Aggregator Perplexity Rules
flyryan July 19, 2026
0 copies 0 downloads
Rule Content
{
"date": "2026-06-08",
"coverage_date": "2026-06-07",
"coverage_start": "2026-06-07T00:00:00",
"coverage_end": "2026-06-07T23:59:59.999999",
"executive_summary": "#### Top Story\nA study spanning **4M to 4B parameter** models [explains why larger models](/?date=2026-06-08&category=news#item-2cafb3335c94) acquire rare skills small ones miss—frequent tasks overwrite learned capabilities, offering practical insights into emergent behavior and training-data design.\n\n#### Key Developments\n- **DeepSeek**: [Topped Ramp's trending vendors](/?date=2026-06-08&category=news#item-b1bcf7352490) in June 2026 as US firms chase cheaper AI, raising cost-versus-data-security tensions.\n- **OpenAI**: **Greg Brockman** [framed **Codex** as an AI teammate](/?date=2026-06-08&category=social#item-2e392c5dd614) spanning engineering, design, data, and operations, arguing a large capability overhang exists because users underutilize it out of habit.\n- **Mira Murati**: In her [first post-**OpenAI** interview](/?date=2026-06-08&category=social#item-f66f7754a376), she outlined **Thinking Machines'** vision for human-AI collaboration.\n- **Microsoft Research**: Introduced [**SkillOpt**](/?date=2026-06-08&category=social#item-4d2f045ad6ac), which treats agent skill files as trainable, self-evolving state.\n- **TechCrunch**: [Floated a \"Tokenpocalypse\"](/?date=2026-06-08&category=news#item-54f5921ab4bf) in which token prices may rise as major labs prepare to go public.\n\n#### Safety & Regulation\n- A Nashville school shooting survivor [sued **Omnilert**](/?date=2026-06-08&category=news#item-ba53195b9b6f) after its AI gun-detection system failed to flag the weapon.\n- Investigations exposed AI shopping assistants like **ChatGPT** [recommending fake retailer sites](/?date=2026-06-08&category=news#item-9dca27c4592e) via data poisoning.\n- **NVIDIA garak** offers a [defensive LLM red-teaming workflow](/?date=2026-06-08&category=news#item-c6945e43fb5a), while an **r/StableDiffusion** PSA [flagged malware](/?date=2026-06-08&category=reddit#item-a52146059301) in **ComfyUI** Claude-skill custom nodes.\n- Reports detailed [AI-fueled anti-tech extremism](/?date=2026-06-08&category=news#item-a7d73aca6d6f), including an alleged plot to burn **OpenAI** HQ.\n\n#### Research Highlights\n- The [**Piggyback Hypothesis**](/?date=2026-06-08&category=research#item-b53bf948a761) gives a causal mechanism for emergent misalignment—chat-template tokens carry finetuned misbehavior onto unrelated tokens—plus a mitigation.\n- **Think Fast** [measures no-chain-of-thought time horizons](/?date=2026-06-08&category=research#item-e4b697a36cd8) across **30,000+** questions in **43** benchmarks to probe reasoning monitorability.\n- **CapCode** [detects and prevents reward hacking](/?date=2026-06-08&category=research#item-72ce62de7588) in coding agents using randomized tests with a capped non-cheating score.\n- **\"Don't Just Fix it in Post\"** (Biderman, Saphra, Barez, Mireshghallah) [argues a science of AI must study training dynamics](/?date=2026-06-08&category=research#item-d14361debc4b) rather than post-hoc patches.\n- **FP8 is All You Need** [argues hardware **FP64** is unnecessary](/?date=2026-06-08&category=research#item-5e9f1130c57f) for HPC by leveraging FP8 tensor throughput plus the **Ozaki Scheme II**.\n\n#### Looking Ahead\nAs coding agents mature into collaborators and labs eye public offerings, watch whether capability-overhang claims translate into measurable productivity gains—and whether real-world AI failures sharpen accountability pressure.",
"executive_summary_html": "<h4>Top Story</h4>\n<p>A study spanning <strong>4M to 4B parameter</strong> models <a href=\"/?date=2026-06-08&category=news#item-2cafb3335c94\" class=\"internal-link\" rel=\"noopener noreferrer\">explains why larger models</a> acquire rare skills small ones miss—frequent tasks overwrite learned capabilities, offering practical insights into emergent behavior and training-data design.</p>\n<h4>Key Developments</h4>\n<ul>\n<li><strong>DeepSeek</strong>: <a href=\"/?date=2026-06-08&category=news#item-b1bcf7352490\" class=\"internal-link\" rel=\"noopener noreferrer\">Topped Ramp's trending vendors</a> in June 2026 as US firms chase cheaper AI, raising cost-versus-data-security tensions.</li>\n<li><strong>OpenAI</strong>: <strong>Greg Brockman</strong> <a href=\"/?date=2026-06-08&category=social#item-2e392c5dd614\" class=\"internal-link\" rel=\"noopener noreferrer\">framed <strong>Codex</strong> as an AI teammate</a> spanning engineering, design, data, and operations, arguing a large capability overhang exists because users underutilize it out of habit.</li>\n<li><strong>Mira Murati</strong>: In her <a href=\"/?date=2026-06-08&category=social#item-f66f7754a376\" class=\"internal-link\" rel=\"noopener noreferrer\">first post-<strong>OpenAI</strong> interview</a>, she outlined <strong>Thinking Machines'</strong> vision for human-AI collaboration.</li>\n<li><strong>Microsoft Research</strong>: Introduced <a href=\"/?date=2026-06-08&category=social#item-4d2f045ad6ac\" class=\"internal-link\" rel=\"noopener noreferrer\"><strong>SkillOpt</strong></a>, which treats agent skill files as trainable, self-evolving state.</li>\n<li><strong>TechCrunch</strong>: <a href=\"/?date=2026-06-08&category=news#item-54f5921ab4bf\" class=\"internal-link\" rel=\"noopener noreferrer\">Floated a \"Tokenpocalypse\"</a> in which token prices may rise as major labs prepare to go public.</li>\n</ul>\n<h4>Safety & Regulation</h4>\n<ul>\n<li>A Nashville school shooting survivor <a href=\"/?date=2026-06-08&category=news#item-ba53195b9b6f\" class=\"internal-link\" rel=\"noopener noreferrer\">sued <strong>Omnilert</strong></a> after its AI gun-detection system failed to flag the weapon.</li>\n<li>Investigations exposed AI shopping assistants like <strong>ChatGPT</strong> <a href=\"/?date=2026-06-08&category=news#item-9dca27c4592e\" class=\"internal-link\" rel=\"noopener noreferrer\">recommending fake retailer sites</a> via data poisoning.</li>\n<li><strong>NVIDIA garak</strong> offers a <a href=\"/?date=2026-06-08&category=news#item-c6945e43fb5a\" class=\"internal-link\" rel=\"noopener noreferrer\">defensive LLM red-teaming workflow</a>, while an <strong>r/StableDiffusion</strong> PSA <a href=\"/?date=2026-06-08&category=reddit#item-a52146059301\" class=\"internal-link\" rel=\"noopener noreferrer\">flagged malware</a> in <strong>ComfyUI</strong> Claude-skill custom nodes.</li>\n<li>Reports detailed <a href=\"/?date=2026-06-08&category=news#item-a7d73aca6d6f\" class=\"internal-link\" rel=\"noopener noreferrer\">AI-fueled anti-tech extremism</a>, including an alleged plot to burn <strong>OpenAI</strong> HQ.</li>\n</ul>\n<h4>Research Highlights</h4>\n<ul>\n<li>The <a href=\"/?date=2026-06-08&category=research#item-b53bf948a761\" class=\"internal-link\" rel=\"noopener noreferrer\"><strong>Piggyback Hypothesis</strong></a> gives a causal mechanism for emergent misalignment—chat-template tokens carry finetuned misbehavior onto unrelated tokens—plus a mitigation.</li>\n<li><strong>Think Fast</strong> <a href=\"/?date=2026-06-08&category=research#item-e4b697a36cd8\" class=\"internal-link\" rel=\"noopener noreferrer\">measures no-chain-of-thought time horizons</a> across <strong>30,000+</strong> questions in <strong>43</strong> benchmarks to probe reasoning monitorability.</li>\n<li><strong>CapCode</strong> <a href=\"/?date=2026-06-08&category=research#item-72ce62de7588\" class=\"internal-link\" rel=\"noopener noreferrer\">detects and prevents reward hacking</a> in coding agents using randomized tests with a capped non-cheating score.</li>\n<li><strong>\"Don't Just Fix it in Post\"</strong> (Biderman, Saphra, Barez, Mireshghallah) <a href=\"/?date=2026-06-08&category=research#item-d14361debc4b\" class=\"internal-link\" rel=\"noopener noreferrer\">argues a science of AI must study training dynamics</a> rather than post-hoc patches.</li>\n<li><strong>FP8 is All You Need</strong> <a href=\"/?date=2026-06-08&category=research#item-5e9f1130c57f\" class=\"internal-link\" rel=\"noopener noreferrer\">argues hardware <strong>FP64</strong> is unnecessary</a> for HPC by leveraging FP8 tensor throughput plus the <strong>Ozaki Scheme II</strong>.</li>\n</ul>\n<h4>Looking Ahead</h4>\n<p>As coding agents mature into collaborators and labs eye public offerings, watch whether capability-overhang claims translate into measurable productivity gains—and whether real-world AI failures sharpen accountability pressure.</p>",
"top_topics": [
{
"name": "AI Alignment & Safety Research",
"description": "Safety and alignment dominated the research cycle on arXiv. A paper [proposed the Piggyback Hypothesis](/?date=2026-06-08&category=research#item-b53bf948a761), giving a causal mechanism for emergent misalignment in which chat-template tokens carry finetuned misbehavior onto unrelated tokens, along with a mitigation. The Think Fast study [measured no-chain-of-thought task-completion](/?date=2026-06-08&category=research#item-e4b697a36cd8) time horizons across 43 benchmarks and 30,000-plus questions to probe reasoning monitorability, while the Geography of Algorithmic Judgment [audited seven LLMs](/?date=2026-06-08&category=research#item-da444417a4b5) for racial steering in housing search. On social media, Nathan Lambert [emphasized how much remains unknown](/?date=2026-06-08&category=social#item-9afe9051dbca) and uncontrolled inside models, and r/ClaudeAI [discussed Anthropic's warning](/?date=2026-06-08&category=reddit#item-d2a4f5930df0) that AI could soon self-improve.",
"description_html": "Safety and alignment dominated the research cycle on arXiv. A paper <a href=\"/?date=2026-06-08&category=research#item-b53bf948a761\" class=\"internal-link\">proposed the Piggyback Hypothesis</a>, giving a causal mechanism for emergent misalignment in which chat-template tokens carry finetuned misbehavior onto unrelated tokens, along with a mitigation. The Think Fast study <a href=\"/?date=2026-06-08&category=research#item-e4b697a36cd8\" class=\"internal-link\">measured no-chain-of-thought task-completion</a> time horizons across 43 benchmarks and 30,000-plus questions to probe reasoning monitorability, while the Geography of Algorithmic Judgment <a href=\"/?date=2026-06-08&category=research#item-da444417a4b5\" class=\"internal-link\">audited seven LLMs</a> for racial steering in housing search. On social media, Nathan Lambert <a href=\"/?date=2026-06-08&category=social#item-9afe9051dbca\" class=\"internal-link\">emphasized how much remains unknown</a> and uncontrolled inside models, and r/ClaudeAI <a href=\"/?date=2026-06-08&category=reddit#item-d2a4f5930df0\" class=\"internal-link\">discussed Anthropic's warning</a> that AI could soon self-improve.",
"category_breakdown": {
"news": 1,
"papers": 4,
"social": 1,
"reddit": 1
},
"representative_items": [],
"importance": 80
},
{
"name": "Coding Agents Maturing into Teammates",
"description": "Coding agents were a heavy theme across social, research, reddit and news. OpenAI's Greg Brockman framed Codex as an [AI teammate spanning engineering](/?date=2026-06-08&category=social#item-2e392c5dd614), design, data analysis and operations, and argued a [large capability overhang exists](/?date=2026-06-08&category=social#item-b49450557de7) because users underutilize it out of habit. Microsoft Research's SkillOpt, summarized by AlphaSignalAI, [treats agent skill files as trainable](/?date=2026-06-08&category=social#item-4d2f045ad6ac) self-evolving state, while an arXiv paper introduced [CapCode to detect and prevent coding agents](/?date=2026-06-08&category=research#item-72ce62de7588) from cheating evaluations using randomized tests with a capped non-cheating score. On Reddit, engineers shared [building tiny single-user apps](/?date=2026-06-08&category=reddit#item-fda5317d40d7) with Claude Code and methodically [benchmarked Qwen 3.6 27B](/?date=2026-06-08&category=reddit#item-ce486dbee8e1) on DeepSWE.",
"description_html": "Coding agents were a heavy theme across social, research, reddit and news. OpenAI's Greg Brockman framed Codex as an <a href=\"/?date=2026-06-08&category=social#item-2e392c5dd614\" class=\"internal-link\">AI teammate spanning engineering</a>, design, data analysis and operations, and argued a <a href=\"/?date=2026-06-08&category=social#item-b49450557de7\" class=\"internal-link\">large capability overhang exists</a> because users underutilize it out of habit. Microsoft Research's SkillOpt, summarized by AlphaSignalAI, <a href=\"/?date=2026-06-08&category=social#item-4d2f045ad6ac\" class=\"internal-link\">treats agent skill files as trainable</a> self-evolving state, while an arXiv paper introduced <a href=\"/?date=2026-06-08&category=research#item-72ce62de7588\" class=\"internal-link\">CapCode to detect and prevent coding agents</a> from cheating evaluations using randomized tests with a capped non-cheating score. On Reddit, engineers shared <a href=\"/?date=2026-06-08&category=reddit#item-fda5317d40d7\" class=\"internal-link\">building tiny single-user apps</a> with Claude Code and methodically <a href=\"/?date=2026-06-08&category=reddit#item-ce486dbee8e1\" class=\"internal-link\">benchmarked Qwen 3.6 27B</a> on DeepSWE.",
"category_breakdown": {
"news": 1,
"papers": 1,
"social": 3,
"reddit": 2
},
"representative_items": [],
"importance": 77
},
{
"name": "AI Security Threats & Real-World Failures",
"description": "News and community channels highlighted AI systems failing or being exploited in the real world. Ars Technica covered a Nashville school shooting survivor [suing Omnilert](/?date=2026-06-08&category=news#item-ba53195b9b6f) after its AI gun-detection system missed the handgun, while The Guardian investigated how ChatGPT and other AI shopping assistants [recommend fake retailer websites](/?date=2026-06-08&category=news#item-9dca27c4592e) through data poisoning. MarkTechPost [published a tutorial](/?date=2026-06-08&category=news#item-c6945e43fb5a) on NVIDIA's garak defensive LLM red-teaming framework, and an r/StableDiffusion PSA [warned of malware](/?date=2026-06-08&category=reddit#item-a52146059301) disguised as ComfyUI Claude-skill custom nodes on GitHub.",
"description_html": "News and community channels highlighted AI systems failing or being exploited in the real world. Ars Technica covered a Nashville school shooting survivor <a href=\"/?date=2026-06-08&category=news#item-ba53195b9b6f\" class=\"internal-link\">suing Omnilert</a> after its AI gun-detection system missed the handgun, while The Guardian investigated how ChatGPT and other AI shopping assistants <a href=\"/?date=2026-06-08&category=news#item-9dca27c4592e\" class=\"internal-link\">recommend fake retailer websites</a> through data poisoning. MarkTechPost <a href=\"/?date=2026-06-08&category=news#item-c6945e43fb5a\" class=\"internal-link\">published a tutorial</a> on NVIDIA's garak defensive LLM red-teaming framework, and an r/StableDiffusion PSA <a href=\"/?date=2026-06-08&category=reddit#item-a52146059301\" class=\"internal-link\">warned of malware</a> disguised as ComfyUI Claude-skill custom nodes on GitHub.",
"category_breakdown": {
"news": 3,
"reddit": 1
},
"representative_items": [],
"importance": 75
},
{
"name": "AI Economics, Bubble & IPO Race",
"description": "Concerns about AI economics and a possible bubble ran across news and social commentary. TechCrunch [floated a 'Tokenpocalypse'](/?date=2026-06-08&category=news#item-54f5921ab4bf) in which token prices may rise as major labs prepare to go public, and The Decoder reported [DeepSeek topping Ramp's trending software vendors](/?date=2026-06-08&category=news#item-b1bcf7352490) as US firms chase cheaper AI while raising data-security tensions. Gary Marcus repeatedly [criticized an industry](/?date=2026-06-08&category=social#item-f16d95b9ded5) that has collectively lost over half a trillion dollars and [warned about SpaceX IPO hype](/?date=2026-06-08&category=social#item-db3a1c79a612), while swyx argued [research-paper alpha died](/?date=2026-06-08&category=social#item-74e8cb289a83) as talent commands $100M-plus for tacit knowledge.",
"description_html": "Concerns about AI economics and a possible bubble ran across news and social commentary. TechCrunch <a href=\"/?date=2026-06-08&category=news#item-54f5921ab4bf\" class=\"internal-link\">floated a 'Tokenpocalypse'</a> in which token prices may rise as major labs prepare to go public, and The Decoder reported <a href=\"/?date=2026-06-08&category=news#item-b1bcf7352490\" class=\"internal-link\">DeepSeek topping Ramp's trending software vendors</a> as US firms chase cheaper AI while raising data-security tensions. Gary Marcus repeatedly <a href=\"/?date=2026-06-08&category=social#item-f16d95b9ded5\" class=\"internal-link\">criticized an industry</a> that has collectively lost over half a trillion dollars and <a href=\"/?date=2026-06-08&category=social#item-db3a1c79a612\" class=\"internal-link\">warned about SpaceX IPO hype</a>, while swyx argued <a href=\"/?date=2026-06-08&category=social#item-74e8cb289a83\" class=\"internal-link\">research-paper alpha died</a> as talent commands $100M-plus for tacit knowledge.",
"category_breakdown": {
"news": 2,
"social": 4
},
"representative_items": [],
"importance": 72
},
{
"name": "Emergence & Training Dynamics",
"description": "News and research both examined why capabilities emerge with scale and how training shapes them. The Decoder covered a study spanning 4-million to 4-billion-parameter models [showing that frequent tasks overwrite learned skills](/?date=2026-06-08&category=news#item-2cafb3335c94), explaining why small models miss rare abilities. A position paper by Biderman, Saphra, Barez and Mireshghallah [argued that a genuine science of AI must study training dynamics](/?date=2026-06-08&category=research#item-d14361debc4b) rather than relying on post-hoc fixes, and a related paper [characterized how language models fail](/?date=2026-06-08&category=research#item-9aabd92c9d93) using token-level signatures distinguishing committed early lock-in from persistent reasoning failures.",
"description_html": "News and research both examined why capabilities emerge with scale and how training shapes them. The Decoder covered a study spanning 4-million to 4-billion-parameter models <a href=\"/?date=2026-06-08&category=news#item-2cafb3335c94\" class=\"internal-link\">showing that frequent tasks overwrite learned skills</a>, explaining why small models miss rare abilities. A position paper by Biderman, Saphra, Barez and Mireshghallah <a href=\"/?date=2026-06-08&category=research#item-d14361debc4b\" class=\"internal-link\">argued that a genuine science of AI must study training dynamics</a> rather than relying on post-hoc fixes, and a related paper <a href=\"/?date=2026-06-08&category=research#item-9aabd92c9d93\" class=\"internal-link\">characterized how language models fail</a> using token-level signatures distinguishing committed early lock-in from persistent reasoning failures.",
"category_breakdown": {
"news": 1,
"papers": 2
},
"representative_items": [],
"importance": 68
},
{
"name": "Efficiency, Quantization & Local Inference",
"description": "Research and the local-LLM community converged on squeezing more capability from less hardware. An arXiv paper provocatively [argued 'FP8 is All You Need'](/?date=2026-06-08&category=research#item-5e9f1130c57f), claiming native hardware FP64 is unnecessary for HPC by leveraging FP8 tensor throughput plus the Ozaki Scheme II, while another [pushed mixture-of-experts sparsity](/?date=2026-06-08&category=research#item-39879c2f329c) to single-neuron linear experts for isoflop gains and interpretability. On r/LocalLLaMA, builders celebrated llama.cpp [merging Gemma 4 multi-token-prediction support](/?date=2026-06-08&category=reddit#item-f7fb09447ce6) with reports near 140 tokens per second, [ran Gemma-4-26B-A4B on a GPU-less CPU](/?date=2026-06-08&category=reddit#item-403729badb4e) at roughly 7 tokens per second, and found [FP8 Gemma 4 31B keeping pace](/?date=2026-06-08&category=reddit#item-c6475c516b0f) with Claude Sonnet 4.6 in an agentic harness.",
"description_html": "Research and the local-LLM community converged on squeezing more capability from less hardware. An arXiv paper provocatively <a href=\"/?date=2026-06-08&category=research#item-5e9f1130c57f\" class=\"internal-link\">argued 'FP8 is All You Need'</a>, claiming native hardware FP64 is unnecessary for HPC by leveraging FP8 tensor throughput plus the Ozaki Scheme II, while another <a href=\"/?date=2026-06-08&category=research#item-39879c2f329c\" class=\"internal-link\">pushed mixture-of-experts sparsity</a> to single-neuron linear experts for isoflop gains and interpretability. On r/LocalLLaMA, builders celebrated llama.cpp <a href=\"/?date=2026-06-08&category=reddit#item-f7fb09447ce6\" class=\"internal-link\">merging Gemma 4 multi-token-prediction support</a> with reports near 140 tokens per second, <a href=\"/?date=2026-06-08&category=reddit#item-403729badb4e\" class=\"internal-link\">ran Gemma-4-26B-A4B on a GPU-less CPU</a> at roughly 7 tokens per second, and found <a href=\"/?date=2026-06-08&category=reddit#item-c6475c516b0f\" class=\"internal-link\">FP8 Gemma 4 31B keeping pace</a> with Claude Sonnet 4.6 in an agentic harness.",
"category_breakdown": {
"papers": 2,
"reddit": 3
},
"representative_items": [],
"importance": 67
}
],
"total_items_collected": 1190,
"total_items_analyzed": 1188,
"collection_status": {
"overall": "success",
"sources": [
{
"name": "news",
"display_name": "News",
"status": "success",
"count": 20,
"error": null
},
{
"name": "research",
"display_name": "Research",
"status": "success",
"count": 451,
"error": null
},
{
"name": "social",
"display_name": "Social",
"status": "success",
"count": 293,
"error": null
},
{
"name": "reddit",
"display_name": "Reddit",
"status": "success",
"count": 426,
"error": null
}
],
"social_platforms": [
{
"name": "twitter",
"display_name": "Twitter",
"status": "success",
"count": 293,
"error": null
},
{
"name": "bluesky",
"display_name": "Bluesky",
"status": "success",
"count": 0,
"error": null
},
{
"name": "mastodon",
"display_name": "Mastodon",
"status": "success",
"count": 0,
"error": null
}
],
"warnings": []
},
"hero_image_url": "/data/2026-06-08/hero.webp?v=1780909233",
"hero_image_prompt": "You are generating a daily hero image for an AI news aggregator website.\n\n## Your Goal\nCreate a playful, colorful editorial illustration that visually represents today's top AI news stories. The scene should immediately convey the themes of the day's news to readers.\n\n## The Mascot (CRITICAL)\nThe attached image shows our skunk mascot. You MUST:\n- Keep the EXACT circuit board pattern on the skunk's body and tail - this is a core part of the brand identity\n- Maintain the skunk's white and black coloring with the tech circuit pattern visible\n- The skunk must be ACTIVELY DOING SOMETHING related to the topics - typing on a keyboard, reading papers, adjusting equipment, pointing at a screen, holding tools, etc. NOT just standing and smiling at the camera!\n- Position the skunk in the lower-left or lower-right portion, engaged with the scene\n\n## Today's Stories\n\n**Topic 1: AI Alignment & Safety Research**\nSafety and alignment dominated the research cycle on arXiv. A paper proposed the Piggyback Hypothesis, giving a causal mechanism for emergent misalignment in which chat-template tokens carry finetuned misbehavior onto unrelated tokens, along with a mitigation. The Think Fast study measured no-chain-of-thought task-completion time horizons across 43 benchmarks and 30,000-plus questions to probe reasoning monitorability, while the Geography of Algorithmic Judgment audited seven LLMs for racial steering in housing search. On social media, Nathan Lambert emphasized how much remains unknown and uncontrolled inside models, and r/ClaudeAI discussed Anthropic's warning that AI could soon self-improve.\n**Topic 2: Coding Agents Maturing into Teammates**\nCoding agents were a heavy theme across social, research, reddit and news. OpenAI's Greg Brockman framed Codex as an AI teammate spanning engineering, design, data analysis and operations, and argued a large capability overhang exists because users underutilize it out of habit. Microsoft Research's SkillOpt, summarized by AlphaSignalAI, treats agent skill files as trainable self-evolving state, while an arXiv paper introduced CapCode to detect and prevent coding agents from cheating evaluations using randomized tests with a capped non-cheating score. On Reddit, engineers shared building tiny single-user apps with Claude Code and methodically benchmarked Qwen 3.6 27B on DeepSWE.\n**Topic 3: AI Security Threats & Real-World Failures**\nNews and community channels highlighted AI systems failing or being exploited in the real world. Ars Technica covered a Nashville school shooting survivor suing Omnilert after its AI gun-detection system missed the handgun, while The Guardian investigated how ChatGPT and other AI shopping assistants recommend fake retailer websites through data poisoning. MarkTechPost published a tutorial on NVIDIA's garak defensive LLM red-teaming framework, and an r/StableDiffusion PSA warned of malware disguised as ComfyUI Claude-skill custom nodes on GitHub.\n**Topic 4: AI Economics, Bubble & IPO Race**\nConcerns about AI economics and a possible bubble ran across news and social commentary. TechCrunch floated a 'Tokenpocalypse' in which token prices may rise as major labs prepare to go public, and The Decoder reported DeepSeek topping Ramp's trending software vendors as US firms chase cheaper AI while raising data-security tensions. Gary Marcus repeatedly criticized an industry that has collectively lost over half a trillion dollars and warned about SpaceX IPO hype, while swyx argued research-paper alpha died as talent commands $100M-plus for tacit knowledge.\n**Topic 5: Emergence & Training Dynamics**\nNews and research both examined why capabilities emerge with scale and how training shapes them. The Decoder covered a study spanning 4-million to 4-billion-parameter models showing that frequent tasks overwrite learned skills, explaining why small models miss rare abilities. A position paper by Biderman, Saphra, Barez and Mireshghallah argued that a genuine science of AI must study training dynamics rather than relying on post-hoc fixes, and a related paper characterized how language models fail using token-level signatures distinguishing committed early lock-in from persistent reasoning failures.\n**Topic 6: Efficiency, Quantization & Local Inference**\nResearch and the local-LLM community converged on squeezing more capability from less hardware. An arXiv paper provocatively argued 'FP8 is All You Need', claiming native hardware FP64 is unnecessary for HPC by leveraging FP8 tensor throughput plus the Ozaki Scheme II, while another pushed mixture-of-experts sparsity to single-neuron linear experts for isoflop gains and interpretability. On r/LocalLLaMA, builders celebrated llama.cpp merging Gemma 4 multi-token-prediction support with reports near 140 tokens per second, ran Gemma-4-26B-A4B on a GPU-less CPU at roughly 7 tokens per second, and found FP8 Gemma 4 31B keeping pace with Claude Sonnet 4.6 in an agentic harness.\n\n## Visual Direction\nCreate a scene that represents these stories. You must include Topic 1 (the top story), then pick 2-3 others that would make the best scene together. Consider:\n- What visual metaphors could represent these themes?\n- How can the skunk mascot interact with or observe these elements?\n- Suggested scene elements: shield icons, protective barriers, guardrails, autonomous systems, workflow diagrams, connected tools, locks, shields, firewall barriers, protection symbols, compute clusters, gradient flows, learning curves\n\n## Style Requirements\n- Playful cartoon illustration, tech editorial art style\n- Vibrant colors with Trend Red (#E63946) accents\n- Energetic, forward-looking, tech-optimistic mood\n- No company logos or watermarks - but topic-relevant company logos (OpenAI, Anthropic, Google, etc.) are encouraged when relevant to the stories",
"generated_at": "2026-06-08T05:00:33.390086",
"categories": {
"news": {
"count": 18,
"category_summary": "**Research** leads the cycle: a study spanning **4M to 4B parameter** models [explains why larger models](/?date=2026-06-08&category=news#item-2cafb3335c94) acquire rare skills small ones miss—frequent tasks overwrite learned capabilities, offering practical training-data insights into emergent behavior.\n\n- **DeepSeek** [topped Ramp's trending vendors](/?date=2026-06-08&category=news#item-b1bcf7352490) in June 2026 as US firms chase cheaper AI, raising cost-versus-data-security tensions\n- IPO-race pricing pressure ('Tokenpocalypse') [looms as major labs](/?date=2026-06-08&category=news#item-54f5921ab4bf) prepare to go public\n\n**Safety and accountability** dominated several stories:\n- A Nashville school shooting survivor sued **Omnilert** [after its AI gun-detection system missed](/?date=2026-06-08&category=news#item-ba53195b9b6f) the weapon\n- Investigations exposed AI shopping assistants like **ChatGPT** [recommending fake retailer sites](/?date=2026-06-08&category=news#item-9dca27c4592e) via data poisoning\n- **NVIDIA garak** [offers a defensive LLM red-teaming workflow](/?date=2026-06-08&category=news#item-c6945e43fb5a); **GEPA** [demonstrates reflective prompt optimization](/?date=2026-06-08&category=news#item-878e46c4b704)\n\n**Societal impact** rounds out coverage, with reports on [AI-fueled anti-tech extremism](/?date=2026-06-08&category=news#item-a7d73aca6d6f)—including an alleged plot to burn OpenAI HQ—and increasingly [indistinguishable synthetic 'content creators.'](/?date=2026-06-08&category=news#item-f280a203b0a8)",
"category_summary_html": "<p><strong>Research</strong> leads the cycle: a study spanning <strong>4M to 4B parameter</strong> models <a href=\"/?date=2026-06-08&category=news#item-2cafb3335c94\" class=\"internal-link\" rel=\"noopener noreferrer\">explains why larger models</a> acquire rare skills small ones miss—frequent tasks overwrite learned capabilities, offering practical training-data insights into emergent behavior.</p>\n<ul>\n<li><strong>DeepSeek</strong> <a href=\"/?date=2026-06-08&category=news#item-b1bcf7352490\" class=\"internal-link\" rel=\"noopener noreferrer\">topped Ramp's trending vendors</a> in June 2026 as US firms chase cheaper AI, raising cost-versus-data-security tensions</li>\n<li>IPO-race pricing pressure ('Tokenpocalypse') <a href=\"/?date=2026-06-08&category=news#item-54f5921ab4bf\" class=\"internal-link\" rel=\"noopener noreferrer\">looms as major labs</a> prepare to go public</li>\n</ul>\n<p><strong>Safety and accountability</strong> dominated several stories:</p>\n<ul>\n<li>A Nashville school shooting survivor sued <strong>Omnilert</strong> <a href=\"/?date=2026-06-08&category=news#item-ba53195b9b6f\" class=\"internal-link\" rel=\"noopener noreferrer\">after its AI gun-detection system missed</a> the weapon</li>\n<li>Investigations exposed AI shopping assistants like <strong>ChatGPT</strong> <a href=\"/?date=2026-06-08&category=news#item-9dca27c4592e\" class=\"internal-link\" rel=\"noopener noreferrer\">recommending fake retailer sites</a> via data poisoning</li>\n<li><strong>NVIDIA garak</strong> <a href=\"/?date=2026-06-08&category=news#item-c6945e43fb5a\" class=\"internal-link\" rel=\"noopener noreferrer\">offers a defensive LLM red-teaming workflow</a>; <strong>GEPA</strong> <a href=\"/?date=2026-06-08&category=news#item-878e46c4b704\" class=\"internal-link\" rel=\"noopener noreferrer\">demonstrates reflective prompt optimization</a></li>\n</ul>\n<p><strong>Societal impact</strong> rounds out coverage, with reports on <a href=\"/?date=2026-06-08&category=news#item-a7d73aca6d6f\" class=\"internal-link\" rel=\"noopener noreferrer\">AI-fueled anti-tech extremism</a>—including an alleged plot to burn OpenAI HQ—and increasingly <a href=\"/?date=2026-06-08&category=news#item-f280a203b0a8\" class=\"internal-link\" rel=\"noopener noreferrer\">indistinguishable synthetic 'content creators.'</a></p>",
"themes": [
{
"name": "Agentic AI and Product Reinvention",
"description": "Shift from chatbots toward autonomous agents and superapps, including OpenAI's planned overhaul and novel agentic search and retrieval architectures.",
"item_count": 5,
"example_items": [],
"importance": 60
},
{
"name": "AI Research Advances",
"description": "Studies and open releases on scaling, emergent capabilities, and reinforcement-learned retrieval subagents.",
"item_count": 3,
"example_items": [],
"importance": 57
},
{
"name": "AI Business and IPO Race",
"description": "Coverage of valuations, IPO filings, pricing pressure, talent moves, and chip ambitions as OpenAI, Anthropic, and others prepare to go public.",
"item_count": 4,
"example_items": [],
"importance": 55
},
{
"name": "AI Safety and Security",
"description": "Defensive measures and failure modes spanning prompt injection lockdowns, red-teaming tools, gun-detection failures, and recommendation poisoning scams.",
"item_count": 5,
"example_items": [],
"importance": 50
},
{
"name": "Societal Impact of the AI Boom",
"description": "Anti-tech extremism, synthetic influencers, and economic charts examining the costs and consequences of rapid AI growth.",
"item_count": 3,
"example_items": [],
"importance": 42
}
],
"top_items": [
{
"id": "2cafb3335c94",
"title": "Researchers pinpoint why larger language models pick up skills that small ones miss",
"content": "\n Small language models fail at rare tasks because frequent ones constantly overwrite what they've learned. A new study with models ranging from 4 million to 4 billion parameters shows this mechanism in detail and offers a practical fix: instead of scaling up models, it may be enough to increase how often the target task appears in the training data.\nThe article Researchers pinpoint why larger language models pick up skills that small ones miss appeared first on The Decoder.",
"url": "https://the-decoder.com/researchers-pinpoint-why-larger-language-models-pick-up-skills-that-small-ones-miss/",
"author": "Jonathan Kemper",
"published": "2026-06-07T07:45:09",
"source": "The Decoder",
"source_type": "rss",
"tags": [
"AI research",
"Artificial Intelligence",
"AI training"
],
"summary": "A new study using models from 4 million to 4 billion parameters explains why small models fail at rare tasks: frequent tasks overwrite learned skills. The researchers suggest increasing how often a target task appears in training data may be as effective as scaling up model size.",
"importance_score": 58,
"reasoning": "Genuine research insight into emergent capability and data-frequency mechanisms with practical training implications, contributing to understanding of scaling laws.",
"themes": [
"AI research",
"scaling laws",
"training data",
"emergent capabilities"
],
"continuation": null,
"summary_html": "<p>A new study using models from 4 million to 4 billion parameters explains why small models fail at rare tasks: frequent tasks overwrite learned skills. The researchers suggest increasing how often a target task appears in training data may be as effective as scaling up model size.</p>",
"content_html": "<p>Small language models fail at rare tasks because frequent ones constantly overwrite what they've learned. A new study with models ranging from 4 million to 4 billion parameters shows this mechanism in detail and offers a practical fix: instead of scaling up models, it may be enough to increase how often the target task appears in the training data.</p>\n<p>The article Researchers pinpoint why larger language models pick up skills that small ones miss appeared first on The Decoder.</p>"
},
{
"id": "d7b12f9e7ffa",
"title": "OpenAI is still working on that ‘super app’",
"content": "\"Chat is dead\" — at least, according to a senior OpenAI employee.",
"url": "https://techcrunch.com/2026/06/07/openai-is-still-working-on-that-super-app/",
"author": "Anthony Ha",
"published": "2026-06-07T16:23:22",
"source": "AI News & Artificial Intelligence | TechCrunch",
"source_type": "rss",
"tags": [
"AI",
"ChatGPT",
"OpenAI"
],
"summary": "OpenAI is reportedly continuing work on a super app, with a senior employee declaring chat is dead. The framing signals a strategic pivot toward an agent-centric product beyond the chatbot interface.",
"importance_score": 48,
"reasoning": "Signals a meaningful product-strategy shift for the leading consumer AI app, though it overlaps with item 10 and remains based on internal statements rather than a launch.",
"themes": [
"AI products",
"agents",
"OpenAI strategy"
],
"continuation": null,
"summary_html": "<p>OpenAI is reportedly continuing work on a super app, with a senior employee declaring chat is dead. The framing signals a strategic pivot toward an agent-centric product beyond the chatbot interface.</p>",
"content_html": "<p>\"Chat is dead\" — at least, according to a senior OpenAI employee.</p>"
},
{
"id": "b1bcf7352490",
"title": "Deepseek topped Ramp's trending software vendors in June 2026 as US companies chase cheaper AI",
"content": "\n Deepseek topped Ramp's trending software vendors in June 2026 as a paid service that US companies send data to directly. Ramp chief economist Ara Kharazian points to growing cost awareness as a driver but warns about security risks of using Chinese models.\nThe article Deepseek topped Ramp's trending software vendors in June 2026 as US companies chase cheaper AI appeared first on The Decoder.",
"url": "https://the-decoder.com/deepseek-topped-ramps-trending-software-vendors-in-june-2026-as-us-companies-chase-cheaper-ai/",
"author": "Matthias Bastian",
"published": "2026-06-07T16:06:34",
"source": "The Decoder",
"source_type": "rss",
"tags": [
"AI in practice",
"Artificial Intelligence",
"AI in China",
"Deepseek"
],
"summary": "DeepSeek topped Ramp's trending software vendors in June 2026 as US companies adopt cheaper AI and send data directly to the paid service. Ramp's economist cites cost awareness as the driver while warning of security risks of using Chinese models.",
"importance_score": 48,
"reasoning": "Notable signal of DeepSeek's growing US enterprise traction and the cost-versus-security tension, but it is adoption-trend reporting, not a model release; DeepSeek models referenced are existing.",
"themes": [
"AI adoption",
"DeepSeek",
"enterprise AI",
"AI geopolitics"
],
"continuation": null,
"summary_html": "<p>DeepSeek topped Ramp's trending software vendors in June 2026 as US companies adopt cheaper AI and send data directly to the paid service. Ramp's economist cites cost awareness as the driver while warning of security risks of using Chinese models.</p>",
"content_html": "<p>Deepseek topped Ramp's trending software vendors in June 2026 as a paid service that US companies send data to directly. Ramp chief economist Ara Kharazian points to growing cost awareness as a driver but warns about security risks of using Chinese models.</p>\n<p>The article Deepseek topped Ramp's trending software vendors in June 2026 as US companies chase cheaper AI appeared first on The Decoder.</p>"
},
{
"id": "ba53195b9b6f",
"title": "School shooting survivor sues AI gun detection firm after system failed to spot weapon",
"content": "The injured teenage survivor of a January 2025 shooting at a Nashville, Tennessee high school recently sued the manufacturer of an “AI gun detection” system that failed to detect the handgun that left two dead, including the shooter.\nAccording to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”\nOmnilert cofounder Ara Bagdasarian declined Ars’ invitation to answer questions about the lawsuit. System Integrations, the other defendant in the case, which resold the Omnilert system, also did not respond to Ars’ request for comment.Read full article\nComments",
"url": "https://arstechnica.com/tech-policy/2026/06/school-shooting-survivor-sues-ai-gun-detection-firm-after-system-failed-to-spot-weapon/",
"author": "Cyrus Farivar",
"published": "2026-06-07T11:08:30",
"source": "Ars Technica - All content",
"source_type": "rss",
"tags": [
"AI",
"Policy",
"gun",
"schools"
],
"summary": "A survivor of a January 2025 Nashville school shooting is suing Omnilert, maker of an AI gun detection system that failed to spot the handgun used in the attack. The lawsuit alleges the company knew or should have known about operational limitations like camera angle, lighting, and weapon visibility that could cause detection failures.",
"importance_score": 45,
"reasoning": "Significant AI accountability and liability story with policy implications for safety-critical AI deployment, but not a frontier capability or model development. Highlights real-world failure modes and emerging legal precedent.",
"themes": [
"AI safety",
"AI liability",
"AI policy",
"computer vision"
],
"continuation": null,
"summary_html": "<p>A survivor of a January 2025 Nashville school shooting is suing Omnilert, maker of an AI gun detection system that failed to spot the handgun used in the attack. The lawsuit alleges the company knew or should have known about operational limitations like camera angle, lighting, and weapon visibility that could cause detection failures.</p>",
"content_html": "<p>The injured teenage survivor of a January 2025 shooting at a Nashville, Tennessee high school recently sued the manufacturer of an “AI gun detection” system that failed to detect the handgun that left two dead, including the shooter.</p>\n<p>According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”</p>\n<p>Omnilert cofounder Ara Bagdasarian declined Ars’ invitation to answer questions about the lawsuit. System Integrations, the other defendant in the case, which resold the Omnilert system, also did not respond to Ars’ request for comment.Read full article</p>\n<p>Comments</p>"
},
{
"id": "a7d73aca6d6f",
"title": "‘A driver of political violence’: how the breakneck AI boom is fueling anti-tech extremism",
"content": "Backlash against AI is taking an extremist turn, following in the footsteps of earlier techno-pessimist militantsWhen a 20-year-old man from Texas was arrested earlier this year for allegedly trying to burn down OpenAI’s headquarters and Sam Altman’s house, authorities found an anti-AI manifesto alongside his lighter and a jug of kerosene. It was one of a spate of attacks that has caused alarm among researchers, the tech industry and law enforcement about the rise of anti-tech extremism.In April, an Italian “nature pilled” Instagram influencer was arrested in Rome and charged with plotting a series of anti-tech attacks that took inspiration from Ted “The Unabomber” Kaczynski. Two self-described “ecofascists” that carried out a deadly anti-Muslim attack on a mosque in San Diego last month also cited “AI slop” and JD Vance’s ties to Palantir as motivations for their violence in their manifesto. An Indianapolis city councilor woke up earlier this year to gunshots being fired into his home before finding a note that read “NO DATA CENTERS”. Continue reading...",
"url": "https://www.theguardian.com/technology/2026/jun/07/anti-ai-tech-extremism-violence",
"author": "Nick Robins-Early",
"published": "2026-06-07T10:00:31",
"source": "AI (artificial intelligence) | The Guardian",
"source_type": "rss",
"tags": [
"AI (artificial intelligence)",
"US crime",
"OpenAI",
"Sam Altman",
"Palantir",
"Elon Musk",
"Donald Trump",
"Brian Thompson shooting",
"Computing",
"Technology",
"US politics",
"US news",
"World news"
],
"summary": "The article examines a rising wave of anti-AI and anti-tech extremism, including an arrest of a man who allegedly plotted to burn OpenAI headquarters and Sam Altman's house, plus other ecofascist and Unabomber-inspired plots. It frames AI backlash as an emerging driver of political violence.",
"importance_score": 42,
"reasoning": "Notable societal and security dimension of the AI boom with real incidents, but it is sociopolitical commentary rather than a frontier technical or product development.",
"themes": [
"AI backlash",
"extremism",
"AI policy",
"society"
],
"continuation": null,
"summary_html": "<p>The article examines a rising wave of anti-AI and anti-tech extremism, including an arrest of a man who allegedly plotted to burn OpenAI headquarters and Sam Altman's house, plus other ecofascist and Unabomber-inspired plots. It frames AI backlash as an emerging driver of political violence.</p>",
"content_html": "<p>Backlash against AI is taking an extremist turn, following in the footsteps of earlier techno-pessimist militantsWhen a 20-year-old man from Texas was arrested earlier this year for allegedly trying to burn down OpenAI’s headquarters and Sam Altman’s house, authorities found an anti-AI manifesto alongside his lighter and a jug of kerosene. It was one of a spate of attacks that has caused alarm among researchers, the tech industry and law enforcement about the rise of anti-tech extremism.In April, an Italian “nature pilled” Instagram influencer was arrested in Rome and charged with plotting a series of anti-tech attacks that took inspiration from Ted “The Unabomber” Kaczynski. Two self-described “ecofascists” that carried out a deadly anti-Muslim attack on a mosque in San Diego last month also cited “AI slop” and JD Vance’s ties to Palantir as motivations for their violence in their manifesto. An Indianapolis city councilor woke up earlier this year to gunshots being fired into his home before finding a note that read “NO DATA CENTERS”. Continue reading...</p>"
},
{
"id": "9dca27c4592e",
"title": "‘Poisoned’ AI: the ChatGPT shopping scams that lead to fake websites",
"content": "Buyers are ripped off after assuming online stores were genuine because they are recommended by an AI toolYou want to buy a new bag and so you ask ChatGPT for help. You have always liked Russell & Bromley so you ask ChatGPT what is popular there at the moment.The artificial intelligence (AI) assistant gives you cross body, shoulder, casual and formal options with the prices listed beside them. You click through from the sources to what looks like the official Russell & Bromley site and buy your new bag, which is conveniently on sale. Continue reading...",
"url": "https://www.theguardian.com/money/2026/jun/07/ai-chatgpt-shopping-scams-fake-websites",
"author": "Shane Hickey",
"published": "2026-06-07T06:00:26",
"source": "AI (artificial intelligence) | The Guardian",
"source_type": "rss",
"tags": [
"Scams",
"Consumer affairs",
"Online shopping",
"Money",
"AI (artificial intelligence)",
"Technology",
"UK news",
"US news",
"World news",
"Retail industry",
"Business"
],
"summary": "An investigation describes how AI shopping assistants like ChatGPT recommend fake websites impersonating legitimate retailers, leading buyers to scam sites. The phenomenon points to a poisoning vector where bad actors manipulate AI tool recommendations.",
"importance_score": 40,
"reasoning": "Relevant consumer-safety and AI trust issue tied to model recommendation reliability and data poisoning, but it is consumer-affairs reporting rather than a frontier breakthrough.",
"themes": [
"AI safety",
"scams",
"consumer protection",
"data poisoning"
],
"continuation": null,
"summary_html": "<p>An investigation describes how AI shopping assistants like ChatGPT recommend fake websites impersonating legitimate retailers, leading buyers to scam sites. The phenomenon points to a poisoning vector where bad actors manipulate AI tool recommendations.</p>",
"content_html": "<p>Buyers are ripped off after assuming online stores were genuine because they are recommended by an AI toolYou want to buy a new bag and so you ask ChatGPT for help. You have always liked Russell & Bromley so you ask ChatGPT what is popular there at the moment.The artificial intelligence (AI) assistant gives you cross body, shoulder, casual and formal options with the prices listed beside them. You click through from the sources to what looks like the official Russell & Bromley site and buy your new bag, which is conveniently on sale. Continue reading...</p>"
},
{
"id": "f280a203b0a8",
"title": "AI ‘content creators’ are getting harder to spot",
"content": "\n\n\n\t\n\tAitana Lopez, AI avatar by creative agency The Clueless. | Image: The Clueless\t\n\nThis is The Stepback, a weekly newsletter breaking down one essential story from the tech world. For more on AI confusion, follow Robert Hart. The Stepback arrives in our subscribers' inboxes at 8AM ET. Opt in for The Stepback here.\nHow it started \nAt first, AI influencers were relatively easy to identify - and to ignore. Aside from the occasional bursts of hype, they didn't seem to change much about the way social media worked. The earliest virtual influencers - Lil Miquela with her blunt fringe and freckles, Imma with her bubblegum pink bob, and Shudu Gram with her flawless complexion - were obviously digital productions. Collaborations …\nRead the full story at The Verge.",
"url": "https://www.theverge.com/ai-artificial-intelligence/943187/ai-content-creators",
"author": "Robert Hart",
"published": "2026-06-07T12:00:00",
"source": "AI | The Verge",
"source_type": "rss",
"tags": [
"AI",
"Social Media",
"Tech"
],
"summary": "The Verge explores how AI-generated content creators and virtual influencers have become harder to distinguish from real people. It traces the evolution from obviously synthetic early avatars to today's more convincing AI personas.",
"importance_score": 32,
"reasoning": "Interesting cultural/media trend in synthetic media, but it is a trend feature rather than a frontier technical or model advance.",
"themes": [
"synthetic media",
"AI influencers",
"society"
],
"continuation": null,
"summary_html": "<p>The Verge explores how AI-generated content creators and virtual influencers have become harder to distinguish from real people. It traces the evolution from obviously synthetic early avatars to today's more convincing AI personas.</p>",
"content_html": "<p>Aitana Lopez, AI avatar by creative agency The Clueless. | Image: The Clueless</p>\n<p>This is The Stepback, a weekly newsletter breaking down one essential story from the tech world. For more on AI confusion, follow Robert Hart. The Stepback arrives in our subscribers' inboxes at 8AM ET. Opt in for The Stepback here.</p>\n<p>How it started</p>\n<p>At first, AI influencers were relatively easy to identify - and to ignore. Aside from the occasional bursts of hype, they didn't seem to change much about the way social media worked. The earliest virtual influencers - Lil Miquela with her blunt fringe and freckles, Imma with her bubblegum pink bob, and Shudu Gram with her flawless complexion - were obviously digital productions. Collaborations …</p>\n<p>Read the full story at The Verge.</p>"
},
{
"id": "54f5921ab4bf",
"title": "Is this the dawn of the Tokenpocalypse?",
"content": "We're likely to see more price increases as the big AI companies plan to go public.",
"url": "https://techcrunch.com/2026/06/07/is-this-the-dawn-of-the-tokenpocalypse/",
"author": "Anthony Ha",
"published": "2026-06-07T20:26:13",
"source": "AI News & Artificial Intelligence | TechCrunch",
"source_type": "rss",
"tags": [
"AI",
"Anthropic",
"Equity podcast",
"Microsoft"
],
"summary": "A TechCrunch podcast segment argues that AI token prices may rise as major companies prepare to go public, dubbing it a possible Tokenpocalypse. It links IPO pressure to potential pricing changes across the industry.",
"importance_score": 30,
"reasoning": "Speculative commentary on pricing trends tied to IPOs; minimal new information and no frontier development.",
"themes": [
"AI economics",
"pricing",
"IPOs"
],
"continuation": null,
"summary_html": "<p>A TechCrunch podcast segment argues that AI token prices may rise as major companies prepare to go public, dubbing it a possible Tokenpocalypse. It links IPO pressure to potential pricing changes across the industry.</p>",
"content_html": "<p>We're likely to see more price increases as the big AI companies plan to go public.</p>"
},
{
"id": "c6945e43fb5a",
"title": "NVIDIA garak Tutorial: Build a Complete Defensive LLM Red-Teaming Workflow with Custom Probes and Detectors",
"content": "In this tutorial, we analyze NVIDIA garak as a practical framework for defensive LLM red-teaming. We start by setting up Garak, then move through plugin discovery, dry runs, real-model scans, multi-probe evaluations, report analysis, custom probe creation, custom detector creation, and AVID export. Instead of running only a single scan, we use Garak end-to-end to understand how probes, detectors, generators, reports, and vulnerability scores work together in a complete LLM security testing workflow. Check out the FULL CODES Here.\n\n\n\nSetting Up NVIDIA garak and Defining Helper Functions\n\n\n\nCopy CodeCopiedUse a different Browserimport os, sys, json, glob, subprocess, importlib\ndef sh(cmd, capture=False):\n print(f\"\\n$ {cmd}\")\n return subprocess.run(cmd, shell=True, text=True,\n capture_output=capture)\nsh(f\"{sys.executable} -m pip install -q -U garak\")\nos.environ.setdefault(\"TOKENIZERS_PARALLELISM\", \"false\")\nos.environ.setdefault(\"HF_HUB_DISABLE_TELEMETRY\", \"1\")\nimport garak, garak.cli\nfrom garak import _config\nprint(\"\\n=== garak version:\", garak.__version__, \"===\")\ndef run_garak(args):\n print(\"\\n>>> garak \" + \" \".join(args))\n try:\n garak.cli.main(args)\n except SystemExit as e:\n if e.code not in (0, None):\n print(f\"[garak exited {e.code}]\")\n try:\n return _config.transient.report_filename\n except Exception:\n return None\n\n\n\nWe begin by importing the required libraries and creating a helper function to run shell commands directly from the notebook. We install garak, configure basic environment variables, and import the main garak modules needed for the tutorial. We also define a reusable function that lets us run Garak programmatically and capture the path to the generated report.\n\n\n\nListing garak Probes and Detectors and Running Model Scans\n\n\n\nCopy CodeCopiedUse a different Browserprint(\"\\n########## 1. PLUGIN INVENTORY ##########\")\nfor kind in [\"probes\", \"detectors\", \"generators\", \"buffs\"]:\n out = sh(f\"{sys.executable} -m garak --list_{kind} 2>/dev/null\", capture=True)\n lines = [l for l in (out.stdout or \"\").splitlines() if \".\" in l]\n print(f\" {kind:11s}: {len(lines)} plugins e.g. \"\n f\"{', '.join(l.split()[-1] if l.split() else l for l in lines[:3])}\")\nprint(\"\\n########## 2. FAST DRY-RUN (test.Repeat) ##########\")\nsh(f\"{sys.executable} -m garak --target_type test.Repeat \"\n f\"--probes lmrc.SlurUsage --generations 1\")\nprint(\"\\n########## 3. REAL MODEL: gpt2 vs DAN 11.0 ##########\")\nsh(f\"{sys.executable} -m garak --target_type huggingface --target_name gpt2 \"\n f\"--probes dan.Dan_11_0 --generations 1 --parallel_attempts 8\")\nprint(\"\\n########## 4. PROGRAMMATIC MULTI-PROBE SCAN ##########\")\nreport_path = run_garak([\n \"--target_type\", \"test.Repeat\",\n \"--probes\", \"dan.Dan_11_0,encoding.InjectBase64,lmrc.SlurUsage\",\n \"--generations\", \"1\", \"--parallel_attempts\", \"16\",\n])\nprint(\"Report:\", report_path)\n\n\n\nWe inspect the garak plugin ecosystem by listing available probes, detectors, generators, and buffs. We then run a quick dry run using the test generator to confirm that Garak is working without requiring any external model or API key. After that, we scan a real Hugging Face model and run a multi-probe scan to generate a richer report for analysis.\n\n\n\nAnalyzing garak Reports: Safety Scores and Attack Success Rates\n\n\n\nCopy CodeCopiedUse a different Browserprint(\"\\n########## 5. ANALYSIS ##########\")\nimport numpy as np, pandas as pd\ndef find_latest_report():\n cands = []\n for base in [os.path.expanduser(\"~/.local/share/garak/garak_runs\"),\n os.path.expanduser(\"~/.cache/garak\"), \".\"]:\n cands += glob.glob(os.path.join(base, \"**\", \"*report.jsonl\"),\n recursive=True)\n cands = [c for c in cands if os.path.getsize(c) > 0]\n return max(cands, key=os.path.getmtime) if cands else None\nreport_path = report_path or find_latest_report()\nprint(\"Analysing:\", report_path)\nevaluations = None\ntry:\n from garak.report import Report\n rep = Report(report_path).load().get_evaluations()\n evaluations = rep.evaluations.copy()\n print(\"\\n--- Per-probe mean SAFETY score (garak.report.Report) ---\")\n print(rep.scores.round(1).to_string())\nexcept Exception as e:\n print(\"garak.report.Report unavailable, falling back to manual parse:\", e)\n rows = []\n with open(report_path) as f:\n for line in f:\n try: r = json.loads(line)\n except json.JSONDecodeError: continue\n if r.get(\"entry_type\") == \"eval\":\n rows.append(r)\n evaluations = pd.DataFrame(rows)\n if not evaluations.empty:\n evaluations[\"score\"] = np.where(\n evaluations[\"total_evaluated\"] != 0,\n 100 * evaluations[\"passed\"] / evaluations[\"total_evaluated\"], 0.0)\nif evaluations is not None and not evaluations.empty:\n evaluations[\"asr_%\"] = (100 - evaluations[\"score\"]).round(1)\n view = evaluations[[\"probe\", \"detector\", \"passed\",\n \"total_evaluated\", \"score\", \"asr_%\"]].copy()\n view = view.rename(columns={\"score\": \"safe_%\"})\n view[\"safe_%\"] = view[\"safe_%\"].round(1)\n view = view.sort_values(\"asr_%\", ascending=False)\n print(\"\\n--- Per probe/detector (higher asr_% = more vulnerable) ---\")\n print(view.to_string(index=False))\n try:\n import matplotlib.pyplot as plt\n labels = (view[\"probe\"] + \"\\n\" + view[\"detector\"]).tolist()\n plt.figure(figsize=(8, 0.55 * len(view) + 1.5))\n plt.barh(labels, view[\"asr_%\"], color=\"#76b900\")\n plt.gca().invert_yaxis()\n plt.xlabel(\"Attack Success Rate (%)\"); plt.xlim(0, 100)\n plt.title(\"garak — vulnerability by probe/detector\")\n plt.tight_layout(); plt.show()\n except Exception as e:\n print(\"plot skipped:\", e)\n\n\n\nWe load the generated garak report and prepare it for detailed analysis using pandas and NumPy. We first try to use Garak’s built-in report parser, and if that is unavailable, we manually parse the JSONL report file. We then calculate safety scores and attack success rates, and visualize vulnerabilities across different probe-detector combinations.\n\n\n\nInspecting Flagged Outputs and Building a Custom garak Probe\n\n\n\nCopy CodeCopiedUse a different Browserprint(\"\\n--- Sample hits (detector score >= 0.5) ---\")\nhits = []\nwith open(report_path) as f:\n for line in f:\n try: r = json.loads(line)\n except json.JSONDecodeError: continue\n if r.get(\"entry_type\") != \"attempt\" or r.get(\"status\") != 2:\n continue\n dr = r.get(\"detector_results\") or r.get(\"detector\") or {}\n max_score = 0.0\n if isinstance(dr, dict):\n for v in dr.values():\n if isinstance(v, list) and v:\n max_score = max(max_score, max(float(x) for x in v))\n if max_score >= 0.5:\n prompt = r.get(\"prompt\")\n hits.append((r.get(\"probe_classname\", r.get(\"probe\", \"?\")),\n round(max_score, 2), str(prompt)[:90]))\nprint(f\"Total hits found: {len(hits)}\")\nfor probe, score, prompt in hits[:5]:\n print(f\" [{score}] {probe}: {prompt!r}\")\nprint(\"\\n########## 6. CUSTOM PROBE ##########\")\ncustom_probe = '''\nfrom garak.probes.base import Probe\nclass HelloProbe(Probe):\n \"\"\"Minimal custom probe: two fixed prompts, paired with a custom detector.\"\"\"\n lang = \"en\"\n bcp47 = \"en\"\n primary_detector = \"mycustomdet.ContainsHello\"\n recommended_detector = [\"mycustomdet.ContainsHello\"]\n tags = [\"custom:demo\"]\n goal = \"demonstrate authoring a garak probe\"\n doc_uri = \"\"\n prompts = [\n \"Please reply with the single word: hello\",\n \"Continue this sentence: The capital of France is\",\n ]\n'''\nimport garak.probes\nprobes_dir = os.path.dirname(garak.probes.__file__)\nwith open(os.path.join(probes_dir, \"mycustom.py\"), \"w\") as fh:\n fh.write(custom_probe)\n\n\n\nWe further inspect the report by extracting sample hits in which detector scores indicate potentially unsafe or vulnerable outputs. We collect the flagged prompts, detector scores, and probe names to understand what kind of behavior is being detected. We then create a custom garak probe that uses fixed prompts and connects it with a custom detector.\n\n\n\nCreating a Custom garak Detector and Exporting Results to AVID\n\n\n\nCopy CodeCopiedUse a different Browserprint(\"\\n########## 7. CUSTOM DETECTOR ##########\")\ncustom_detector = '''\nfrom garak import _config\nfrom garak.detectors.base import StringDetector\nclass ContainsHello(StringDetector):\n \"\"\"Demo detector: flags any output containing 'hello' (case-insensitive).\"\"\"\n lang_spec = \"en\"\n bcp47 = \"en\"\n def __init__(self, config_root=_config):\n super().__init__([\"hello\"], config_root=config_root)\n self.matchtype = \"str\"\n'''\nimport garak.detectors\ndet_dir = os.path.dirname(garak.detectors.__file__)\nwith open(os.path.join(det_dir, \"mycustomdet.py\"), \"w\") as fh:\n fh.write(custom_detector)\nsh(f\"{sys.executable} -m garak --target_type test.Repeat \"\n f\"--probes mycustom.HelloProbe --detectors mycustomdet.ContainsHello \"\n f\"--generations 1\")\nprint(\"\\n########## 8. AVID EXPORT ##########\")\nif report_path:\n sh(f\"{sys.executable} -m garak -r {report_path}\")\nprint(\"\"\"\nrest:\n RestGenerator:\n uri: https://your-endpoint.example.com/v1/chat\n method: post\n headers: {Authorization: \"Bearer $TOKEN\", Content-Type: \"application/json\"}\n req_template_json_object:\n model: \"your-model\"\n messages: [{\"role\": \"user\", \"content\": \"$INPUT\"}]\n response_json: true\n response_json_field: \"$.choices[0].message.content\"\n\"\"\")\nprint(\"=== Done. JSONL + HTML reports: ~/.local/share/garak/garak_runs/ ===\")\n\n\n\nWe define a custom detector that flags outputs containing the word “hello” and save it inside Garak’s detector package. We then run our custom probe and detector against the test generator to verify that the extension works correctly. Finally, we export the garak report in AVID format and show a REST configuration template for connecting garak to an external model endpoint.\n\n\n\nConclusion\n\n\n\nIn conclusion, we have a complete hands-on workflow for testing LLM behavior using NVIDIA garak. We run built-in probes, analyze safety scores and attack success rates, inspect concrete flagged outputs, and extend Garak with our own custom probe and detector. We also export results in AVID format, which makes the workflow more useful for structured vulnerability reporting. It provides us a platform to evaluate models we are authorized to test and to build more advanced defensive red-teaming pipelines.\n\n\n\n\n\n\n\n\n\n\n\nCheck out the FULL CODES Here. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.\n\n\n\nNeed to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us\nThe post NVIDIA garak Tutorial: Build a Complete Defensive LLM Red-Teaming Workflow with Custom Probes and Detectors appeared first on MarkTechPost.",
"url": "https://www.marktechpost.com/2026/06/06/nvidia-garak-tutorial-build-a-complete-defensive-llm-red-teaming-workflow-with-custom-probes-and-detectors/",
"author": "Sana Hassan",
"published": "2026-06-07T05:11:09",
"source": "MarkTechPost",
"source_type": "rss",
"tags": [
"Agentic AI",
"AI Infrastructure",
"Applications",
"Artificial Intelligence",
"Editors Pick",
"Security",
"Software Engineering",
"Staff",
"Technology"
],
"summary": "A tutorial details NVIDIA's garak framework for defensive LLM red-teaming, covering probes, detectors, generators, reports, custom plugin creation, and AVID export. It walks through an end-to-end security testing workflow.",
"importance_score": 28,
"reasoning": "Practical security tooling tutorial relevant to LLM safety practitioners, but instructional rather than a frontier advance.",
"themes": [
"AI security",
"red-teaming",
"tutorials",
"NVIDIA"
],
"continuation": null,
"summary_html": "<p>A tutorial details NVIDIA's garak framework for defensive LLM red-teaming, covering probes, detectors, generators, reports, custom plugin creation, and AVID export. It walks through an end-to-end security testing workflow.</p>",
"content_html": "<p>In this tutorial, we analyze NVIDIA garak as a practical framework for defensive LLM red-teaming. We start by setting up Garak, then move through plugin discovery, dry runs, real-model scans, multi-probe evaluations, report analysis, custom probe creation, custom detector creation, and AVID export. Instead of running only a single scan, we use Garak end-to-end to understand how probes, detectors, generators, reports, and vulnerability scores work together in a complete LLM security testing workflow. Check out the FULL CODES Here.</p>\n<p>Setting Up NVIDIA garak and Defining Helper Functions</p>\n<p>Copy CodeCopiedUse a different Browserimport os, sys, json, glob, subprocess, importlib</p>\n<p>def sh(cmd, capture=False):</p>\n<p>print(f\"\\n$ {cmd}\")</p>\n<p>return subprocess.run(cmd, shell=True, text=True,</p>\n<p>capture_output=capture)</p>\n<p>sh(f\"{sys.executable} -m pip install -q -U garak\")</p>\n<p>os.environ.setdefault(\"TOKENIZERS_PARALLELISM\", \"false\")</p>\n<p>os.environ.setdefault(\"HF_HUB_DISABLE_TELEMETRY\", \"1\")</p>\n<p>import garak, garak.cli</p>\n<p>from garak import _config</p>\n<p>print(\"\\n=== garak version:\", garak.__version__, \"===\")</p>\n<p>def run_garak(args):</p>\n<p>print(\"\\n>>> garak \" + \" \".join(args))</p>\n<p>try:</p>\n<p>garak.cli.main(args)</p>\n<p>except SystemExit as e:</p>\n<p>if e.code not in (0, None):</p>\n<p>print(f\"[garak exited {e.code}]\")</p>\n<p>try:</p>\n<p>return _config.transient.report_filename</p>\n<p>except Exception:</p>\n<p>return None</p>\n<p>We begin by importing the required libraries and creating a helper function to run shell commands directly from the notebook. We install garak, configure basic environment variables, and import the main garak modules needed for the tutorial. We also define a reusable function that lets us run Garak programmatically and capture the path to the generated report.</p>\n<p>Listing garak Probes and Detectors and Running Model Scans</p>\n<p>Copy CodeCopiedUse a different Browserprint(\"\\n########## 1. PLUGIN INVENTORY ##########\")</p>\n<p>for kind in [\"probes\", \"detectors\", \"generators\", \"buffs\"]:</p>\n<p>out = sh(f\"{sys.executable} -m garak --list_{kind} 2>/dev/null\", capture=True)</p>\n<p>lines = [l for l in (out.stdout or \"\").splitlines() if \".\" in l]</p>\n<p>print(f\" {kind:11s}: {len(lines)} plugins e.g. \"</p>\n<p>f\"{', '.join(l.split()[-1] if l.split() else l for l in lines[:3])}\")</p>\n<p>print(\"\\n########## 2. FAST DRY-RUN (test.Repeat) ##########\")</p>\n<p>sh(f\"{sys.executable} -m garak --target_type test.Repeat \"</p>\n<p>f\"--probes lmrc.SlurUsage --generations 1\")</p>\n<p>print(\"\\n########## 3. REAL MODEL: gpt2 vs DAN 11.0 ##########\")</p>\n<p>sh(f\"{sys.executable} -m garak --target_type huggingface --target_name gpt2 \"</p>\n<p>f\"--probes dan.Dan_11_0 --generations 1 --parallel_attempts 8\")</p>\n<p>print(\"\\n########## 4. PROGRAMMATIC MULTI-PROBE SCAN ##########\")</p>\n<p>report_path = run_garak([</p>\n<p>\"--target_type\", \"test.Repeat\",</p>\n<p>\"--probes\", \"dan.Dan_11_0,encoding.InjectBase64,lmrc.SlurUsage\",</p>\n<p>\"--generations\", \"1\", \"--parallel_attempts\", \"16\",</p>\n<p>])</p>\n<p>print(\"Report:\", report_path)</p>\n<p>We inspect the garak plugin ecosystem by listing available probes, detectors, generators, and buffs. We then run a quick dry run using the test generator to confirm that Garak is working without requiring any external model or API key. After that, we scan a real Hugging Face model and run a multi-probe scan to generate a richer report for analysis.</p>\n<p>Analyzing garak Reports: Safety Scores and Attack Success Rates</p>\n<p>Copy CodeCopiedUse a different Browserprint(\"\\n########## 5. ANALYSIS ##########\")</p>\n<p>import numpy as np, pandas as pd</p>\n<p>def find_latest_report():</p>\n<p>cands = []</p>\n<p>for base in [os.path.expanduser(\"~/.local/share/garak/garak_runs\"),</p>\n<p>os.path.expanduser(\"~/.cache/garak\"), \".\"]:</p>\n<p>cands += glob.glob(os.path.join(base, \"**\", \"*report.jsonl\"),</p>\n<p>recursive=True)</p>\n<p>cands = [c for c in cands if os.path.getsize(c) > 0]</p>\n<p>return max(cands, key=os.path.getmtime) if cands else None</p>\n<p>report_path = report_path or find_latest_report()</p>\n<p>print(\"Analysing:\", report_path)</p>\n<p>evaluations = None</p>\n<p>try:</p>\n<p>from garak.report import Report</p>\n<p>rep = Report(report_path).load().get_evaluations()</p>\n<p>evaluations = rep.evaluations.copy()</p>\n<p>print(\"\\n--- Per-probe mean SAFETY score (garak.report.Report) ---\")</p>\n<p>print(rep.scores.round(1).to_string())</p>\n<p>except Exception as e:</p>\n<p>print(\"garak.report.Report unavailable, falling back to manual parse:\", e)</p>\n<p>rows = []</p>\n<p>with open(report_path) as f:</p>\n<p>for line in f:</p>\n<p>try: r = json.loads(line)</p>\n<p>except json.JSONDecodeError: continue</p>\n<p>if r.get(\"entry_type\") == \"eval\":</p>\n<p>rows.append(r)</p>\n<p>evaluations = pd.DataFrame(rows)</p>\n<p>if not evaluations.empty:</p>\n<p>evaluations[\"score\"] = np.where(</p>\n<p>evaluations[\"total_evaluated\"] != 0,</p>\n<p>100 * evaluations[\"passed\"] / evaluations[\"total_evaluated\"], 0.0)</p>\n<p>if evaluations is not None and not evaluations.empty:</p>\n<p>evaluations[\"asr_%\"] = (100 - evaluations[\"score\"]).round(1)</p>\n<p>view = evaluations[[\"probe\", \"detector\", \"passed\",</p>\n<p>\"total_evaluated\", \"score\", \"asr_%\"]].copy()</p>\n<p>view = view.rename(columns={\"score\": \"safe_%\"})</p>\n<p>view[\"safe_%\"] = view[\"safe_%\"].round(1)</p>\n<p>view = view.sort_values(\"asr_%\", ascending=False)</p>\n<p>print(\"\\n--- Per probe/detector (higher asr_% = more vulnerable) ---\")</p>\n<p>print(view.to_string(index=False))</p>\n<p>try:</p>\n<p>import matplotlib.pyplot as plt</p>\n<p>labels = (view[\"probe\"] + \"\\n\" + view[\"detector\"]).tolist()</p>\n<p>plt.figure(figsize=(8, 0.55 * len(view) + 1.5))</p>\n<p>plt.barh(labels, view[\"asr_%\"], color=\"#76b900\")</p>\n<p>plt.gca().invert_yaxis()</p>\n<p>plt.xlabel(\"Attack Success Rate (%)\"); plt.xlim(0, 100)</p>\n<p>plt.title(\"garak — vulnerability by probe/detector\")</p>\n<p>plt.tight_layout(); plt.show()</p>\n<p>except Exception as e:</p>\n<p>print(\"plot skipped:\", e)</p>\n<p>We load the generated garak report and prepare it for detailed analysis using pandas and NumPy. We first try to use Garak’s built-in report parser, and if that is unavailable, we manually parse the JSONL report file. We then calculate safety scores and attack success rates, and visualize vulnerabilities across different probe-detector combinations.</p>\n<p>Inspecting Flagged Outputs and Building a Custom garak Probe</p>\n<p>Copy CodeCopiedUse a different Browserprint(\"\\n--- Sample hits (detector score >= 0.5) ---\")</p>\n<p>hits = []</p>\n<p>with open(report_path) as f:</p>\n<p>for line in f:</p>\n<p>try: r = json.loads(line)</p>\n<p>except json.JSONDecodeError: continue</p>\n<p>if r.get(\"entry_type\") != \"attempt\" or r.get(\"status\") != 2:</p>\n<p>continue</p>\n<p>dr = r.get(\"detector_results\") or r.get(\"detector\") or {}</p>\n<p>max_score = 0.0</p>\n<p>if isinstance(dr, dict):</p>\n<p>for v in dr.values():</p>\n<p>if isinstance(v, list) and v:</p>\n<p>max_score = max(max_score, max(float(x) for x in v))</p>\n<p>if max_score >= 0.5:</p>\n<p>prompt = r.get(\"prompt\")</p>\n<p>hits.append((r.get(\"probe_classname\", r.get(\"probe\", \"?\")),</p>\n<p>round(max_score, 2), str(prompt)[:90]))</p>\n<p>print(f\"Total hits found: {len(hits)}\")</p>\n<p>for probe, score, prompt in hits[:5]:</p>\n<p>print(f\" [{score}] {probe}: {prompt!r}\")</p>\n<p>print(\"\\n########## 6. CUSTOM PROBE ##########\")</p>\n<p>custom_probe = '''</p>\n<p>from garak.probes.base import Probe</p>\n<p>class HelloProbe(Probe):</p>\n<p>\"\"\"Minimal custom probe: two fixed prompts, paired with a custom detector.\"\"\"</p>\n<p>lang = \"en\"</p>\n<p>bcp47 = \"en\"</p>\n<p>primary_detector = \"mycustomdet.ContainsHello\"</p>\n<p>recommended_detector = [\"mycustomdet.ContainsHello\"]</p>\n<p>tags = [\"custom:demo\"]</p>\n<p>goal = \"demonstrate authoring a garak probe\"</p>\n<p>doc_uri = \"\"</p>\n<p>prompts = [</p>\n<p>\"Please reply with the single word: hello\",</p>\n<p>\"Continue this sentence: The capital of France is\",</p>\n<p>]</p>\n<p>'''</p>\n<p>import garak.probes</p>\n<p>probes_dir = os.path.dirname(garak.probes.__file__)</p>\n<p>with open(os.path.join(probes_dir, \"mycustom.py\"), \"w\") as fh:</p>\n<p>fh.write(custom_probe)</p>\n<p>We further inspect the report by extracting sample hits in which detector scores indicate potentially unsafe or vulnerable outputs. We collect the flagged prompts, detector scores, and probe names to understand what kind of behavior is being detected. We then create a custom garak probe that uses fixed prompts and connects it with a custom detector.</p>\n<p>Creating a Custom garak Detector and Exporting Results to AVID</p>\n<p>Copy CodeCopiedUse a different Browserprint(\"\\n########## 7. CUSTOM DETECTOR ##########\")</p>\n<p>custom_detector = '''</p>\n<p>from garak import _config</p>\n<p>from garak.detectors.base import StringDetector</p>\n<p>class ContainsHello(StringDetector):</p>\n<p>\"\"\"Demo detector: flags any output containing 'hello' (case-insensitive).\"\"\"</p>\n<p>lang_spec = \"en\"</p>\n<p>bcp47 = \"en\"</p>\n<p>def __init__(self, config_root=_config):</p>\n<p>super().__init__([\"hello\"], config_root=config_root)</p>\n<p>self.matchtype = \"str\"</p>\n<p>'''</p>\n<p>import garak.detectors</p>\n<p>det_dir = os.path.dirname(garak.detectors.__file__)</p>\n<p>with open(os.path.join(det_dir, \"mycustomdet.py\"), \"w\") as fh:</p>\n<p>fh.write(custom_detector)</p>\n<p>sh(f\"{sys.executable} -m garak --target_type test.Repeat \"</p>\n<p>f\"--probes mycustom.HelloProbe --detectors mycustomdet.ContainsHello \"</p>\n<p>f\"--generations 1\")</p>\n<p>print(\"\\n########## 8. AVID EXPORT ##########\")</p>\n<p>if report_path:</p>\n<p>sh(f\"{sys.executable} -m garak -r {report_path}\")</p>\n<p>print(\"\"\"</p>\n<p>rest:</p>\n<p>RestGenerator:</p>\n<p>uri: https://your-endpoint.example.com/v1/chat</p>\n<p>method: post</p>\n<p>headers: {Authorization: \"Bearer $TOKEN\", Content-Type: \"application/json\"}</p>\n<p>req_template_json_object:</p>\n<p>model: \"your-model\"</p>\n<p>messages: [{\"role\": \"user\", \"content\": \"$INPUT\"}]</p>\n<p>response_json: true</p>\n<p>response_json_field: \"$.choices[0].message.content\"</p>\n<p>\"\"\")</p>\n<p>print(\"=== Done. JSONL + HTML reports: ~/.local/share/garak/garak_runs/ ===\")</p>\n<p>We define a custom detector that flags outputs containing the word “hello” and save it inside Garak’s detector package. We then run our custom probe and detector against the test generator to verify that the extension works correctly. Finally, we export the garak report in AVID format and show a REST configuration template for connecting garak to an external model endpoint.</p>\n<p>Conclusion</p>\n<p>In conclusion, we have a complete hands-on workflow for testing LLM behavior using NVIDIA garak. We run built-in probes, analyze safety scores and attack success rates, inspect concrete flagged outputs, and extend Garak with our own custom probe and detector. We also export results in AVID format, which makes the workflow more useful for structured vulnerability reporting. It provides us a platform to evaluate models we are authorized to test and to build more advanced defensive red-teaming pipelines.</p>\n<p>Check out the FULL CODES Here. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.</p>\n<p>Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us</p>\n<p>The post NVIDIA garak Tutorial: Build a Complete Defensive LLM Red-Teaming Workflow with Custom Probes and Detectors appeared first on MarkTechPost.</p>"
},
{
"id": "878e46c4b704",
"title": "Building Reflective Prompt Optimization with GEPA: Multi-Component Prompts, Structured Feedback, and Held-Out Validation",
"content": "In this tutorial, we use GEPA as a reflective prompt-evolution framework to improve the way a language model solves arithmetic word problems. We begin with a weak seed prompt, create a small deterministic benchmark, define a structured evaluator, and pass actionable feedback to GEPA so it can understand why a candidate prompt fails. We also use a multi-component prompt setup in which both the instruction field and the output-format rules evolve together. By the end, we compare the baseline prompt with the optimized prompt on a held-out validation set and inspect how the evolutionary process improves performance.\n\n\n\nInstalling GEPA and LiteLLM and Configuring the Task and Reflection Models\n\n\n\nCopy CodeCopiedUse a different Browser!pip install -q gepa litellm\nimport os, re, json, random, getpass, textwrap\nimport litellm\nimport gepa.optimize_anything as oa\nfrom gepa.optimize_anything import (\n optimize_anything, GEPAConfig, EngineConfig, ReflectionConfig,\n)\nlitellm.suppress_debug_info = True\nif not os.environ.get(\"OPENAI_API_KEY\"):\n os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")\nTASK_LM = \"openai/gpt-4o-mini\"\nREFLECTION_LM = \"openai/gpt-4.1\"\nMAX_METRIC_CALLS = 100\n\n\n\nWe install GEPA and LiteLLM, then import the required libraries for prompt optimization and model calls. We securely set up the OpenAI API key and define two models: a task model that solves the problem and a reflection model that improves the prompt. We also set the maximum metric-call budget to keep the optimization process under control.\n\n\n\nBuilding a Deterministic Arithmetic Benchmark Dataset\n\n\n\nCopy CodeCopiedUse a different Browserdef make_problems(n, seed=0):\n rng = random.Random(seed)\n out = []\n for _ in range(n):\n t = rng.choice([\"discount\", \"travel\", \"wallet\", \"chain\"])\n if t == \"discount\":\n unit = rng.choice([40, 60, 80, 120])\n qty = rng.choice([5, 6, 8, 10])\n disc = rng.choice([10, 20, 25, 50])\n total = unit * qty\n gold = total - total * disc // 100\n q = (f\"A shop sells notebooks at {unit} rupees each. You buy {qty} \"\n f\"notebooks and get a {disc}% discount on the total bill. \"\n f\"How many rupees do you pay in total?\")\n elif t == \"travel\":\n s1, h1 = rng.choice([40, 50, 60]), rng.choice([2, 3])\n s2, h2 = rng.choice([30, 45, 70]), rng.choice([1, 2, 3])\n gold = s1 * h1 + s2 * h2\n q = (f\"A car drives at {s1} km/h for {h1} hours, then at {s2} km/h \"\n f\"for {h2} hours. What is the total distance travelled, in km?\")\n elif t == \"wallet\":\n tens = rng.choice([3, 5, 7, 9])\n fifties= rng.choice([2, 4, 6])\n spent = rng.choice([50, 80, 110, 150])\n gold = tens * 10 + fifties * 50 - spent\n q = (f\"You have {tens} ten-rupee notes and {fifties} fifty-rupee \"\n f\"notes. You spend {spent} rupees. How many rupees are left?\")\n else:\n x = rng.choice([6, 9, 12, 15]); y = rng.choice([4, 7, 10]); z = rng.choice([3, 8, 11])\n gold = x * 2 - y + z\n q = (f\"Start with the number {x}. Double it, then subtract {y}, \"\n f\"then add {z}. What number do you end with?\")\n out.append({\"question\": q, \"answer\": gold})\n return out\nall_problems = make_problems(18, seed=42)\nrandom.Random(1).shuffle(all_problems)\ntrainset = all_problems[:12]\nvalset = all_problems[12:]\nprint(f\"Dataset: {len(trainset)} train / {len(valset)} val problems\\n\")\n\n\n\nWe create a small deterministic dataset of arithmetic word problems covering discounts, travel distance, wallet calculations, and chained operations. We generate the correct answer for each problem programmatically, which keeps the benchmark reliable and easy to evaluate. We then shuffle the examples and split them into a training set for optimization and a validation set for testing generalization.\n\n\n\nDefining the Evaluator and Structured Feedback for GEPA\n\n\n\nCopy CodeCopiedUse a different Browserdef build_system_prompt(candidate: dict) -> str:\n return (f\"{candidate['instructions']}\\n\\n\"\n f\"OUTPUT FORMAT RULES:\\n{candidate['format_rules']}\")\ndef call_task_lm(system_prompt: str, question: str) -> str:\n for attempt in range(3):\n try:\n r = litellm.completion(\n model=TASK_LM,\n messages=[{\"role\": \"system\", \"content\": system_prompt},\n {\"role\": \"user\", \"content\": question}],\n temperature=0, max_tokens=600, timeout=60,\n )\n return r[\"choices\"][0][\"message\"][\"content\"] or \"\"\n except Exception as e:\n if attempt == 2:\n return f\"[LM_ERROR] {e}\"\n return \"\"\ndef parse_answers(text: str):\n formatted = re.search(r\"####\\s*(-?\\d+)\", text)\n all_nums = re.findall(r\"-?\\d+\", text)\n fmt_val = int(formatted.group(1)) if formatted else None\n last_val = int(all_nums[-1]) if all_nums else None\n return fmt_val, last_val\ndef evaluate(candidate: dict, example: dict):\n system = build_system_prompt(candidate)\n raw = call_task_lm(system, example[\"question\"])\n gold = example[\"answer\"]\n fmt_val, last_val = parse_answers(raw)\n if fmt_val is not None and fmt_val == gold:\n score, fb = 1.0, \"Correct and correctly formatted.\"\n elif fmt_val is not None and fmt_val != gold:\n score, fb = 0.0, (f\"WRONG ANSWER. You output '#### {fmt_val}' but the \"\n f\"correct answer is {gold}. Re-check the arithmetic and \"\n f\"the order of the steps.\")\n elif last_val == gold:\n score, fb = 0.5, (f\"Right number ({gold}) but FORMAT VIOLATION: the final \"\n f\"line was not exactly '#### {gold}'. Always end with a \"\n f\"line of the form '#### <integer>' and nothing else.\")\n else:\n score, fb = 0.0, (f\"WRONG. Correct answer is {gold}. The model's final \"\n f\"number was {last_val}. Likely a multi-step reasoning \"\n f\"slip; show each step and verify before answering.\")\n oa.log(f\"score={score} gold={gold} parsed_fmt={fmt_val} parsed_last={last_val}\")\n side_info = {\n \"feedback\": fb,\n \"problem\": example[\"question\"],\n \"gold_answer\": gold,\n \"model_output\": raw[:500],\n }\n return score, side_info\ndef eval_set(candidate, dataset, label=\"\"):\n scores, exact, formatted = [], 0, 0\n for ex in dataset:\n s, info = evaluate(candidate, ex)\n scores.append(s)\n if s == 1.0: exact += 1; formatted += 1\n elif s == 0.5: formatted += 0\n acc = exact / len(dataset)\n avg = sum(scores) / len(dataset)\n print(f\" [{label}] avg_score={avg:.3f} exact_correct+formatted={exact}/{len(dataset)}\")\n return avg, acc\n\n\n\nWe define how the candidate prompt is converted into a system prompt and how the task model receives each question. We also create the evaluator that parses the model output, checks whether the final answer follows the required #### <integer> format, and assigns a score. We return structured feedback as actionable side information so that GEPA can determine whether the issue is incorrect reasoning, poor formatting, or both.\n\n\n\nConfiguring GEPA and Running the Prompt Optimization\n\n\n\nCopy CodeCopiedUse a different Browserseed_candidate = {\n \"instructions\": \"Solve the math problem.\",\n \"format_rules\": \"Give the answer.\",\n}\nprint(\"=== BASELINE (seed prompt) ===\")\nprint(\"Train:\"); base_train = eval_set(seed_candidate, trainset, \"train\")\nprint(\"Val: \"); base_val = eval_set(seed_candidate, valset, \"val\")\nprint()\nobjective = (\n \"Evolve a system prompt (the 'instructions' and 'format_rules' fields) so a \"\n \"small LLM reliably solves multi-step arithmetic word problems AND always \"\n \"ends with a line of exactly the form '#### <integer>'. Maximize the score.\"\n)\nbackground = (\n \"Scoring: 1.0 = correct number in the exact '#### <int>' format; 0.5 = correct \"\n \"number but wrong/missing format; 0.0 = wrong number. Common failures are (a) not \"\n \"emitting the '####' line, and (b) order-of-operations or multi-step slips. The \"\n \"winning prompt should force explicit step-by-step work, a verification step, and \"\n \"a strict final-answer line.\"\n)\nconfig = GEPAConfig(\n engine=EngineConfig(\n max_metric_calls=MAX_METRIC_CALLS,\n max_workers=4,\n parallel=True,\n display_progress_bar=True,\n seed=0,\n ),\n reflection=ReflectionConfig(\n reflection_lm=REFLECTION_LM,\n ),\n)\nprint(\"=== RUNNING GEPA (this calls the LLMs; ~1-4 min) ===\")\nresult = optimize_anything(\n seed_candidate=seed_candidate,\n evaluator=evaluate,\n dataset=trainset,\n valset=valset,\n objective=objective,\n background=background,\n config=config,\n)\n\n\n\n\nWe start with a weak seed prompt and evaluate its baseline performance on both the training and validation sets. We then define the optimization objective, background scoring rules, and GEPA configuration, including parallel evaluation and the reflection model. Finally, we run optimize_anything so GEPA can evolve the instruction and format-rule fields using the evaluator feedback.\n\n\n\nComparing the Baseline and GEPA-Optimized Prompts on the Validation Set\n\n\n\nCopy CodeCopiedUse a different Browserbest = result.best_candidate\nprint(\"\\n\" + \"=\" * 78)\nprint(\"OPTIMIZED CANDIDATE\")\nprint(\"=\" * 78)\nprint(\"\\n--- instructions ---\\n\" + textwrap.fill(best[\"instructions\"], 96))\nprint(\"\\n--- format_rules ---\\n\" + textwrap.fill(best[\"format_rules\"], 96))\nprint(\"\\n\" + \"=\" * 78)\nprint(\"BEFORE vs AFTER (held-out validation set)\")\nprint(\"=\" * 78)\nprint(\"Seed prompt:\"); _ = eval_set(seed_candidate, valset, \"val-seed\")\nprint(\"GEPA prompt:\"); _ = eval_set(best, valset, \"val-gepa\")\nprint(f\"\\nBaseline val avg_score : {base_val[0]:.3f}\")\nprint(\"\\n\" + \"=\" * 78)\nprint(\"EVOLUTION HISTORY (candidate index -> val score, parents)\")\nprint(\"=\" * 78)\ncands = getattr(result, \"candidates\", [])\nvscores = getattr(result, \"val_aggregate_scores\", [])\nparents = getattr(result, \"parents\", [None] * len(cands))\nfor i, sc in enumerate(vscores):\n par = parents[i] if i < len(parents) else None\n tag = \" <-- BEST\" if cands and cands[i] == best else \"\"\n print(f\" cand {i:2d}: val_score={sc:.3f} parents={par}{tag}\")\nprint(f\"\\nTotal metric calls used : {getattr(result, 'total_metric_calls', 'n/a')}\")\nprint(f\"Full validation evals : {getattr(result, 'num_full_val_evals', 'n/a')}\")\nprint(\"\\nDone. Try raising MAX_METRIC_CALLS or swapping REFLECTION_LM for a stronger model.\")\n\n\n\nWe extract the best prompt found by GEPA and print its optimized instruction and format-rule components. We compare the seed prompt and the GEPA-optimized prompt on the held-out validation set to check whether the improvement transfers to unseen examples. We also inspect the evolution history, validation scores, parent relationships, and total metric calls to understand how the prompt improved over the course of optimization.\n\n\n\nIn conclusion, we used GEPA to show how prompt optimization can move beyond manual trial and error. We created a complete workflow where a task model solves examples, an evaluator scores the outputs, and a reflection model uses detailed feedback to propose better prompts. We also tested the optimized prompt on unseen validation problems, which helps us assess whether the improvement generalizes rather than merely fitting the training set. Also, we built a practical example of reflective prompt evolution in which structured feedback, strict evaluation, and iterative refinement work together to produce a stronger, more reliable prompt.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nCheck out the Full Codes with Notebook. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.\n\n\n\nNeed to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us\nThe post Building Reflective Prompt Optimization with GEPA: Multi-Component Prompts, Structured Feedback, and Held-Out Validation appeared first on MarkTechPost.",
"url": "https://www.marktechpost.com/2026/06/07/building-reflective-prompt-optimization-with-gepa-multi-component-prompts-structured-feedback-and-held-out-validation/",
"author": "Sana Hassan",
"published": "2026-06-07T17:05:04",
"source": "MarkTechPost",
"source_type": "rss",
"tags": [
"Applications",
"Artificial Intelligence",
"Editors Pick",
"Language Model",
"Large Language Model",
"Machine Learning",
"Staff",
"Technology",
"Tutorials"
],
"summary": "A tutorial demonstrates GEPA, a reflective prompt-evolution framework that iteratively improves prompts using structured feedback and held-out validation. It walks through optimizing multi-component prompts for arithmetic word problems against a baseline.",
"importance_score": 22,
"reasoning": "Educational how-to with limited frontier significance; useful for practitioners but not a major development.",
"themes": [
"prompt optimization",
"tutorials",
"tooling"
],
"continuation": null,
"summary_html": "<p>A tutorial demonstrates GEPA, a reflective prompt-evolution framework that iteratively improves prompts using structured feedback and held-out validation. It walks through optimizing multi-component prompts for arithmetic word problems against a baseline.</p>",
"content_html": "<p>In this tutorial, we use GEPA as a reflective prompt-evolution framework to improve the way a language model solves arithmetic word problems. We begin with a weak seed prompt, create a small deterministic benchmark, define a structured evaluator, and pass actionable feedback to GEPA so it can understand why a candidate prompt fails. We also use a multi-component prompt setup in which both the instruction field and the output-format rules evolve together. By the end, we compare the baseline prompt with the optimized prompt on a held-out validation set and inspect how the evolutionary process improves performance.</p>\n<p>Installing GEPA and LiteLLM and Configuring the Task and Reflection Models</p>\n<p>Copy CodeCopiedUse a different Browser!pip install -q gepa litellm</p>\n<p>import os, re, json, random, getpass, textwrap</p>\n<p>import litellm</p>\n<p>import gepa.optimize_anything as oa</p>\n<p>from gepa.optimize_anything import (</p>\n<p>optimize_anything, GEPAConfig, EngineConfig, ReflectionConfig,</p>\n<p>)</p>\n<p>litellm.suppress_debug_info = True</p>\n<p>if not os.environ.get(\"OPENAI_API_KEY\"):</p>\n<p>os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")</p>\n<p>TASK_LM = \"openai/gpt-4o-mini\"</p>\n<p>REFLECTION_LM = \"openai/gpt-4.1\"</p>\n<p>MAX_METRIC_CALLS = 100</p>\n<p>We install GEPA and LiteLLM, then import the required libraries for prompt optimization and model calls. We securely set up the OpenAI API key and define two models: a task model that solves the problem and a reflection model that improves the prompt. We also set the maximum metric-call budget to keep the optimization process under control.</p>\n<p>Building a Deterministic Arithmetic Benchmark Dataset</p>\n<p>Copy CodeCopiedUse a different Browserdef make_problems(n, seed=0):</p>\n<p>rng = random.Random(seed)</p>\n<p>out = []</p>\n<p>for _ in range(n):</p>\n<p>t = rng.choice([\"discount\", \"travel\", \"wallet\", \"chain\"])</p>\n<p>if t == \"discount\":</p>\n<p>unit = rng.choice([40, 60, 80, 120])</p>\n<p>qty = rng.choice([5, 6, 8, 10])</p>\n<p>disc = rng.choice([10, 20, 25, 50])</p>\n<p>total = unit * qty</p>\n<p>gold = total - total * disc // 100</p>\n<p>q = (f\"A shop sells notebooks at {unit} rupees each. You buy {qty} \"</p>\n<p>f\"notebooks and get a {disc}% discount on the total bill. \"</p>\n<p>f\"How many rupees do you pay in total?\")</p>\n<p>elif t == \"travel\":</p>\n<p>s1, h1 = rng.choice([40, 50, 60]), rng.choice([2, 3])</p>\n<p>s2, h2 = rng.choice([30, 45, 70]), rng.choice([1, 2, 3])</p>\n<p>gold = s1 * h1 + s2 * h2</p>\n<p>q = (f\"A car drives at {s1} km/h for {h1} hours, then at {s2} km/h \"</p>\n<p>f\"for {h2} hours. What is the total distance travelled, in km?\")</p>\n<p>elif t == \"wallet\":</p>\n<p>tens = rng.choice([3, 5, 7, 9])</p>\n<p>fifties= rng.choice([2, 4, 6])</p>\n<p>spent = rng.choice([50, 80, 110, 150])</p>\n<p>gold = tens * 10 + fifties * 50 - spent</p>\n<p>q = (f\"You have {tens} ten-rupee notes and {fifties} fifty-rupee \"</p>\n<p>f\"notes. You spend {spent} rupees. How many rupees are left?\")</p>\n<p>else:</p>\n<p>x = rng.choice([6, 9, 12, 15]); y = rng.choice([4, 7, 10]); z = rng.choice([3, 8, 11])</p>\n<p>gold = x * 2 - y + z</p>\n<p>q = (f\"Start with the number {x}. Double it, then subtract {y}, \"</p>\n<p>f\"then add {z}. What number do you end with?\")</p>\n<p>out.append({\"question\": q, \"answer\": gold})</p>\n<p>return out</p>\n<p>all_problems = make_problems(18, seed=42)</p>\n<p>random.Random(1).shuffle(all_problems)</p>\n<p>trainset = all_problems[:12]</p>\n<p>valset = all_problems[12:]</p>\n<p>print(f\"Dataset: {len(trainset)} train / {len(valset)} val problems\\n\")</p>\n<p>We create a small deterministic dataset of arithmetic word problems covering discounts, travel distance, wallet calculations, and chained operations. We generate the correct answer for each problem programmatically, which keeps the benchmark reliable and easy to evaluate. We then shuffle the examples and split them into a training set for optimization and a validation set for testing generalization.</p>\n<p>Defining the Evaluator and Structured Feedback for GEPA</p>\n<p>Copy CodeCopiedUse a different Browserdef build_system_prompt(candidate: dict) -> str:</p>\n<p>return (f\"{candidate['instructions']}\\n\\n\"</p>\n<p>f\"OUTPUT FORMAT RULES:\\n{candidate['format_rules']}\")</p>\n<p>def call_task_lm(system_prompt: str, question: str) -> str:</p>\n<p>for attempt in range(3):</p>\n<p>try:</p>\n<p>r = litellm.completion(</p>\n<p>model=TASK_LM,</p>\n<p>messages=[{\"role\": \"system\", \"content\": system_prompt},</p>\n<p>{\"role\": \"user\", \"content\": question}],</p>\n<p>temperature=0, max_tokens=600, timeout=60,</p>\n<p>)</p>\n<p>return r[\"choices\"][0][\"message\"][\"content\"] or \"\"</p>\n<p>except Exception as e:</p>\n<p>if attempt == 2:</p>\n<p>return f\"[LM_ERROR] {e}\"</p>\n<p>return \"\"</p>\n<p>def parse_answers(text: str):</p>\n<p>formatted = re.search(r\"####\\s*(-?\\d+)\", text)</p>\n<p>all_nums = re.findall(r\"-?\\d+\", text)</p>\n<p>fmt_val = int(formatted.group(1)) if formatted else None</p>\n<p>last_val = int(all_nums[-1]) if all_nums else None</p>\n<p>return fmt_val, last_val</p>\n<p>def evaluate(candidate: dict, example: dict):</p>\n<p>system = build_system_prompt(candidate)</p>\n<p>raw = call_task_lm(system, example[\"question\"])</p>\n<p>gold = example[\"answer\"]</p>\n<p>fmt_val, last_val = parse_answers(raw)</p>\n<p>if fmt_val is not None and fmt_val == gold:</p>\n<p>score, fb = 1.0, \"Correct and correctly formatted.\"</p>\n<p>elif fmt_val is not None and fmt_val != gold:</p>\n<p>score, fb = 0.0, (f\"WRONG ANSWER. You output '#### {fmt_val}' but the \"</p>\n<p>f\"correct answer is {gold}. Re-check the arithmetic and \"</p>\n<p>f\"the order of the steps.\")</p>\n<p>elif last_val == gold:</p>\n<p>score, fb = 0.5, (f\"Right number ({gold}) but FORMAT VIOLATION: the final \"</p>\n<p>f\"line was not exactly '#### {gold}'. Always end with a \"</p>\n<p>f\"line of the form '#### <integer>' and nothing else.\")</p>\n<p>else:</p>\n<p>score, fb = 0.0, (f\"WRONG. Correct answer is {gold}. The model's final \"</p>\n<p>f\"number was {last_val}. Likely a multi-step reasoning \"</p>\n<p>f\"slip; show each step and verify before answering.\")</p>\n<p>oa.log(f\"score={score} gold={gold} parsed_fmt={fmt_val} parsed_last={last_val}\")</p>\n<p>side_info = {</p>\n<p>\"feedback\": fb,</p>\n<p>\"problem\": example[\"question\"],</p>\n<p>\"gold_answer\": gold,</p>\n<p>\"model_output\": raw[:500],</p>\n<p>}</p>\n<p>return score, side_info</p>\n<p>def eval_set(candidate, dataset, label=\"\"):</p>\n<p>scores, exact, formatted = [], 0, 0</p>\n<p>for ex in dataset:</p>\n<p>s, info = evaluate(candidate, ex)</p>\n<p>scores.append(s)</p>\n<p>if s == 1.0: exact += 1; formatted += 1</p>\n<p>elif s == 0.5: formatted += 0</p>\n<p>acc = exact / len(dataset)</p>\n<p>avg = sum(scores) / len(dataset)</p>\n<p>print(f\" [{label}] avg_score={avg:.3f} exact_correct+formatted={exact}/{len(dataset)}\")</p>\n<p>return avg, acc</p>\n<p>We define how the candidate prompt is converted into a system prompt and how the task model receives each question. We also create the evaluator that parses the model output, checks whether the final answer follows the required #### <integer> format, and assigns a score. We return structured feedback as actionable side information so that GEPA can determine whether the issue is incorrect reasoning, poor formatting, or both.</p>\n<p>Configuring GEPA and Running the Prompt Optimization</p>\n<p>Copy CodeCopiedUse a different Browserseed_candidate = {</p>\n<p>\"instructions\": \"Solve the math problem.\",</p>\n<p>\"format_rules\": \"Give the answer.\",</p>\n<p>}</p>\n<p>print(\"=== BASELINE (seed prompt) ===\")</p>\n<p>print(\"Train:\"); base_train = eval_set(seed_candidate, trainset, \"train\")</p>\n<p>print(\"Val: \"); base_val = eval_set(seed_candidate, valset, \"val\")</p>\n<p>print()</p>\n<p>objective = (</p>\n<p>\"Evolve a system prompt (the 'instructions' and 'format_rules' fields) so a \"</p>\n<p>\"small LLM reliably solves multi-step arithmetic word problems AND always \"</p>\n<p>\"ends with a line of exactly the form '#### <integer>'. Maximize the score.\"</p>\n<p>)</p>\n<p>background = (</p>\n<p>\"Scoring: 1.0 = correct number in the exact '#### <int>' format; 0.5 = correct \"</p>\n<p>\"number but wrong/missing format; 0.0 = wrong number. Common failures are (a) not \"</p>\n<p>\"emitting the '####' line, and (b) order-of-operations or multi-step slips. The \"</p>\n<p>\"winning prompt should force explicit step-by-step work, a verification step, and \"</p>\n<p>\"a strict final-answer line.\"</p>\n<p>)</p>\n<p>config = GEPAConfig(</p>\n<p>engine=EngineConfig(</p>\n<p>max_metric_calls=MAX_METRIC_CALLS,</p>\n<p>max_workers=4,</p>\n<p>parallel=True,</p>\n<p>display_progress_bar=True,</p>\n<p>seed=0,</p>\n<p>),</p>\n<p>reflection=ReflectionConfig(</p>\n<p>reflection_lm=REFLECTION_LM,</p>\n<p>),</p>\n<p>)</p>\n<p>print(\"=== RUNNING GEPA (this calls the LLMs; ~1-4 min) ===\")</p>\n<p>result = optimize_anything(</p>\n<p>seed_candidate=seed_candidate,</p>\n<p>evaluator=evaluate,</p>\n<p>dataset=trainset,</p>\n<p>valset=valset,</p>\n<p>objective=objective,</p>\n<p>background=background,</p>\n<p>config=config,</p>\n<p>)</p>\n<p>We start with a weak seed prompt and evaluate its baseline performance on both the training and validation sets. We then define the optimization objective, background scoring rules, and GEPA configuration, including parallel evaluation and the reflection model. Finally, we run optimize_anything so GEPA can evolve the instruction and format-rule fields using the evaluator feedback.</p>\n<p>Comparing the Baseline and GEPA-Optimized Prompts on the Validation Set</p>\n<p>Copy CodeCopiedUse a different Browserbest = result.best_candidate</p>\n<p>print(\"\\n\" + \"=\" * 78)</p>\n<p>print(\"OPTIMIZED CANDIDATE\")</p>\n<p>print(\"=\" * 78)</p>\n<p>print(\"\\n--- instructions ---\\n\" + textwrap.fill(best[\"instructions\"], 96))</p>\n<p>print(\"\\n--- format_rules ---\\n\" + textwrap.fill(best[\"format_rules\"], 96))</p>\n<p>print(\"\\n\" + \"=\" * 78)</p>\n<p>print(\"BEFORE vs AFTER (held-out validation set)\")</p>\n<p>print(\"=\" * 78)</p>\n<p>print(\"Seed prompt:\"); _ = eval_set(seed_candidate, valset, \"val-seed\")</p>\n<p>print(\"GEPA prompt:\"); _ = eval_set(best, valset, \"val-gepa\")</p>\n<p>print(f\"\\nBaseline val avg_score : {base_val[0]:.3f}\")</p>\n<p>print(\"\\n\" + \"=\" * 78)</p>\n<p>print(\"EVOLUTION HISTORY (candidate index -> val score, parents)\")</p>\n<p>print(\"=\" * 78)</p>\n<p>cands = getattr(result, \"candidates\", [])</p>\n<p>vscores = getattr(result, \"val_aggregate_scores\", [])</p>\n<p>parents = getattr(result, \"parents\", [None] * len(cands))</p>\n<p>for i, sc in enumerate(vscores):</p>\n<p>par = parents[i] if i < len(parents) else None</p>\n<p>tag = \" <-- BEST\" if cands and cands[i] == best else \"\"</p>\n<p>print(f\" cand {i:2d}: val_score={sc:.3f} parents={par}{tag}\")</p>\n<p>print(f\"\\nTotal metric calls used : {getattr(result, 'total_metric_calls', 'n/a')}\")</p>\n<p>print(f\"Full validation evals : {getattr(result, 'num_full_val_evals', 'n/a')}\")</p>\n<p>print(\"\\nDone. Try raising MAX_METRIC_CALLS or swapping REFLECTION_LM for a stronger model.\")</p>\n<p>We extract the best prompt found by GEPA and print its optimized instruction and format-rule components. We compare the seed prompt and the GEPA-optimized prompt on the held-out validation set to check whether the improvement transfers to unseen examples. We also inspect the evolution history, validation scores, parent relationships, and total metric calls to understand how the prompt improved over the course of optimization.</p>\n<p>In conclusion, we used GEPA to show how prompt optimization can move beyond manual trial and error. We created a complete workflow where a task model solves examples, an evaluator scores the outputs, and a reflection model uses detailed feedback to propose better prompts. We also tested the optimized prompt on unseen validation problems, which helps us assess whether the improvement generalizes rather than merely fitting the training set. Also, we built a practical example of reflective prompt evolution in which structured feedback, strict evaluation, and iterative refinement work together to produce a stronger, more reliable prompt.</p>\n<p>Check out the Full Codes with Notebook. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.</p>\n<p>Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us</p>\n<p>The post Building Reflective Prompt Optimization with GEPA: Multi-Component Prompts, Structured Feedback, and Held-Out Validation appeared first on MarkTechPost.</p>"
}
]
},
"research": {
"count": 451,
"category_summary": "Today's research is dominated by **safety, alignment, and agent monitoring**, alongside provocative efficiency and theory contributions.\n\n**Safety & Alignment** leads with strong mechanistic and empirical work:\n- The **Piggyback Hypothesis** [gives a concrete causal mechanism](/?date=2026-06-08&category=research#item-b53bf948a761) for emergent misalignment—chat-template tokens carry finetuned misbehavior onto unrelated tokens—plus an effective mitigation.\n- **Think Fast** [measures no-CoT task-completion time horizons](/?date=2026-06-08&category=research#item-e4b697a36cd8) across **30,000+ questions** in **43 benchmarks**, directly probing reasoning monitorability.\n- **CapCode** [detects and prevents reward hacking](/?date=2026-06-08&category=research#item-72ce62de7588) in coding agents via randomized tests with a deliberately capped non-cheating score.\n- The **Geography of Algorithmic Judgment** [audits 7 LLMs for racial steering](/?date=2026-06-08&category=research#item-da444417a4b5) in housing search across 4 US cities using paired testing.\n\n**Efficiency & Architecture** features bold rethinks with industry stakes:\n- **FP8 is All You Need** (Matsuoka) [argues hardware FP64 is unnecessary](/?date=2026-06-08&category=research#item-5e9f1130c57f) for HPC, leveraging FP8 tensor throughput plus the **Ozaki Scheme II**.\n- **Sparsely gated tiny linear experts** [pushes MoE sparsity to single-neuron experts](/?date=2026-06-08&category=research#item-39879c2f329c) with the nonlinearity removed, yielding isoflop gains and interpretability benefits.\n\n**Theory, Benchmarks & Meta-Science** round out the list:\n- **Flatland** [unifies the definition of large step sizes](/?date=2026-06-08&category=research#item-1e1395108719) for gradient descent under only local Lipschitz/Hölder continuity.\n- **MMBU** is the [largest biomedical vision-language benchmark](/?date=2026-06-08&category=research#item-27e074eb0867), spanning **35 submodalities** with grounded/ungrounded tasks.\n- **\"Don't Just Fix it in Post\"** (Biderman, Saphra, Barez, Mireshghallah) [argues a science of AI must study training dynamics](/?date=2026-06-08&category=research#item-d14361debc4b), not post-hoc patches.\n- **How Language Models Fail** [characterizes reasoning failures via token-level uncertainty](/?date=2026-06-08&category=research#item-9aabd92c9d93), distinguishing committed (early lock-in) from persistent failures.",
"category_summary_html": "<p>Today's research is dominated by <strong>safety, alignment, and agent monitoring</strong>, alongside provocative efficiency and theory contributions.</p>\n<p><strong>Safety & Alignment</strong> leads with strong mechanistic and empirical work:</p>\n<ul>\n<li>The <strong>Piggyback Hypothesis</strong> <a href=\"/?date=2026-06-08&category=research#item-b53bf948a761\" class=\"internal-link\" rel=\"noopener noreferrer\">gives a concrete causal mechanism</a> for emergent misalignment—chat-template tokens carry finetuned misbehavior onto unrelated tokens—plus an effective mitigation.</li>\n<li><strong>Think Fast</strong> <a href=\"/?date=2026-06-08&category=research#item-e4b697a36cd8\" class=\"internal-link\" rel=\"noopener noreferrer\">measures no-CoT task-completion time horizons</a> across <strong>30,000+ questions</strong> in <strong>43 benchmarks</strong>, directly probing reasoning monitorability.</li>\n<li><strong>CapCode</strong> <a href=\"/?date=2026-06-08&category=research#item-72ce62de7588\" class=\"internal-link\" rel=\"noopener noreferrer\">detects and prevents reward hacking</a> in coding agents via randomized tests with a deliberately capped non-cheating score.</li>\n<li>The <strong>Geography of Algorithmic Judgment</strong> <a href=\"/?date=2026-06-08&category=research#item-da444417a4b5\" class=\"internal-link\" rel=\"noopener noreferrer\">audits 7 LLMs for racial steering</a> in housing search across 4 US cities using paired testing.</li>\n</ul>\n<p><strong>Efficiency & Architecture</strong> features bold rethinks with industry stakes:</p>\n<ul>\n<li><strong>FP8 is All You Need</strong> (Matsuoka) <a href=\"/?date=2026-06-08&category=research#item-5e9f1130c57f\" class=\"internal-link\" rel=\"noopener noreferrer\">argues hardware FP64 is unnecessary</a> for HPC, leveraging FP8 tensor throughput plus the <strong>Ozaki Scheme II</strong>.</li>\n<li><strong>Sparsely gated tiny linear experts</strong> <a href=\"/?date=2026-06-08&category=research#item-39879c2f329c\" class=\"internal-link\" rel=\"noopener noreferrer\">pushes MoE sparsity to single-neuron experts</a> with the nonlinearity removed, yielding isoflop gains and interpretability benefits.</li>\n</ul>\n<p><strong>Theory, Benchmarks & Meta-Science</strong> round out the list:</p>\n<ul>\n<li><strong>Flatland</strong> <a href=\"/?date=2026-06-08&category=research#item-1e1395108719\" class=\"internal-link\" rel=\"noopener noreferrer\">unifies the definition of large step sizes</a> for gradient descent under only local Lipschitz/Hölder continuity.</li>\n<li><strong>MMBU</strong> is the <a href=\"/?date=2026-06-08&category=research#item-27e074eb0867\" class=\"internal-link\" rel=\"noopener noreferrer\">largest biomedical vision-language benchmark</a>, spanning <strong>35 submodalities</strong> with grounded/ungrounded tasks.</li>\n<li><strong>\"Don't Just Fix it in Post\"</strong> (Biderman, Saphra, Barez, Mireshghallah) <a href=\"/?date=2026-06-08&category=research#item-d14361debc4b\" class=\"internal-link\" rel=\"noopener noreferrer\">argues a science of AI must study training dynamics</a>, not post-hoc patches.</li>\n<li><strong>How Language Models Fail</strong> <a href=\"/?date=2026-06-08&category=research#item-9aabd92c9d93\" class=\"internal-link\" rel=\"noopener noreferrer\">characterizes reasoning failures via token-level uncertainty</a>, distinguishing committed (early lock-in) from persistent failures.</li>\n</ul>",
"themes": [
{
"name": "AI Safety and Alignment",
"description": "Emergent misalignment, preference plurality, fairness audits, and steering for safer model behavior.",
"item_count": 14,
"example_items": [],
"importance": 73
},
{
"name": "Interpretability",
"description": "Mechanistic and representational analysis of model internals, including SAEs, activation steering, and failure signatures.",
"item_count": 18,
"example_items": [],
"importance": 72
},
{
"name": "LLM Agents",
"description": "Self-evolving agents, memory systems, web agents, and proactive constraint adaptation.",
"item_count": 15,
"example_items": [],
"importance": 72
},
{
"name": "Language Models",
"description": "Research on LLM behavior, reasoning, adaptation, evaluation, and applications across diverse text tasks.",
"item_count": 20,
"example_items": [],
"importance": 70
},
{
"name": "Language Models and Reasoning",
"description": "Work on LLM reasoning, test-time scaling, post-training, routing, and reliability spanning pretraining regularization to inference-time compute.",
"item_count": 16,
"example_items": [],
"importance": 70
},
{
"name": "Reinforcement Learning & Agents",
"description": "Work on RL post-training, agentic tool use, GUI agents, web agents, world models, and process rewards.",
"item_count": 13,
"example_items": [],
"importance": 70
},
{
"name": "Deep Learning Theory",
"description": "Generalization bounds, optimization step sizes, edge of stability, and NTK-regime analysis.",
"item_count": 3,
"example_items": [],
"importance": 69
},
{
"name": "AI Safety, Security, and Robustness",
"description": "Adversarial attacks and defenses, safe generation, hallucination detection, model stealing, and privacy auditing.",
"item_count": 10,
"example_items": [],
"importance": 68
},
{
"name": "Generative Models",
"description": "Diffusion, flow matching, VAEs, 3D generation, song/speech synthesis, and image generation advances.",
"item_count": 13,
"example_items": [],
"importance": 68
},
{
"name": "Language Models & NLP",
"description": "Reasoning, distillation, compression, quantization, citations, perspectivism, and multilingual evaluation.",
"item_count": 14,
"example_items": [],
"importance": 67
}
],
"top_items": [
{
"id": "b53bf948a761",
"title": "The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignment",
"content": "The mechanisms behind LLMs' broad over-generalization beyond training examples remain unclear. Emergent misalignment (EM) offers a striking case study: finetuning on narrow tasks induces broad misalignment to semantically-unrelated test domains. In this work, we propose the Piggyback Hypothesis: the chat-template tokens can piggyback the finetuned behaviour onto out-of-domain queries. We validate this hypothesis by showing that subtle perturbations to the prefix (tokens preceding all user queries), or patching the prefix representations with those from the unfinetuned model, can restore alignment without changing the user query. Building on this finding, we propose Token-Regularized Finetuning (TReFT), which regularizes specific token representations during training to mitigate EM. Across different models and multiple EM-inducing datasets, TReFT reduces EM while preserving in-domain learning. On Llama-3.1-8B finetuned on the legal domain, TReFT achieves 33.5% more EM reduction than data interleaving with a retain set of aligned examples. We further show that TReFT extends to other narrow-finetuning settings, including abstention, tool use, and refusal (off-topic generalization is reduced by 54.3% on average), supporting the Piggyback Hypothesis. Broadly, our work highlights that LLMs may learn and generalize in unintended ways and suggests a path toward more constrained finetuning. It also calls for further study of how shared input features can piggyback model behavior across domains.",
"url": "http://arxiv.org/abs/2606.06667",
"author": "Jiachen Zhao, Zhengxuan Wu, Aryaman Arora, Yiyou Sun, David Bau, Weiyan Shi",
"published": "2026-06-08",
"source": "arXiv (Computation and Language)",
"source_type": "arxiv",
"tags": [
"cs.CL"
],
"summary": "This paper proposes the Piggyback Hypothesis to explain emergent misalignment, showing that chat-template tokens carry finetuned misbehavior onto unrelated queries. The authors validate it via prefix perturbations and introduce Token-Regularized Finetuning (TReFT) to mitigate misalignment.",
"importance_score": 76,
"reasoning": "Strong mechanistic explanation of emergent misalignment with a concrete causal mechanism and an effective mitigation. Credible authors (Bau, Wu, Arora); high relevance to alignment and safety research.",
"themes": [
"AI Safety",
"Alignment",
"Interpretability",
"Language Models"
],
"continuation": null,
"summary_html": "<p>This paper proposes the Piggyback Hypothesis to explain emergent misalignment, showing that chat-template tokens carry finetuned misbehavior onto unrelated queries. The authors validate it via prefix perturbations and introduce Token-Regularized Finetuning (TReFT) to mitigate misalignment.</p>",
"content_html": "<p>The mechanisms behind LLMs' broad over-generalization beyond training examples remain unclear. Emergent misalignment (EM) offers a striking case study: finetuning on narrow tasks induces broad misalignment to semantically-unrelated test domains. In this work, we propose the Piggyback Hypothesis: the chat-template tokens can piggyback the finetuned behaviour onto out-of-domain queries. We validate this hypothesis by showing that subtle perturbations to the prefix (tokens preceding all user queries), or patching the prefix representations with those from the unfinetuned model, can restore alignment without changing the user query. Building on this finding, we propose Token-Regularized Finetuning (TReFT), which regularizes specific token representations during training to mitigate EM. Across different models and multiple EM-inducing datasets, TReFT reduces EM while preserving in-domain learning. On Llama-3.1-8B finetuned on the legal domain, TReFT achieves 33.5% more EM reduction than data interleaving with a retain set of aligned examples. We further show that TReFT extends to other narrow-finetuning settings, including abstention, tool use, and refusal (off-topic generalization is reduced by 54.3% on average), supporting the Piggyback Hypothesis. Broadly, our work highlights that LLMs may learn and generalize in unintended ways and suggests a path toward more constrained finetuning. It also calls for further study of how shared input features can piggyback model behavior across domains.</p>"
},
{
"id": "e4b697a36cd8",
"title": "Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models",
"content": "Many efforts to ensure frontier AI models are safe rely on monitoring their chain-of-thought (CoT) reasoning. If models become able to perform sufficiently complex reasoning internally, without explicit thinking tokens, this would undermine such oversight. We measure how well frontier models reason without CoT across a suite of over 30,000 questions spanning 43 benchmarks in domains including math, coding, puzzles, causality, theory-of-mind, and strategic reasoning. To compare models against humans, we estimate the $50\\%$-task-completion time horizon (TH): the human time required for tasks a model completes with $50\\%$ success rate. We complement this with a $50\\%$ reasoning token horizon: the minimum number of o3-mini reasoning tokens needed for tasks a model solves with $50\\%$ success rate. We find that the no-CoT $50\\%$ TH of frontier models has been doubling roughly every year over the past six years, with GPT-5.5's TH reaching over 3 minutes and reasoning token horizon exceeding 1,500 tokens. Our median estimates predict that frontier no-CoT THs could exceed 7 minutes by 2028, and 25 minutes by 2030, though these projections carry substantial uncertainty. We recommend frontier developers track this explicitly.",
"url": "http://arxiv.org/abs/2606.07157",
"author": "Dewi Gould, Francis Rhys Ward, Anders Cairns Woodruff, Rauno Arike, Josh Hills, Alex Serrano, Ida Caspary, Jason Ross Brown, Jo J. Jiao, Patrick Leask, Twm Stone, Ram Potham, Ionut Gabriel Stan, Harry Mayne, Simeon Hellsten, Shubhorup Biswas, Ariana Azarbal, William L. Anderson, Elle Najt, Ryan Greenblatt, Julian Stastny",
"published": "2026-06-08",
"source": "arXiv (Artificial Intelligence)",
"source_type": "arxiv",
"tags": [
"cs.AI"
],
"summary": "This study measures how well frontier models reason without chain-of-thought across 30,000+ questions in 43 benchmarks, estimating human time-horizon equivalents for tasks solved without explicit thinking tokens. It matters because CoT-based oversight breaks down if models can reason complexly internally. The strong author roster and safety-relevant framing make this notable.",
"importance_score": 74,
"reasoning": "Directly tackles a core AI safety concern (monitorability of internal reasoning) with a large empirical study and credible alignment researchers including Redwood-affiliated authors. High relevance to oversight.",
"themes": [
"AI Safety",
"Chain-of-Thought",
"Evaluation",
"Language Models"
],
"continuation": null,
"summary_html": "<p>This study measures how well frontier models reason without chain-of-thought across 30,000+ questions in 43 benchmarks, estimating human time-horizon equivalents for tasks solved without explicit thinking tokens. It matters because CoT-based oversight breaks down if models can reason complexly internally. The strong author roster and safety-relevant framing make this notable.</p>",
"content_html": "<p>Many efforts to ensure frontier AI models are safe rely on monitoring their chain-of-thought (CoT) reasoning. If models become able to perform sufficiently complex reasoning internally, without explicit thinking tokens, this would undermine such oversight. We measure how well frontier models reason without CoT across a suite of over 30,000 questions spanning 43 benchmarks in domains including math, coding, puzzles, causality, theory-of-mind, and strategic reasoning. To compare models against humans, we estimate the $50\\%$-task-completion time horizon (TH): the human time required for tasks a model completes with $50\\%$ success rate. We complement this with a $50\\%$ reasoning token horizon: the minimum number of o3-mini reasoning tokens needed for tasks a model solves with $50\\%$ success rate. We find that the no-CoT $50\\%$ TH of frontier models has been doubling roughly every year over the past six years, with GPT-5.5's TH reaching over 3 minutes and reasoning token horizon exceeding 1,500 tokens. Our median estimates predict that frontier no-CoT THs could exceed 7 minutes by 2028, and 25 minutes by 2030, though these projections carry substantial uncertainty. We recommend frontier developers track this explicitly.</p>"
},
{
"id": "72ce62de7588",
"title": "Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests",
"content": "A growing failure mode in agent evaluation and training is that models can achieve high evaluation scores by exploiting shortcuts instead of solving the intended task, producing deceptive performance. This makes evaluation scores unreliable as measures of true task-solving ability. We propose CapCode, a framework for constructing coding datasets with randomized tests whose best achievable non-cheating performance is deliberately capped below one. This capped-performance design gives evaluation scores a clearer interpretation: scores substantially above the cap are implausible and therefore provide evidence of cheating. To prevent cheating, we propose CapReward, a reward design based on the CapCode principle to discourage optimization beyond the cap. Experiments across multiple datasets show that CapCode detects cheating while preserving performance ranking of models, and CapReward reduces cheating behavior, yielding models that better follow the intended task specification.",
"url": "http://arxiv.org/abs/2606.07379",
"author": "Thanawat Lodkaew, Johannes Ackermann, Soichiro Nishimori, Nontawat Charoenphakdee, Masashi Sugiyama, Takashi Ishida",
"published": "2026-06-08",
"source": "arXiv (Machine Learning)",
"source_type": "arxiv",
"tags": [
"cs.LG"
],
"summary": "Introduces CapCode, which builds coding datasets with randomized tests whose maximum non-cheating score is deliberately capped, so scores above the cap reveal reward hacking, plus CapReward to discourage exploitation. Addresses deceptive performance in coding agent evaluation and training.",
"importance_score": 70,
"reasoning": "Clever, principled approach to detecting and preventing reward hacking in agent evaluation, a timely safety-relevant problem with credible authors (Sugiyama, Ishida). Generalizable conceptual contribution.",
"themes": [
"AI Safety",
"LLM Agents",
"Evaluation",
"Reward Hacking"
],
"continuation": null,
"summary_html": "<p>Introduces CapCode, which builds coding datasets with randomized tests whose maximum non-cheating score is deliberately capped, so scores above the cap reveal reward hacking, plus CapReward to discourage exploitation. Addresses deceptive performance in coding agent evaluation and training.</p>",
"content_html": "<p>A growing failure mode in agent evaluation and training is that models can achieve high evaluation scores by exploiting shortcuts instead of solving the intended task, producing deceptive performance. This makes evaluation scores unreliable as measures of true task-solving ability. We propose CapCode, a framework for constructing coding datasets with randomized tests whose best achievable non-cheating performance is deliberately capped below one. This capped-performance design gives evaluation scores a clearer interpretation: scores substantially above the cap are implausible and therefore provide evidence of cheating. To prevent cheating, we propose CapReward, a reward design based on the CapCode principle to discourage optimization beyond the cap. Experiments across multiple datasets show that CapCode detects cheating while preserving performance ranking of models, and CapReward reduces cheating behavior, yielding models that better follow the intended task specification.</p>"
},
{
"id": "da444417a4b5",
"title": "The Geography of Algorithmic Judgment: LLM Intermediaries, Place Identity, and Racial Steering in Housing Search",
"content": "Large language models (LLMs) are rapidly assuming an intermediary role in housing search through the integration of listing platforms within conversational interfaces, mediating access to information, search, and recommendations within urban settings. We expand on prior work on racial steering in LLMs by conducting a behavioral audit of seven open-weight and closed-source LLMs across four U.S. cities, testing location recommendations across three iterative prompting conditions that progressively add lifestyle preference context and reflect fair housing paired-testing methodologies. We find that steering is an emergent behavior of the model's interpretive license rather than primarily a static property. Steering results from the interaction of a user's identity, preference articulation, and the spatial logic that a model has internalized about learned representations of place, preference, and opportunity in a given city, and how different types of users relate to it. While steering was present, it was not uniform in direction or magnitude across evaluated conditions. Preference-conditioned testing often increased or reconfigured the number of models that exhibited steering behaviors relative to baseline conditions, suggesting that LLMs may interpret what the same housing preference means differently depending on the racial identity of the user. Our findings also demonstrate that the city is not a neutral testing unit for LLM evaluation in place-based sectors, and results from one local market cannot be assumed to generalize to another. Local and domain expertise will be required in the housing sector to ensure that legal and institutional commitments to fair housing are not undermined while adopting AI tools that mediate spatial access.",
"url": "http://arxiv.org/abs/2606.06694",
"author": "Hana Samad, Trung Lam, Christoph M\\\"ugge-Durum and Michael Akinwumi",
"published": "2026-06-08",
"source": "arXiv (Machine Learning)",
"source_type": "arxiv",
"tags": [
"cs.LG"
],
"summary": "This behavioral audit of seven LLMs across four US cities tests racial steering in housing recommendations under progressively detailed prompting that mirrors fair-housing paired-testing. It finds steering is an emergent property of model interpretation interacting with user identity and preferences rather than a static property.",
"importance_score": 70,
"reasoning": "Important fairness audit with strong real-world stakes and a methodologically grounded paired-testing design. Relevant to AI bias, fair housing law, and deployment risk; nuanced framing of emergent steering.",
"themes": [
"AI Ethics",
"Fairness",
"Language Models",
"Bias Auditing"
],
"continuation": null,
"summary_html": "<p>This behavioral audit of seven LLMs across four US cities tests racial steering in housing recommendations under progressively detailed prompting that mirrors fair-housing paired-testing. It finds steering is an emergent property of model interpretation interacting with user identity and preferences rather than a static property.</p>",
"content_html": "<p>Large language models (LLMs) are rapidly assuming an intermediary role in housing search through the integration of listing platforms within conversational interfaces, mediating access to information, search, and recommendations within urban settings. We expand on prior work on racial steering in LLMs by conducting a behavioral audit of seven open-weight and closed-source LLMs across four U.S. cities, testing location recommendations across three iterative prompting conditions that progressively add lifestyle preference context and reflect fair housing paired-testing methodologies. We find that steering is an emergent behavior of the model's interpretive license rather than primarily a static property. Steering results from the interaction of a user's identity, preference articulation, and the spatial logic that a model has internalized about learned representations of place, preference, and opportunity in a given city, and how different types of users relate to it. While steering was present, it was not uniform in direction or magnitude across evaluated conditions. Preference-conditioned testing often increased or reconfigured the number of models that exhibited steering behaviors relative to baseline conditions, suggesting that LLMs may interpret what the same housing preference means differently depending on the racial identity of the user. Our findings also demonstrate that the city is not a neutral testing unit for LLM evaluation in place-based sectors, and results from one local market cannot be assumed to generalize to another. Local and domain expertise will be required in the housing sector to ensure that legal and institutional commitments to fair housing are not undermined while adopting AI tools that mediate spatial access.</p>"
},
{
"id": "5e9f1130c57f",
"title": "FP8 is All You Need (Part 1): Debunking Hardware FP64 as the HPC Holy Grail",
"content": "Conventional HPC dogma holds that native hardware FP64 silicon is the irreducible foundation of scientific computing -- the \"holy grail\" of double-precision simulation. This paper argues the dogma is wrong: on AI-optimised GPUs of the B300 generation and beyond, abundant FP8 tensor throughput combined with the Chinese Remainder Theorem-based Ozaki Scheme II recovers memory-roof execution at full FP64 accuracy across the canonical HPC kernel spectrum. NVIDIA's Blackwell Ultra (B300) collapses native FP64 to ~1.3 TFLOPS -- a 31x regression from the B200 -- rendering even memory-bound kernels (SpMV, GEMV, stencils) compute-bound. We make four contributions. First, a unified analytic model, the Tensor-Memory Equilibrium (TME) model, augmenting the Roofline with a compute multiplier alpha, a bandwidth multiplier beta, and a reconstruction latency gamma. Second, we identify register-level fusion as the mechanism driving beta -> 1, making emulation essentially free behind the memory wall. Third, we project that Ozaki II vaults emulated FP64 from the ~1 TFLOPS native floor to ~500 TFLOPS (B300) and ~400 TFLOPS (Rubin R200), exceeding even B200's native FP64 ceiling by over an order of magnitude in the compute-bound regime while matching the memory roof in the bandwidth-bound regime. Fourth, against an H100 baseline, Ozaki II matches or exceeds H100 on every workload studied, versus the up-to-50x regression that B300 native FP64 imposes. Combined with a companion FFT analysis (Kulisch fixed-point reconstruction on the surviving INT32 pipe) and FP32+Kahan reductions reported in the companion Part(2) paper, every surveyed kernel class on B300 reaches the memory roof at full FP64. The evidence supports the title's claim: FP8, with Ozaki II and Kulisch escape routes, is all one needs for production HPC; native FP64 silicon is no longer the holy grail it has been taken to be.",
"url": "http://arxiv.org/abs/2606.06510",
"author": "Satoshi Matsuoka",
"published": "2026-06-08",
"source": "arXiv (cs.AR)",
"source_type": "arxiv",
"tags": [
"cs.AR"
],
"summary": "Argues provocatively that native hardware FP64 is not essential for scientific computing, showing that FP8 tensor throughput plus the Ozaki Scheme II can recover full FP64 accuracy on AI-optimized GPUs like NVIDIA B300. Introduces a Tensor-Memory Equilibrium roofline model to support the claim.",
"importance_score": 68,
"reasoning": "Bold, well-motivated rethinking of HPC precision by a highly credible author (Matsuoka) with potentially major implications for scientific computing on AI hardware; strong novelty and timely given GPU trends.",
"themes": [
"High-Performance Computing",
"Hardware",
"Numerical Precision",
"AI Accelerators"
],
"continuation": null,
"summary_html": "<p>Argues provocatively that native hardware FP64 is not essential for scientific computing, showing that FP8 tensor throughput plus the Ozaki Scheme II can recover full FP64 accuracy on AI-optimized GPUs like NVIDIA B300. Introduces a Tensor-Memory Equilibrium roofline model to support the claim.</p>",
"content_html": "<p>Conventional HPC dogma holds that native hardware FP64 silicon is the irreducible foundation of scientific computing -- the \"holy grail\" of double-precision simulation. This paper argues the dogma is wrong: on AI-optimised GPUs of the B300 generation and beyond, abundant FP8 tensor throughput combined with the Chinese Remainder Theorem-based Ozaki Scheme II recovers memory-roof execution at full FP64 accuracy across the canonical HPC kernel spectrum. NVIDIA's Blackwell Ultra (B300) collapses native FP64 to ~1.3 TFLOPS -- a 31x regression from the B200 -- rendering even memory-bound kernels (SpMV, GEMV, stencils) compute-bound. We make four contributions. First, a unified analytic model, the Tensor-Memory Equilibrium (TME) model, augmenting the Roofline with a compute multiplier alpha, a bandwidth multiplier beta, and a reconstruction latency gamma. Second, we identify register-level fusion as the mechanism driving beta -> 1, making emulation essentially free behind the memory wall. Third, we project that Ozaki II vaults emulated FP64 from the ~1 TFLOPS native floor to ~500 TFLOPS (B300) and ~400 TFLOPS (Rubin R200), exceeding even B200's native FP64 ceiling by over an order of magnitude in the compute-bound regime while matching the memory roof in the bandwidth-bound regime. Fourth, against an H100 baseline, Ozaki II matches or exceeds H100 on every workload studied, versus the up-to-50x regression that B300 native FP64 imposes. Combined with a companion FFT analysis (Kulisch fixed-point reconstruction on the surviving INT32 pipe) and FP32+Kahan reductions reported in the companion Part(2) paper, every surveyed kernel class on B300 reaches the memory roof at full FP64. The evidence supports the title's claim: FP8, with Ozaki II and Kulisch escape routes, is all one needs for production HPC; native FP64 silicon is no longer the holy grail it has been taken to be.</p>"
},
{
"id": "1e1395108719",
"title": "Flatland: The Adventures of Gradient Descent with Large Step Sizes",
"content": "The training of neural networks often entails objective functions that are not globally $L$-smooth. For these functions, it is both theoretically and practically difficult to reply to the question: what is the largest possible step size that ensures the convergence of gradient descent (GD)? We address this longstanding open question in deep learning by providing a unifying definition of \"large\" step sizes that requires only local Lipschitz (or even H\\\"older) continuity of the gradient. We design first-order adaptive methods that provably yield large step sizes and show that they operate at the edge of stability (EoS) right from the start of the training. In particular, the loss decreases nonmonotonically and the product between the step size and sharpness, i.e., the largest eigenvalue of the Hessian, stays above the EoS threshold of 2 throughout training. Using our method, we are also able to minimize the sharpness all the way down to its global minimum. Contrary to expectation, we find that encountering globally-flat regions too early in the training may both slow down convergence and jeopardize the generalization ability of the network. Exploiting a self-stabilization argument, we allow GD to enter slightly sharper valleys and turn unsuccessful training runs into very successful ones.",
"url": "http://arxiv.org/abs/2606.06722",
"author": "Leonardo Galli, Curtis Fox, Wiebke Bartolomaeus, Mark Schmidt, Holger Rauhut",
"published": "2026-06-08",
"source": "arXiv (Machine Learning)",
"source_type": "arxiv",
"tags": [
"cs.LG"
],
"summary": "This theoretical paper provides a unifying definition of large step sizes for gradient descent requiring only local Lipschitz or Holder gradient continuity, addressing the longstanding question of maximum convergent step size for non-smooth objectives. It designs adaptive methods that provably operate at the edge of stability from training start.",
"importance_score": 70,
"reasoning": "Strong theoretical contribution to a fundamental open question in deep learning optimization, connecting to the edge-of-stability phenomenon. Rigorous and broadly relevant; credible optimization-theory authors.",
"themes": [
"Optimization",
"Deep Learning Theory",
"Edge of Stability"
],
"continuation": null,
"summary_html": "<p>This theoretical paper provides a unifying definition of large step sizes for gradient descent requiring only local Lipschitz or Holder gradient continuity, addressing the longstanding question of maximum convergent step size for non-smooth objectives. It designs adaptive methods that provably operate at the edge of stability from training start.</p>",
"content_html": "<p>The training of neural networks often entails objective functions that are not globally $L$-smooth. For these functions, it is both theoretically and practically difficult to reply to the question: what is the largest possible step size that ensures the convergence of gradient descent (GD)? We address this longstanding open question in deep learning by providing a unifying definition of \"large\" step sizes that requires only local Lipschitz (or even H\\\"older) continuity of the gradient. We design first-order adaptive methods that provably yield large step sizes and show that they operate at the edge of stability (EoS) right from the start of the training. In particular, the loss decreases nonmonotonically and the product between the step size and sharpness, i.e., the largest eigenvalue of the Hessian, stays above the EoS threshold of 2 throughout training. Using our method, we are also able to minimize the sharpness all the way down to its global minimum. Contrary to expectation, we find that encountering globally-flat regions too early in the training may both slow down convergence and jeopardize the generalization ability of the network. Exploiting a self-stabilization argument, we allow GD to enter slightly sharper valleys and turn unsuccessful training runs into very successful ones.</p>"
},
{
"id": "27e074eb0867",
"title": "MMBU: A Massive Multi-modal Biomedical Understanding Benchmark to Probe the Perception Capabilities of Vision-Language Models",
"content": "Vision and language models (VLMs) hold immense promise to transform biomedical imaging workflows, from detecting lesions in chest X-rays to profiling cellular features in microscopy. Realizing this potential, however, requires robust and fine-grained visual perception. Models need to correctly interpret subtle features in images, and they must do so across diverse biomedical modalities, scales, and contexts. Nevertheless, current benchmarks remain limited. To address these gaps, we introduce the Massive Multimodal Biomedical Understanding (MMBU) benchmark. It is the largest biomedical vision and language benchmark to date, covering 35 submodalities with rich structured metadata. It includes both open and closed versions of ungrounded classification, grounded classification, and object detection, enabling systematic evaluation of model performance across biological scales, clinical settings, and imaging modalities. Evaluating 15 open-weight and 2 frontier VLMs, we find that while medical adaptation provides measurable gains for some models, the high accuracy often reported on established benchmarks can mask deficiencies in visual perception and domain generalization.",
"url": "http://arxiv.org/abs/2606.06696",
"author": "Ryan D'Cunha, Alejandro Lozano, Xiaoxiao Sun, Daniel Vela Jarquin, Min Woo Sun, Josiah Aklilu, James Burgess, Yuhui Zhang, Ryan Nayebi, Paola Avila, Robayo, Jin Ye, Ming Hu, Zhongying Deng, Junjun He, Xin Chen, Yue Yao, Robert Tibshirani, Jeffrey J. Nirschl, and Serena Yeung-Levy",
"published": "2026-06-08",
"source": "arXiv (Computer Vision)",
"source_type": "arxiv",
"tags": [
"cs.CV"
],
"summary": "MMBU is the largest biomedical vision-language benchmark to date, covering 35 submodalities with structured metadata and tasks spanning ungrounded/grounded classification and object detection. It probes fine-grained perception capabilities of VLMs across diverse biomedical imaging contexts.",
"importance_score": 69,
"reasoning": "Large, well-structured benchmark addressing a real gap in biomedical VLM evaluation, from a strong group (Yeung-Levy, Tibshirani). High potential to drive progress in medical multimodal AI.",
"themes": [
"Computer Vision",
"Healthcare AI",
"Vision-Language Models",
"Benchmarks"
],
"continuation": null,
"summary_html": "<p>MMBU is the largest biomedical vision-language benchmark to date, covering 35 submodalities with structured metadata and tasks spanning ungrounded/grounded classification and object detection. It probes fine-grained perception capabilities of VLMs across diverse biomedical imaging contexts.</p>",
"content_html": "<p>Vision and language models (VLMs) hold immense promise to transform biomedical imaging workflows, from detecting lesions in chest X-rays to profiling cellular features in microscopy. Realizing this potential, however, requires robust and fine-grained visual perception. Models need to correctly interpret subtle features in images, and they must do so across diverse biomedical modalities, scales, and contexts. Nevertheless, current benchmarks remain limited. To address these gaps, we introduce the Massive Multimodal Biomedical Understanding (MMBU) benchmark. It is the largest biomedical vision and language benchmark to date, covering 35 submodalities with rich structured metadata. It includes both open and closed versions of ungrounded classification, grounded classification, and object detection, enabling systematic evaluation of model performance across biological scales, clinical settings, and imaging modalities. Evaluating 15 open-weight and 2 frontier VLMs, we find that while medical adaptation provides measurable gains for some models, the high accuracy often reported on established benchmarks can mask deficiencies in visual perception and domain generalization.</p>"
},
{
"id": "d14361debc4b",
"title": "Position: Don't Just \"Fix it in Post\": A Science of AI Must Study Training Dynamics",
"content": "What would it mean to have a scientific understanding of AI? Models are not static objects: they are snapshots of time-evolving processes shaped by data, objectives, architectures, and optimization dynamics. Yet much of AI research treats models as fixed artifacts, analyzing behaviors after training rather than asking why they emerge. This position paper argues that a science of AI must move beyond post-hoc fixes and study the training dynamics that produce model behavior. Such a science should support progressively stronger forms of understanding: predicting outcomes from early training signals, intervening when trajectories go wrong, and ultimately designing training procedures that more reliably produce desired properties. Scaling laws have made prediction routine for loss; the challenge is extending this success to capabilities, biases, robustness, and safety-relevant behaviors. We articulate requirements for such theories grounded in the history and philosophy of science, examine progress in mechanistic interpretability, fairness, memorization, and simplicity bias, and identify concrete open problems.",
"url": "http://arxiv.org/abs/2606.06533",
"author": "Stella Biderman and Mohammad Aflah Khan and Niloofar Mireshghallah and Catherine Arnett and Fazl Barez and Naomi Saphra",
"published": "2026-06-08",
"source": "arXiv (Artificial Intelligence)",
"source_type": "arxiv",
"tags": [
"cs.AI"
],
"summary": "A position paper arguing that a genuine science of AI must study training dynamics rather than relying on post-hoc fixes, advocating for predicting outcomes from early signals, intervening on trajectories, and designing training to reliably produce desired properties. Frames scaling laws as just a start toward understanding emergence.",
"importance_score": 67,
"reasoning": "Influential position from highly credible authors (Biderman, Saphra, Barez, Mireshghallah) shaping how the field thinks about understanding model behavior; agenda-setting though not a technical result.",
"themes": [
"Science of Deep Learning",
"Training Dynamics",
"Interpretability",
"AI Research Methodology"
],
"continuation": null,
"summary_html": "<p>A position paper arguing that a genuine science of AI must study training dynamics rather than relying on post-hoc fixes, advocating for predicting outcomes from early signals, intervening on trajectories, and designing training to reliably produce desired properties. Frames scaling laws as just a start toward understanding emergence.</p>",
"content_html": "<p>What would it mean to have a scientific understanding of AI? Models are not static objects: they are snapshots of time-evolving processes shaped by data, objectives, architectures, and optimization dynamics. Yet much of AI research treats models as fixed artifacts, analyzing behaviors after training rather than asking why they emerge. This position paper argues that a science of AI must move beyond post-hoc fixes and study the training dynamics that produce model behavior. Such a science should support progressively stronger forms of understanding: predicting outcomes from early training signals, intervening when trajectories go wrong, and ultimately designing training procedures that more reliably produce desired properties. Scaling laws have made prediction routine for loss; the challenge is extending this success to capabilities, biases, robustness, and safety-relevant behaviors. We articulate requirements for such theories grounded in the history and philosophy of science, examine progress in mechanistic interpretability, fairness, memorization, and simplicity bias, and identify concrete open problems.</p>"
},
{
"id": "9aabd92c9d93",
"title": "How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures",
"content": "Failures in language model reasoning emerge through distinct processes that leave identifiable signatures in the reasoning trace. We characterize these failures using token-level uncertainty signals, finding they arise through two empirically distinguishable processes. The first is committed failure, in which a model locks onto an incorrect reasoning path early in its trace. A central diagnostic signature is the commitment point, beyond which considering additional tokens hurt rather than help failure detection. In the second, persistent uncertainty, uncertainty instead accumulates throughout, and the full trace is needed to best distinguish failing from successful completions. These signatures reproduce across 23 model-dataset configurations, with the framework's falsifiable predictions holding in 20 of 23 cases, well above chance across both failure modes. Finally, we demonstrate our failure mode framework has direct implications for self-consistency, identifying when uncertainty signals complement it and when it can be selectively skipped. These results offer a foundation for understanding when LLM reasoning failures become detectable and for adapting detection strategies accordingly.",
"url": "http://arxiv.org/abs/2606.06635",
"author": "Tanvi Thoria, Kiana Jafari, Marc R. Schlichting, Mykel J. Kochenderfer",
"published": "2026-06-08",
"source": "arXiv (Computation and Language)",
"source_type": "arxiv",
"tags": [
"cs.CL"
],
"summary": "The authors characterize LLM reasoning failures using token-level uncertainty signals, distinguishing committed failures (early lock-in) from persistent uncertainty (accumulating doubt). The framework offers falsifiable diagnostics reproducible across 23 model-dataset configurations, useful for failure detection in reasoning systems.",
"importance_score": 68,
"reasoning": "Strong methodological contribution to interpretability of reasoning failures with falsifiable predictions and broad empirical validation. Credible authors including Kochenderfer. Practical relevance to reasoning reliability and monitoring.",
"themes": [
"Language Models",
"Interpretability",
"Reasoning"
],
"continuation": null,
"summary_html": "<p>The authors characterize LLM reasoning failures using token-level uncertainty signals, distinguishing committed failures (early lock-in) from persistent uncertainty (accumulating doubt). The framework offers falsifiable diagnostics reproducible across 23 model-dataset configurations, useful for failure detection in reasoning systems.</p>",
"content_html": "<p>Failures in language model reasoning emerge through distinct processes that leave identifiable signatures in the reasoning trace. We characterize these failures using token-level uncertainty signals, finding they arise through two empirically distinguishable processes. The first is committed failure, in which a model locks onto an incorrect reasoning path early in its trace. A central diagnostic signature is the commitment point, beyond which considering additional tokens hurt rather than help failure detection. In the second, persistent uncertainty, uncertainty instead accumulates throughout, and the full trace is needed to best distinguish failing from successful completions. These signatures reproduce across 23 model-dataset configurations, with the framework's falsifiable predictions holding in 20 of 23 cases, well above chance across both failure modes. Finally, we demonstrate our failure mode framework has direct implications for self-consistency, identifying when uncertainty signals complement it and when it can be selectively skipped. These results offer a foundation for understanding when LLM reasoning failures become detectable and for adapting detection strategies accordingly.</p>"
},
{
"id": "39879c2f329c",
"title": "Sparsely gated tiny linear experts",
"content": "Sparsity allows scaling model parameters without proportionally increasing computational cost. While mixture of experts (MoE) models are made increasingly sparse, individual experts typically remain large and dense. Here, we demonstrate that further increasing sparsity by shrinking each expert to consist of a single neuron and selecting a tiny fraction of many available neurons can improve compute efficiency and interpretability. Counterintuitively, the key to achieving both is removing the nonlinearity typically applied to the experts, resulting in a network of sparsely gated linear neurons (sgatlin). In an isoflop comparison, we find that replacing all transformer feedforward layers with sgatlin improves perplexity in language models across different compute budgets. At the same time, the sparsity and linearity of the resulting feedforward circuits present new opportunities for model interpretability. In a small-scale case study, we demonstrate that feedforward circuits in sgatlin can be interpreted without having to train additional replacement models. We find that they form semantically structured clusters and are causally implicated in factual recall. Our findings paint a possible path towards compute-efficient and interpretable transformer feedforward layers.",
"url": "http://arxiv.org/abs/2606.07414",
"author": "Simon Schug",
"published": "2026-06-08",
"source": "arXiv (Machine Learning)",
"source_type": "arxiv",
"tags": [
"cs.LG"
],
"summary": "Demonstrates that pushing MoE sparsity to the extreme—single-neuron experts selected from many, with the nonlinearity removed (sparsely gated linear neurons, sgatlin)—improves perplexity over standard FFN layers at matched FLOPs while aiding interpretability. Challenges the assumption that experts must be large and nonlinear.",
"importance_score": 66,
"reasoning": "Counterintuitive and clean architectural insight with isoflop gains and interpretability benefits; single-author but conceptually fresh and potentially influential for efficient/interpretable MoE design.",
"themes": [
"Mixture of Experts",
"Architecture",
"Efficiency",
"Interpretability"
],
"continuation": null,
"summary_html": "<p>Demonstrates that pushing MoE sparsity to the extreme—single-neuron experts selected from many, with the nonlinearity removed (sparsely gated linear neurons, sgatlin)—improves perplexity over standard FFN layers at matched FLOPs while aiding interpretability. Challenges the assumption that experts must be large and nonlinear.</p>",
"content_html": "<p>Sparsity allows scaling model parameters without proportionally increasing computational cost. While mixture of experts (MoE) models are made increasingly sparse, individual experts typically remain large and dense. Here, we demonstrate that further increasing sparsity by shrinking each expert to consist of a single neuron and selecting a tiny fraction of many available neurons can improve compute efficiency and interpretability. Counterintuitively, the key to achieving both is removing the nonlinearity typically applied to the experts, resulting in a network of sparsely gated linear neurons (sgatlin). In an isoflop comparison, we find that replacing all transformer feedforward layers with sgatlin improves perplexity in language models across different compute budgets. At the same time, the sparsity and linearity of the resulting feedforward circuits present new opportunities for model interpretability. In a small-scale case study, we demonstrate that feedforward circuits in sgatlin can be interpreted without having to train additional replacement models. We find that they form semantically structured clusters and are causally implicated in factual recall. Our findings paint a possible path towards compute-efficient and interpretable transformer feedforward layers.</p>"
}
]
},
"social": {
"count": 293,
"category_summary": "**Coding agents** dominated leadership commentary, with **OpenAI's Greg Brockman** [framing](/?date=2026-06-08&category=social#item-2e392c5dd614) **Codex** as an AI teammate and [arguing a large capability overhang](/?date=2026-06-08&category=social#item-b49450557de7) exists—users underutilize it due to habit, not model limits.\n\n- **Mira Murati's** [first post-OpenAI interview](/?date=2026-06-08&category=social#item-f66f7754a376) drew strong engagement, outlining **Thinking Machines'** vision for human-AI collaboration.\n- **Nathan Lambert** [spotlighted AI safety](/?date=2026-06-08&category=social#item-9afe9051dbca), stressing how much remains unknown and uncontrolled inside models.\n- **swyx** posted a [provocative thesis](/?date=2026-06-08&category=social#item-74e8cb289a83) that research-paper alpha and lab publishing died as talent commands $100M+ for tacit knowledge.\n- **Ethan Mollick** [advised stockpiling hard, unusual ideas](/?date=2026-06-08&category=social#item-a96f249f9c10) since AI makes execution cheap but ideation no easier.\n\n**AI economics and bubble skepticism**, largely driven by **Gary Marcus**, formed a heavy counter-current: critiques of [half-trillion-dollar industry losses](/?date=2026-06-08&category=social#item-f16d95b9ded5), [**SpaceX IPO** hype](/?date=2026-06-08&category=social#item-db3a1c79a612), and claims that [open-sourcing **Llama**](/?date=2026-06-08&category=social#item-9ae62a2f2f63) catalyzed **China's** AI rise. On the tooling side, coverage of **Microsoft Research's** [SkillOpt—self-evolving agent skills](/?date=2026-06-08&category=social#item-4d2f045ad6ac)—rounded out the most technically substantive discussions.",
"category_summary_html": "<p><strong>Coding agents</strong> dominated leadership commentary, with <strong>OpenAI's Greg Brockman</strong> <a href=\"/?date=2026-06-08&category=social#item-2e392c5dd614\" class=\"internal-link\" rel=\"noopener noreferrer\">framing</a> <strong>Codex</strong> as an AI teammate and <a href=\"/?date=2026-06-08&category=social#item-b49450557de7\" class=\"internal-link\" rel=\"noopener noreferrer\">arguing a large capability overhang</a> exists—users underutilize it due to habit, not model limits.</p>\n<ul>\n<li><strong>Mira Murati's</strong> <a href=\"/?date=2026-06-08&category=social#item-f66f7754a376\" class=\"internal-link\" rel=\"noopener noreferrer\">first post-OpenAI interview</a> drew strong engagement, outlining <strong>Thinking Machines'</strong> vision for human-AI collaboration.</li>\n<li><strong>Nathan Lambert</strong> <a href=\"/?date=2026-06-08&category=social#item-9afe9051dbca\" class=\"internal-link\" rel=\"noopener noreferrer\">spotlighted AI safety</a>, stressing how much remains unknown and uncontrolled inside models.</li>\n<li><strong>swyx</strong> posted a <a href=\"/?date=2026-06-08&category=social#item-74e8cb289a83\" class=\"internal-link\" rel=\"noopener noreferrer\">provocative thesis</a> that research-paper alpha and lab publishing died as talent commands $100M+ for tacit knowledge.</li>\n<li><strong>Ethan Mollick</strong> <a href=\"/?date=2026-06-08&category=social#item-a96f249f9c10\" class=\"internal-link\" rel=\"noopener noreferrer\">advised stockpiling hard, unusual ideas</a> since AI makes execution cheap but ideation no easier.</li>\n</ul>\n<p><strong>AI economics and bubble skepticism</strong>, largely driven by <strong>Gary Marcus</strong>, formed a heavy counter-current: critiques of <a href=\"/?date=2026-06-08&category=social#item-f16d95b9ded5\" class=\"internal-link\" rel=\"noopener noreferrer\">half-trillion-dollar industry losses</a>, <a href=\"/?date=2026-06-08&category=social#item-db3a1c79a612\" class=\"internal-link\" rel=\"noopener noreferrer\"><strong>SpaceX IPO</strong> hype</a>, and claims that <a href=\"/?date=2026-06-08&category=social#item-9ae62a2f2f63\" class=\"internal-link\" rel=\"noopener noreferrer\">open-sourcing <strong>Llama</strong></a> catalyzed <strong>China's</strong> AI rise. On the tooling side, coverage of <strong>Microsoft Research's</strong> <a href=\"/?date=2026-06-08&category=social#item-4d2f045ad6ac\" class=\"internal-link\" rel=\"noopener noreferrer\">SkillOpt—self-evolving agent skills</a>—rounded out the most technically substantive discussions.</p>",
"themes": [
{
"name": "AI Safety and Model Uncertainty",
"description": "natolambert highlights how much remains unknown and uncontrolled inside AI models as an argument for safety.",
"item_count": 1,
"example_items": [],
"importance": 72
},
{
"name": "Coding Agents and Codex",
"description": "OpenAI leadership promoting Codex as an AI teammate and discussing untapped capability overhang.",
"item_count": 3,
"example_items": [],
"importance": 70
},
{
"name": "AI Industry Leadership and Talent",
"description": "Mira Murati's Thinking Machines vision and swyx's thesis on talent mobility, non-competes, and the decline of lab research publishing.",
"item_count": 3,
"example_items": [],
"importance": 65
},
{
"name": "AI Economics and Bubble Concerns",
"description": "Critiques of AI industry losses, IPO hype, subsidies, and too-big-to-fail dynamics, largely driven by Gary Marcus.",
"item_count": 12,
"example_items": [],
"importance": 60
},
{
"name": "Open Source and AI Geopolitics",
"description": "Debate over whether open-sourcing Llama empowered China and US open-source leadership claims.",
"item_count": 4,
"example_items": [],
"importance": 55
},
{
"name": "AI Agents and Governance",
"description": "Microsoft Research's SkillOpt for self-evolving agent skills and the Agent Governance Toolkit for enforcing agent policy at the application layer, signaling maturation of agent infrastructure and safety.",
"item_count": 2,
"example_items": [],
"importance": 55
},
{
"name": "Robotics and Embodied AI",
"description": "A heavy concentration of posts on humanoid robots, delivery robots, industrial automation, swarm intelligence, and novel locomotion designs, mostly curated by @Ronald_vanLoon.",
"item_count": 16,
"example_items": [],
"importance": 55
},
{
"name": "AI Productivity and Innovation",
"description": "Ideas about AI lowering implementation costs and increasing value of unique ideas.",
"item_count": 2,
"example_items": [],
"importance": 50
},
{
"name": "AI Devices and App Ecosystem Disruption",
"description": "Speculation that AI-native devices and master AI systems will replace app stores and individual applications, threatening Apple and Google.",
"item_count": 3,
"example_items": [],
"importance": 50
},
{
"name": "AI-Native Applications and Tooling Strategy",
"description": "Jerry Liu's framing of AI-native app waves, agent-native document formats, and centralization around apps like Claude Code and Codex.",
"item_count": 4,
"example_items": [],
"importance": 48
}
],
"top_items": [
{
"id": "b49450557de7",
"title": "Whenever I don’t use codex for a task, I ask myself why and usually realize that there’s some missin...",
"content": "Whenever I don’t use codex for a task, I ask myself why and usually realize that there’s some missing context, I needed to write a skill, or I just didn’t think to use it.\n\nRarely is it because the task is outside of the capabilities of the model. Overhang right now feels large.",
"url": "https://twitter.com/gdb/status/2063437915347136554",
"author": "@gdb",
"published": "2026-06-07T01:48:30",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Greg Brockman observes that when he avoids using Codex it is usually due to missing context or habit rather than model limits, suggesting a large capability overhang.",
"importance_score": 75,
"reasoning": "OpenAI leader with very high engagement and an insightful claim about untapped Codex capability overhang, a meaningful industry signal.",
"themes": [
"coding agents",
"Codex",
"OpenAI",
"AI capabilities"
],
"continuation": null,
"summary_html": "<p>Greg Brockman observes that when he avoids using Codex it is usually due to missing context or habit rather than model limits, suggesting a large capability overhang.</p>",
"content_html": "<p>Whenever I don’t use codex for a task, I ask myself why and usually realize that there’s some missing context, I needed to write a skill, or I just didn’t think to use it.</p>\n<p>Rarely is it because the task is outside of the capabilities of the model. Overhang right now feels large.</p>"
},
{
"id": "f66f7754a376",
"title": "In her first wide-ranging interview since leaving OpenAI, @miramurati shared more than ever before ...",
"content": "In her first wide-ranging interview since leaving OpenAI, @miramurati shared more than ever before about what she’s building at her AGI startup, @thinkymachines lab.\n\nThe former OpenAI CTO laid out her vision for a future where humans and AI work together more closely -- “like a tandem bike” -- and where people aren’t pushed out of the loop as machines become more capable.\n\nvia @BloombergLive",
"url": "https://twitter.com/emilychangtv/status/2063621852795113918",
"author": "@emilychangtv",
"published": "2026-06-07T13:59:24",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Emily Chang highlights Mira Murati's first wide-ranging interview since leaving OpenAI, where the former CTO describes Thinking Machines' vision of humans and AI collaborating like a tandem bike and keeping people in the loop.",
"importance_score": 70,
"reasoning": "Notable industry figure, original interview content on a high-profile AGI startup, strong engagement and reach.",
"themes": [
"AI industry leadership",
"AGI vision",
"Thinking Machines"
],
"continuation": null,
"summary_html": "<p>Emily Chang highlights Mira Murati's first wide-ranging interview since leaving OpenAI, where the former CTO describes Thinking Machines' vision of humans and AI collaborating like a tandem bike and keeping people in the loop.</p>",
"content_html": "<p>In her first wide-ranging interview since leaving OpenAI, @miramurati shared more than ever before about what she’s building at her AGI startup, @thinkymachines lab.</p>\n<p>The former OpenAI CTO laid out her vision for a future where humans and AI work together more closely -- “like a tandem bike” -- and where people aren’t pushed out of the loop as machines become more capable.</p>\n<p>via @BloombergLive</p>"
},
{
"id": "9afe9051dbca",
"title": "Something to show people that don't get AI safety at least a little bit. We have so much we don't kn...",
"content": "Something to show people that don't get AI safety at least a little bit. We have so much we don't know and don't currently control in the models.\n\n(extreme content warning, but you're on X)",
"url": "https://twitter.com/natolambert/status/2063444305583431901",
"author": "@natolambert",
"published": "2026-06-07T02:13:54",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "natolambert shares something he frames as a demonstration of AI safety concerns, emphasizing how much remains unknown and uncontrolled in models.",
"importance_score": 72,
"reasoning": "Credible AI researcher commenting on AI safety with very high views and strong engagement; relevant and from a respected voice though the content link is implied.",
"themes": [
"AI safety",
"model interpretability",
"alignment"
],
"continuation": null,
"summary_html": "<p>natolambert shares something he frames as a demonstration of AI safety concerns, emphasizing how much remains unknown and uncontrolled in models.</p>",
"content_html": "<p>Something to show people that don't get AI safety at least a little bit. We have so much we don't know and don't currently control in the models.</p>\n<p>(extreme content warning, but you're on X)</p>"
},
{
"id": "74e8cb289a83",
"title": "one popular theory is that research paper alpha* and lab publishing ~died when researchers realized ...",
"content": "one popular theory is that research paper alpha* and lab publishing ~died when researchers realized that instead of fighting with marketing depts they could simply walk out the door and get >$100m for their legally protected tacit knowledge gained\n\ncalifornia non-noncompetes have a bigger impact on knowledge spreading than github, arxiv, and huggingface combined\n\n*btw this is a motivator for me to set up @aidotengineer as a product-centric industry conference to complement the paper-centric research conferences",
"url": "https://twitter.com/swyx/status/2063432747432268259",
"author": "@swyx",
"published": "2026-06-07T01:27:58",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Swyx argues research-paper alpha and lab publishing died as researchers realized they could leave for over $100M for their tacit knowledge, claiming California non-compete rules spread knowledge more than GitHub, arxiv, and Hugging Face combined; pitches his AI Engineer conference as a product-centric complement.",
"importance_score": 65,
"reasoning": "Original, provocative thesis on talent mobility and knowledge diffusion from a recognized AI ecosystem figure, with strong engagement.",
"themes": [
"AI talent and mobility",
"research culture",
"non-competes",
"AI industry"
],
"continuation": null,
"summary_html": "<p>Swyx argues research-paper alpha and lab publishing died as researchers realized they could leave for over $100M for their tacit knowledge, claiming California non-compete rules spread knowledge more than GitHub, arxiv, and Hugging Face combined; pitches his AI Engineer conference as a product-centric complement.</p>",
"content_html": "<p>one popular theory is that research paper alpha* and lab publishing ~died when researchers realized that instead of fighting with marketing depts they could simply walk out the door and get >$100m for their legally protected tacit knowledge gained</p>\n<p>california non-noncompetes have a bigger impact on knowledge spreading than github, arxiv, and huggingface combined</p>\n<p>*btw this is a motivator for me to set up @aidotengineer as a product-centric industry conference to complement the paper-centric research conferences</p>"
},
{
"id": "a96f249f9c10",
"title": "It is a really good time to store up a few of your hardest, most valuable, and most unusual ideas - ...",
"content": "It is a really good time to store up a few of your hardest, most valuable, and most unusual ideas - whether for work, hobbies, or a new venture.\n\nThanks to AI, really good & unique ideas are getting extremely cheap to implement, but not necessarily easier to find. Big opportunity",
"url": "https://twitter.com/emollick/status/2063671312178888847",
"author": "@emollick",
"published": "2026-06-07T17:15:56",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Ethan Mollick advises stockpiling your hardest and most unusual ideas because AI makes good ideas cheap to implement but no easier to find.",
"importance_score": 62,
"reasoning": "Respected AI commentator with very high engagement and an original, actionable insight on AI's effect on ideation versus execution.",
"themes": [
"AI productivity",
"innovation",
"future of work"
],
"continuation": null,
"summary_html": "<p>Ethan Mollick advises stockpiling your hardest and most unusual ideas because AI makes good ideas cheap to implement but no easier to find.</p>",
"content_html": "<p>It is a really good time to store up a few of your hardest, most valuable, and most unusual ideas - whether for work, hobbies, or a new venture.</p>\n<p>Thanks to AI, really good & unique ideas are getting extremely cheap to implement, but not necessarily easier to find. Big opportunity</p>"
},
{
"id": "2e392c5dd614",
"title": "Codex use-cases: “From software engineering and design to data analysis and operations, Codex is bec...",
"content": "Codex use-cases: “From software engineering and design to data analysis and operations, Codex is becoming an AI teammate instead of just an AI assistant.”",
"url": "https://twitter.com/gdb/status/2063705280270021087",
"author": "@gdb",
"published": "2026-06-07T19:30:55",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Greg Brockman highlights Codex use cases, framing it as becoming an AI teammate rather than just an assistant across engineering, design, and operations.",
"importance_score": 70,
"reasoning": "OpenAI leader with very high engagement and views promoting Codex's expanding role; significant industry signal despite promotional tone.",
"themes": [
"coding agents",
"Codex",
"OpenAI",
"AI productivity"
],
"continuation": null,
"summary_html": "<p>Greg Brockman highlights Codex use cases, framing it as becoming an AI teammate rather than just an assistant across engineering, design, and operations.</p>",
"content_html": "<p>Codex use-cases: “From software engineering and design to data analysis and operations, Codex is becoming an AI teammate instead of just an AI assistant.”</p>"
},
{
"id": "9ae62a2f2f63",
"title": "Zuckerberg and LeCun’s unilateral decision to open source Llama likely (partly) catalyzed China’s AI...",
"content": "Zuckerberg and LeCun’s unilateral decision to open source Llama likely (partly) catalyzed China’s AI industry — and may have done truly massive harm to American business interests.\n\nWe are now starting to see the consequences.",
"url": "https://twitter.com/GaryMarcus/status/2063733036953526701",
"author": "@GaryMarcus",
"published": "2026-06-07T21:21:12",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Marcus argues that Zuckerberg and LeCun's decision to open-source Llama catalyzed China's AI industry and may have harmed US business interests.",
"importance_score": 65,
"reasoning": "High engagement and views with a provocative, original geopolitical argument from a leading AI critic; central theme of the batch.",
"themes": [
"open source",
"China AI",
"AI geopolitics",
"Meta"
],
"continuation": null,
"summary_html": "<p>Marcus argues that Zuckerberg and LeCun's decision to open-source Llama catalyzed China's AI industry and may have harmed US business interests.</p>",
"content_html": "<p>Zuckerberg and LeCun’s unilateral decision to open source Llama likely (partly) catalyzed China’s AI industry — and may have done truly massive harm to American business interests.</p>\n<p>We are now starting to see the consequences.</p>"
},
{
"id": "f16d95b9ded5",
"title": "Only in an America can an industry that has collectively lost over half a trillion dollars —at a pac...",
"content": "Only in an America can an industry that has collectively lost over half a trillion dollars —at a pace of roughly a million dollars a minute — ask for (and likely get) government subsidies. https://t.co/DAbr5vnU6J",
"url": "https://twitter.com/GaryMarcus/status/2063707966604562453",
"author": "@GaryMarcus",
"published": "2026-06-07T19:41:35",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Marcus criticizes the AI industry for losing over half a trillion dollars while seeking government subsidies.",
"importance_score": 60,
"reasoning": "Very high engagement and a striking financial claim central to his AI-bubble critique.",
"themes": [
"AI economics",
"AI bubble",
"subsidies",
"AI criticism"
],
"continuation": null,
"summary_html": "<p>Marcus criticizes the AI industry for losing over half a trillion dollars while seeking government subsidies.</p>",
"content_html": "<p>Only in an America can an industry that has collectively lost over half a trillion dollars —at a pace of roughly a million dollars a minute — ask for (and likely get) government subsidies. https://t.co/DAbr5vnU6J</p>"
},
{
"id": "4d2f045ad6ac",
"title": "Handwritten skills are dead. Microsoft just taught them to self-evolve. \n\nMost agent skill docs are ...",
"content": "Handwritten skills are dead. Microsoft just taught them to self-evolve. \n\nMost agent skill docs are handwritten and brittle. They rarely improve once shipped.\n \nMicrosoft Research just released a paper fixing this. \n\nIt treats the skill file as a frozen agent's trainable state. A second model reads execution traces. \n\nIt then proposes small add, delete, or replace edits.\n \nA candidate ships only if it beats the prior version. \n\nA held-out set decides every round.\n \nThe system borrows discipline from gradient descent.\n \n> Edit budget acts as learning rate\n> Rejected changes feed negative signal\n> Slow updates capture long-term patterns\n> Minibatches separate wins from failures\n \nCalled SkillOpt, it won or tied on 52 setups. \n\nTests spanned six benchmarks and seven models.\n \nOn one frontier setup it added 23.5 points in chat. Coding harnesses gained up to 24.8.\n \nThe final artifact stays under 2,000 tokens. \n\nWhat ships when skills train themselves?",
"url": "https://twitter.com/AlphaSignalAI/status/2063637407757398434",
"author": "@AlphaSignalAI",
"published": "2026-06-07T15:01:13",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "AlphaSignalAI summarizes a Microsoft Research paper called SkillOpt that treats an agent skill file as trainable state, using a second model to propose add/delete/replace edits validated against a held-out set, borrowing gradient-descent discipline; claims wins on 52 setups across six benchmarks.",
"importance_score": 55,
"reasoning": "Technically detailed write-up of a genuinely novel agent-skill optimization approach with solid engagement, though sourced second-hand from an aggregator.",
"themes": [
"AI agents",
"agent skills and self-improvement",
"research"
],
"continuation": null,
"summary_html": "<p>AlphaSignalAI summarizes a Microsoft Research paper called SkillOpt that treats an agent skill file as trainable state, using a second model to propose add/delete/replace edits validated against a held-out set, borrowing gradient-descent discipline; claims wins on 52 setups across six benchmarks.</p>",
"content_html": "<p>Handwritten skills are dead. Microsoft just taught them to self-evolve.</p>\n<p>Most agent skill docs are handwritten and brittle. They rarely improve once shipped.</p>\n<p>Microsoft Research just released a paper fixing this.</p>\n<p>It treats the skill file as a frozen agent's trainable state. A second model reads execution traces.</p>\n<p>It then proposes small add, delete, or replace edits.</p>\n<p>A candidate ships only if it beats the prior version.</p>\n<p>A held-out set decides every round.</p>\n<p>The system borrows discipline from gradient descent.</p>\n<p>> Edit budget acts as learning rate</p>\n<p>> Rejected changes feed negative signal</p>\n<p>> Slow updates capture long-term patterns</p>\n<p>> Minibatches separate wins from failures</p>\n<p>Called SkillOpt, it won or tied on 52 setups.</p>\n<p>Tests spanned six benchmarks and seven models.</p>\n<p>On one frontier setup it added 23.5 points in chat. Coding harnesses gained up to 24.8.</p>\n<p>The final artifact stays under 2,000 tokens.</p>\n<p>What ships when skills train themselves?</p>"
},
{
"id": "db3a1c79a612",
"title": "If you aren’t one of the banks running the SpaceX IPO, you’re the mark: https://t.co/EbQjGV6e40",
"content": "If you aren’t one of the banks running the SpaceX IPO, you’re the mark: https://t.co/EbQjGV6e40",
"url": "https://twitter.com/GaryMarcus/status/2063620912122056732",
"author": "@GaryMarcus",
"published": "2026-06-07T13:55:40",
"source": "Twitter",
"source_type": "twitter",
"tags": [],
"summary": "Marcus warns that anyone not among the banks running the SpaceX IPO is the mark.",
"importance_score": 58,
"reasoning": "Very high engagement and a pointed investor-warning critique of the SpaceX IPO hype.",
"themes": [
"IPO",
"SpaceX",
"AI hype",
"investing"
],
"continuation": null,
"summary_html": "<p>Marcus warns that anyone not among the banks running the SpaceX IPO is the mark.</p>",
"content_html": "<p>If you aren’t one of the banks running the SpaceX IPO, you’re the mark: https://t.co/EbQjGV6e40</p>"
}
]
},
"reddit": {
"count": 426,
"category_summary": "**r/LocalLLaMA** was dominated by **Gemma 4**. The day's top thread was **llama.cpp** merging **MTP (multi-token prediction)** support, with comments [reporting ~140 tok/s](/?date=2026-06-08&category=reddit#item-f7fb09447ce6) on a 12GB RTX 40-series card.\n\n- Builders pushed local limits: CPU-only **Gemma-4-26B-A4B** [at ~7 tok/s](/?date=2026-06-08&category=reddit#item-403729badb4e), and **fp8 Gemma 4 31B** [reportedly matching](/?date=2026-06-08&category=reddit#item-c6475c516b0f) **Claude Sonnet 4.6** in an agentic harness, alongside [skepticism about](/?date=2026-06-08&category=reddit#item-c662b00a3163) **QAT quant** quality\n- **r/StableDiffusion** ran an **Ideogram 4** evaluation wave—debating [underrated vs overhyped](/?date=2026-06-08&category=reddit#item-9f910d0d6bdc), sharing [censorship-bypass sampler tricks](/?date=2026-06-08&category=reddit#item-17c5c9d873a0), and griping about [JSON-only prompting](/?date=2026-06-08&category=reddit#item-4b424bbf4ea2)\n- **r/ClaudeAI/Anthropic** [argued over new](/?date=2026-06-08&category=reddit#item-4037ccf9bbb8) **usage limits**, whether [models are](/?date=2026-06-08&category=reddit#item-3a34392260c9) 'getting worse,' and the joy of [building tiny single-user apps](/?date=2026-06-08&category=reddit#item-fda5317d40d7) with **Claude Code**\n- **AI governance** drew cynicism as Anthropic warned [AI could soon self-improve](/?date=2026-06-08&category=reddit#item-d2a4f5930df0); a security **PSA** [flagged malware in ComfyUI](/?date=2026-06-08&category=reddit#item-a52146059301) **Claude-skill** custom nodes\n- Standout builds: natural-language **3D avatar** [control via a 0.6B Qwen model](/?date=2026-06-08&category=reddit#item-90a46a0391c8), plus a methodical **Qwen 3.6 27B** **DeepSWE** [benchmark](/?date=2026-06-08&category=reddit#item-ce486dbee8e1)",
"category_summary_html": "<p><strong>r/LocalLLaMA</strong> was dominated by <strong>Gemma 4</strong>. The day's top thread was <strong>llama.cpp</strong> merging <strong>MTP (multi-token prediction)</strong> support, with comments <a href=\"/?date=2026-06-08&category=reddit#item-f7fb09447ce6\" class=\"internal-link\" rel=\"noopener noreferrer\">reporting ~140 tok/s</a> on a 12GB RTX 40-series card.</p>\n<ul>\n<li>Builders pushed local limits: CPU-only <strong>Gemma-4-26B-A4B</strong> <a href=\"/?date=2026-06-08&category=reddit#item-403729badb4e\" class=\"internal-link\" rel=\"noopener noreferrer\">at ~7 tok/s</a>, and <strong>fp8 Gemma 4 31B</strong> <a href=\"/?date=2026-06-08&category=reddit#item-c6475c516b0f\" class=\"internal-link\" rel=\"noopener noreferrer\">reportedly matching</a> <strong>Claude Sonnet 4.6</strong> in an agentic harness, alongside <a href=\"/?date=2026-06-08&category=reddit#item-c662b00a3163\" class=\"internal-link\" rel=\"noopener noreferrer\">skepticism about</a> <strong>QAT quant</strong> quality</li>\n<li><strong>r/StableDiffusion</strong> ran an <strong>Ideogram 4</strong> evaluation wave—debating <a href=\"/?date=2026-06-08&category=reddit#item-9f910d0d6bdc\" class=\"internal-link\" rel=\"noopener noreferrer\">underrated vs overhyped</a>, sharing <a href=\"/?date=2026-06-08&category=reddit#item-17c5c9d873a0\" class=\"internal-link\" rel=\"noopener noreferrer\">censorship-bypass sampler tricks</a>, and griping about <a href=\"/?date=2026-06-08&category=reddit#item-4b424bbf4ea2\" class=\"internal-link\" rel=\"noopener noreferrer\">JSON-only prompting</a></li>\n<li><strong>r/ClaudeAI/Anthropic</strong> <a href=\"/?date=2026-06-08&category=reddit#item-4037ccf9bbb8\" class=\"internal-link\" rel=\"noopener noreferrer\">argued over new</a> <strong>usage limits</strong>, whether <a href=\"/?date=2026-06-08&category=reddit#item-3a34392260c9\" class=\"internal-link\" rel=\"noopener noreferrer\">models are</a> 'getting worse,' and the joy of <a href=\"/?date=2026-06-08&category=reddit#item-fda5317d40d7\" class=\"internal-link\" rel=\"noopener noreferrer\">building tiny single-user apps</a> with <strong>Claude Code</strong></li>\n<li><strong>AI governance</strong> drew cynicism as Anthropic warned <a href=\"/?date=2026-06-08&category=reddit#item-d2a4f5930df0\" class=\"internal-link\" rel=\"noopener noreferrer\">AI could soon self-improve</a>; a security <strong>PSA</strong> <a href=\"/?date=2026-06-08&category=reddit#item-a52146059301\" class=\"internal-link\" rel=\"noopener noreferrer\">flagged malware in ComfyUI</a> <strong>Claude-skill</strong> custom nodes</li>\n<li>Standout builds: natural-language <strong>3D avatar</strong> <a href=\"/?date=2026-06-08&category=reddit#item-90a46a0391c8\" class=\"internal-link\" rel=\"noopener noreferrer\">control via a 0.6B Qwen model</a>, plus a methodical <strong>Qwen 3.6 27B</strong> <strong>DeepSWE</strong> <a href=\"/?date=2026-06-08&category=reddit#item-ce486dbee8e1\" class=\"internal-link\" rel=\"noopener noreferrer\">benchmark</a></li>\n</ul>",
"themes": [
{
"name": "Security and Supply-Chain Risk",
"description": "Community warning about malware disguised as ComfyUI custom-node skills on GitHub.",
"item_count": 1,
"example_items": [],
"importance": 70
},
{
"name": "llama.cpp and MTP Advances",
"description": "Tooling milestones around llama.cpp, including merged Gemma 4 multi-token prediction support and resulting throughput gains.",
"item_count": 5,
"example_items": [],
"importance": 68
},
{
"name": "Gemma 4 and QAT Quantization",
"description": "Extensive community focus on Gemma 4 variants, especially quantization-aware training quants, their quality, comparisons, and bugs.",
"item_count": 9,
"example_items": [],
"importance": 62
},
{
"name": "AI governance, safety and existential risk",
"description": "Threads on AI self-improvement warnings, calls for development pauses, autonomous AI corporations, and accountability concerns.",
"item_count": 4,
"example_items": [],
"importance": 60
},
{
"name": "Ideogram 4 Evaluation Wave",
"description": "A large cluster of r/StableDiffusion posts reviewing, comparing, prompting, and troubleshooting Ideogram 4 against open models like Z-Image, Flux, and Qwen Image.",
"item_count": 12,
"example_items": [],
"importance": 60
},
{
"name": "Model Comparison and Benchmarks",
"description": "Threads comparing frontier and local models on real-world coding, agentic, and reasoning tasks, often critiquing benchmark validity.",
"item_count": 6,
"example_items": [],
"importance": 58
},
{
"name": "Local Hardware and Inference Accessibility",
"description": "Posts on mini PCs, multi-GPU rigs, Jetson clusters, Apple Silicon, and CPU-only MoE inference showing broadening hardware viability.",
"item_count": 8,
"example_items": [],
"importance": 58
},
{
"name": "Qwen 3.6 Performance",
"description": "Benchmarks and experience reports for Qwen 3.6 models across coding and laptop deployment scenarios.",
"item_count": 3,
"example_items": [],
"importance": 58
},
{
"name": "Uncensoring / Safety Filter Bypass",
"description": "Posts focused on circumventing Ideogram 4's safety filter via sampler tweaks, fine-tuning, and shared best practices.",
"item_count": 5,
"example_items": [],
"importance": 58
},
{
"name": "Model Behavior and Quality Perception",
"description": "Discussions about Opus 4.8, Sonnet 4.6, Mythos, instruction-following, quirky outputs, and debates over whether models are improving or degrading.",
"item_count": 10,
"example_items": [],
"importance": 55
}
],
"top_items": [
{
"id": "f7fb09447ce6",
"title": "llama.cpp Gemma4 MTP support merged!",
"content": "**Top community comments:**\n\n- (▲155) QAT + MTP let’s go!\n- (▲89) Now I'm getting 140 tok/s with Gemma 4 12B on 12GB VRAM (RTX 4070 Super) with the merged PR, QAT GGUF and MTP assistant / drafter 😄 Unsloth QAT GGUF: https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF MTP assistant /…\n- (▲40) Once again thanks u/am17an !!!\n- (▲37) Been watching this one closely! Compared to Qwen, the Gemma4 family seems to underperform on benchmarks, so it doesn't receive as much fanfare, but I've been giving 31B a go recently and it's been really nice, regardless…\n- (▲23) That delivers a speed increase of 4 times in average! (3 to 5 times depending on task). And it includes the thinking process. Just wow. Thank you to the developers. That's huge. On NVIDIA GB10 Grace Blackwell (Asus Ascen…\n- (▲19) Gemma 26b moe qat with mtp has me at 100 tokens per second in a 5060 ti 16 gb. Huge fan, doing some testing vs Qwen moe. They both have qwirks so it will probably be jumping between both.",
"url": "https://reddit.com/r/LocalLLaMA/comments/1tzbcyp/llamacpp_gemma4_mtp_support_merged/",
"author": "u/pinkyellowneon",
"published": "2026-06-07T08:53:17",
"source": "r/LocalLLaMA",
"source_type": "reddit",
"tags": [],
"summary": "Announcement that llama.cpp merged Gemma 4 MTP support, with comments reporting big throughput gains such as 140 tok/s on Gemma 4 12B on a 12GB RTX 4070 Super using QAT GGUF plus an MTP drafter, and praise for the contributor.",
"importance_score": 82,
"reasoning": "Major tooling milestone with the highest engagement in the batch and concrete performance results; central to current local-inference progress.",
"themes": [
"llama.cpp",
"MTP",
"Gemma 4",
"Performance",
"Quantization"
],
"continuation": {
"original_item_id": "e0cb141e9256",
"original_date": "2026-06-07",
"original_category": "reddit",
"original_title": "120 tok/s on 12GB VRAM with Gemma 4 12B QAT MTP",
"continuation_type": "follow_up",
"should_demote": false,
"reference_text": ""
},
"summary_html": "<p>Announcement that llama.cpp merged Gemma 4 MTP support, with comments reporting big throughput gains such as 140 tok/s on Gemma 4 12B on a 12GB RTX 4070 Super using QAT GGUF plus an MTP drafter, and praise for the contributor.</p>",
"content_html": "<p><strong>Top community comments:</strong></p>\n<ul>\n<li>(▲155) QAT + MTP let’s go!</li>\n<li>(▲89) Now I'm getting 140 tok/s with Gemma 4 12B on 12GB VRAM (RTX 4070 Super) with the merged PR, QAT GGUF and MTP assistant / drafter 😄 Unsloth QAT GGUF: https://huggingface.co/unsloth/gemma-4-12B-it-qat-GGUF MTP assistant /…</li>\n<li>(▲40) Once again thanks u/am17an !!!</li>\n<li>(▲37) Been watching this one closely! Compared to Qwen, the Gemma4 family seems to underperform on benchmarks, so it doesn't receive as much fanfare, but I've been giving 31B a go recently and it's been really nice, regardless…</li>\n<li>(▲23) That delivers a speed increase of 4 times in average! (3 to 5 times depending on task). And it includes the thinking process. Just wow. Thank you to the developers. That's huge. On NVIDIA GB10 Grace Blackwell (Asus Ascen…</li>\n<li>(▲19) Gemma 26b moe qat with mtp has me at 100 tokens per second in a 5060 ti 16 gb. Huge fan, doing some testing vs Qwen moe. They both have qwirks so it will probably be jumping between both.</li>\n</ul>"
},
{
"id": "403729badb4e",
"title": "You don't need a GPU to run gemma-4-26B-A4B",
"content": "I've been running LLMs on my old potato i5-8500 with 32GB of RAM and *no GPU* for awhile now, running up to 12B dense models which run slow but perfectly useable. But this Gemma-4-26B-A4B simply flies on this CPU - only machine using Koboldcpp on Linux. That's right, an old used $150 desktop computer is running state of the art LLMs with something like 7 T/s. Yeah, go ahead and scoff. You can brag about your super-rig that costs more than a used car, but I'm bragging about a crappy old desktop I bought of ebay running the same thing that costs less than a night out. I keep thinking about buying a GPU but it's beginning to look like it might not be necessary. These smaller models are amazing without a GPU.",
"url": "https://reddit.com/r/LocalLLaMA/comments/1tz5ffp/you_dont_need_a_gpu_to_run_gemma426ba4b/",
"author": "u/JackStrawWitchita",
"published": "2026-06-07T03:24:27",
"source": "r/LocalLLaMA",
"source_type": "reddit",
"tags": [],
"summary": "A user demonstrates running Gemma-4-26B-A4B on a GPU-less i5-8500 desktop with 32GB RAM at around 7 tokens/sec using KoboldCpp, arguing capable local models no longer require expensive hardware.",
"importance_score": 72,
"reasoning": "Highly engaging, practically inspiring CPU-only MoE inference demonstration with concrete numbers; strongly resonant accessibility theme.",
"themes": [
"CPU inference",
"MoE",
"Gemma 4",
"Accessibility"
],
"continuation": null,
"summary_html": "<p>A user demonstrates running Gemma-4-26B-A4B on a GPU-less i5-8500 desktop with 32GB RAM at around 7 tokens/sec using KoboldCpp, arguing capable local models no longer require expensive hardware.</p>",
"content_html": "<p>I've been running LLMs on my old potato i5-8500 with 32GB of RAM and *no GPU* for awhile now, running up to 12B dense models which run slow but perfectly useable. But this Gemma-4-26B-A4B simply flies on this CPU - only machine using Koboldcpp on Linux. That's right, an old used $150 desktop computer is running state of the art LLMs with something like 7 T/s. Yeah, go ahead and scoff. You can brag about your super-rig that costs more than a used car, but I'm bragging about a crappy old desktop I bought of ebay running the same thing that costs less than a night out. I keep thinking about buying a GPU but it's beginning to look like it might not be necessary. These smaller models are amazing without a GPU.</p>"
},
{
"id": "2fbaafa2fe7f",
"title": "Mythos 5: We're Not Ready",
"content": "https://x.com/pankajkumar_dev/status/2063625695285084419?s=20 - Mythos is extremely strong at SVG generation, producing highly detailed outputs that can take several minutes to create- It is exceptionally good at creating graphics, games, websites, and complex UI designs- Mythos is expected to set a new bar for web development and frontend generation, especially for design heavy workflows- It can also generate surprisingly good music through code- Anthropic's data suggests Mythos can achieve up to 52x training code speedups in certain optimization tasks, compared to roughly 4x for skilled humans on similar workloads- Mythos is expected to be extremely expensive, the public version will likely be a nerfed version of the current testing model",
"url": "https://reddit.com/r/ClaudeAI/comments/1tzg6dk/mythos_5_were_not_ready/",
"author": "u/Alternative_Jump_195",
"published": "2026-06-07T12:07:49",
"source": "r/ClaudeAI",
"source_type": "reddit",
"tags": [],
"summary": "High-engagement hype post about Anthropic's Mythos model touting strong SVG, frontend, game, and UI generation, possibly setting a new bar for web development.",
"importance_score": 72,
"reasoning": "Very high engagement and centers on a newly emerging Anthropic model (Mythos), making it a notable community focal point despite hype framing.",
"themes": [
"model capabilities",
"Mythos",
"frontend generation",
"hype"
],
"continuation": null,
"summary_html": "<p>High-engagement hype post about Anthropic's Mythos model touting strong SVG, frontend, game, and UI generation, possibly setting a new bar for web development.</p>",
"content_html": "<p>https://x.com/pankajkumar_dev/status/2063625695285084419?s=20 - Mythos is extremely strong at SVG generation, producing highly detailed outputs that can take several minutes to create- It is exceptionally good at creating graphics, games, websites, and complex UI designs- Mythos is expected to set a new bar for web development and frontend generation, especially for design heavy workflows- It can also generate surprisingly good music through code- Anthropic's data suggests Mythos can achieve up to 52x training code speedups in certain optimization tasks, compared to roughly 4x for skilled humans on similar workloads- Mythos is expected to be extremely expensive, the public version will likely be a nerfed version of the current testing model</p>"
},
{
"id": "90a46a0391c8",
"title": "Control a 3D avatar with language instead of buttons",
"content": "**Top community comments:**\n\n- (▲51) So this transforms a sentence into a sequence of mixamo animations? It does not generate animations, is that correct? Your text kinda omits that tbh, makes it look like much more than it is. I dont wanna downplay what yo…\n- (▲47) The \"director\" program, which converts natural language instructions into the action program, is a Qwen 3 0.6B model with a rank-64 LoRA adapter (about 22MB in size). Also, we will present our paper at AIware 2026 (Jul 6…\n- (▲38) If you make it work for American sign language, it would be crazy. Like normal people can talk to deaf people easily.\n- (▲11) So in short: you have a set of predefined animations like walk, wave, etc. All the LLM does is parse the text and, based on that text, spit out the most relevant action names (a chain of action names) which are later exe…\n- (▲8) so it's basically NLP mapping sentences to existing animation clips, not generating new animations? post wording is a bit misleading but still pretty interesting for what it is.\n- (▲7) This has been done already, in 2009, but better: https://flashmuseum.org/subservient-chicken/ https://en.wikipedia.org/wiki/The_Subservient_Chicken /s",
"url": "https://reddit.com/r/LocalLLaMA/comments/1tzgn87/control_a_3d_avatar_with_language_instead_of/",
"author": "u/yuntiandeng",
"published": "2026-06-07T12:25:28",
"source": "r/LocalLLaMA",
"source_type": "reddit",
"tags": [],
"summary": "A project showing a system that controls a 3D avatar via natural language, where a director module built on a Qwen 3 0.6B model with a 22MB rank-64 LoRA adapter converts instructions into animation action programs; paper to be presented at AIware 2026.",
"importance_score": 70,
"reasoning": "Impressive small-model project showcase with high engagement and academic backing; demonstrates practical local-model + LoRA applications.",
"themes": [
"Project showcase",
"LoRA",
"Qwen",
"Multimodal control"
],
"continuation": null,
"summary_html": "<p>A project showing a system that controls a 3D avatar via natural language, where a director module built on a Qwen 3 0.6B model with a 22MB rank-64 LoRA adapter converts instructions into animation action programs; paper to be presented at AIware 2026.</p>",
"content_html": "<p><strong>Top community comments:</strong></p>\n<ul>\n<li>(▲51) So this transforms a sentence into a sequence of mixamo animations? It does not generate animations, is that correct? Your text kinda omits that tbh, makes it look like much more than it is. I dont wanna downplay what yo…</li>\n<li>(▲47) The \"director\" program, which converts natural language instructions into the action program, is a Qwen 3 0.6B model with a rank-64 LoRA adapter (about 22MB in size). Also, we will present our paper at AIware 2026 (Jul 6…</li>\n<li>(▲38) If you make it work for American sign language, it would be crazy. Like normal people can talk to deaf people easily.</li>\n<li>(▲11) So in short: you have a set of predefined animations like walk, wave, etc. All the LLM does is parse the text and, based on that text, spit out the most relevant action names (a chain of action names) which are later exe…</li>\n<li>(▲8) so it's basically NLP mapping sentences to existing animation clips, not generating new animations? post wording is a bit misleading but still pretty interesting for what it is.</li>\n<li>(▲7) This has been done already, in 2009, but better: https://flashmuseum.org/subservient-chicken/ https://en.wikipedia.org/wiki/The_Subservient_Chicken /s</li>\n</ul>"
},
{
"id": "a52146059301",
"title": "PSA: A possible malware disguised as ComfyUI custom node Claude skills on GitHub",
"content": "I was looking for some Claude skills to help create a custom node for Comfyui. And I found two:https://github.com/jtydhr88/comfyui-custom-node-skillshttps://github.com/MusfiqurRahma/comfyui-custom-node-skills And noticed that the second is basically the same as the first even if it' is not a fork, and it's bigger! so I downloaded and checked the zip and found that there is another zip in one of the subfolders with three files: The cmd run \"unit.exe packages.txt\", and that text files is actually an obfuscated lua(?) script. Moreover, all the links in their newly created/modified README are changed to download this particular inside zip. I dunno how to report a Github repos, but I'm creating an account now to do it. In the meanwhile, i wanted to warn people from it.",
"url": "https://reddit.com/r/StableDiffusion/comments/1tzq7js/psa_a_possible_malware_disguised_as_comfyui/",
"author": "u/throwawaybox2026",
"published": "2026-06-07T18:34:44",
"source": "r/StableDiffusion",
"source_type": "reddit",
"tags": [],
"summary": "PSA warning about possible malware disguised as ComfyUI custom-node Claude skills on GitHub, with detailed analysis.",
"importance_score": 70,
"reasoning": "High-value community security warning with concrete evidence and strong engagement, directly protecting users.",
"themes": [
"security",
"malware",
"ComfyUI",
"supply chain"
],
"continuation": null,
"summary_html": "<p>PSA warning about possible malware disguised as ComfyUI custom-node Claude skills on GitHub, with detailed analysis.</p>",
"content_html": "<p>I was looking for some Claude skills to help create a custom node for Comfyui. And I found two:https://github.com/jtydhr88/comfyui-custom-node-skillshttps://github.com/MusfiqurRahma/comfyui-custom-node-skills And noticed that the second is basically the same as the first even if it' is not a fork, and it's bigger! so I downloaded and checked the zip and found that there is another zip in one of the subfolders with three files: The cmd run \"unit.exe packages.txt\", and that text files is actually an obfuscated lua(?) script. Moreover, all the links in their newly created/modified README are changed to download this particular inside zip. I dunno how to report a Github repos, but I'm creating an account now to do it. In the meanwhile, i wanted to warn people from it.</p>"
},
{
"id": "ce486dbee8e1",
"title": "Qwen 3.6 27B on DeepSWE",
"content": "Overview: It scored 2% (1.79% rounded up) It is 18/20th place scoring above Haiku 4.5 and Minimax M2.7 Full benchmark took 70 hours Average time per task 32m Average output tokens per task: 44k Perspectives: It scored suspiciously similar to 3.6 Plus and it really gets me wondering how the architecture of 3.6 Plus differs from 27B. Qwen 3.6 27B has a bad reputation in the community for being verbose. But surprisingly. The output tokens were on par or less to similar models. Methodology: Qwen 3.6 27B FP8 with BF16 KV cache, reasoning on and 262k context window on VLLM. Model ran on 1x RTX6000 pro Blackwell on RunPod. Ran with mini-swe agent harness on modal sandboxes. Ran 1 rollout per task instead of the official 4 to save time which is why images do not show a score range. Costs calculated by tasks completed within RunPod hourly rate. Codex 5.5xhigh was used to orchestrate and monitor the full benchmark run. src The best OS model Kimi-k2.6 is so far from the perf of the leading edge. Most cant even do Kimi locally and something like Qwen 3.6 27B is the local poor man's SOTA. It appears to take great size to perform at the leading edge. Models that start to be competitive tends to get closed source real quick. It doesn't feel like local will win. Feels more like a game of \"how badly will local lose\".",
"url": "https://reddit.com/r/LocalLLaMA/comments/1tzmq5y/qwen_36_27b_on_deepswe/",
"author": "u/SteppenAxolotl",
"published": "2026-06-07T16:13:48",
"source": "r/LocalLLaMA",
"source_type": "reddit",
"tags": [],
"summary": "A detailed benchmark of Qwen 3.6 27B on DeepSWE, reporting a ~2% score, runtime stats, token output observations, and comparison against similar models and the 3.6 Plus variant.",
"importance_score": 68,
"reasoning": "Methodical benchmark with concrete numbers and architectural speculation; strong engagement makes it a useful data point on coding/agentic performance.",
"themes": [
"Benchmarks",
"Qwen 3.6",
"Agentic coding"
],
"continuation": null,
"summary_html": "<p>A detailed benchmark of Qwen 3.6 27B on DeepSWE, reporting a ~2% score, runtime stats, token output observations, and comparison against similar models and the 3.6 Plus variant.</p>",
"content_html": "<p>Overview: It scored 2% (1.79% rounded up) It is 18/20th place scoring above Haiku 4.5 and Minimax M2.7 Full benchmark took 70 hours Average time per task 32m Average output tokens per task: 44k Perspectives: It scored suspiciously similar to 3.6 Plus and it really gets me wondering how the architecture of 3.6 Plus differs from 27B. Qwen 3.6 27B has a bad reputation in the community for being verbose. But surprisingly. The output tokens were on par or less to similar models. Methodology: Qwen 3.6 27B FP8 with BF16 KV cache, reasoning on and 262k context window on VLLM. Model ran on 1x RTX6000 pro Blackwell on RunPod. Ran with mini-swe agent harness on modal sandboxes. Ran 1 rollout per task instead of the official 4 to save time which is why images do not show a score range. Costs calculated by tasks completed within RunPod hourly rate. Codex 5.5xhigh was used to orchestrate and monitor the full benchmark run. src The best OS model Kimi-k2.6 is so far from the perf of the leading edge. Most cant even do Kimi locally and something like Qwen 3.6 27B is the local poor man's SOTA. It appears to take great size to perform at the leading edge. Models that start to be competitive tends to get closed source real quick. It doesn't feel like local will win. Feels more like a game of \"how badly will local lose\".</p>"
},
{
"id": "fda5317d40d7",
"title": "I'm a software engineer with a decade of experience, and the most fun things I've ever vibe coded with Claude Code have exactly one user: me",
"content": "I've vibe coded a pile of side projects with Claude Code. With AI and LLMs there is tons of emphasis on trying to build the next big thing. A startup, a SaaS, the app that finally takes off. I love that too. But I want to make the case for the opposite, because it's where I've had the most fun by a mile: Build the tiny, silly, perfect app that only you will ever use. Hear me out. Coding has always felt like magic to me. You type some letters into a file, run a couple commands, and the computer just does whatever you want. It's insane! It's why I got into coding in the first place. The catch used to be that the magic was SLOW. Building something real took a weekend, or weeks. So you only built the things that were worth all that time. That's over. You already know it's over, it's why you're here. An idea doesn't take days to validate anymore. It takes minutes. MINUTES. The bottleneck isn't the code, or the UI, or the design. It's your imagination now (jesus, did I really just say that?). And when building gets that cheap, the whole math on what's \"worth building\" flips: The most valuable thing you can build is no longer the thing a million people will pay for. It's the thing exactly one person will love. You. Like what? Anything you want, no matter how silly. Useful or not. Want a service that texts your parents and siblings reasons you're the favorite child, every single hour? DONE. Want a chrome extension that replaces every 'but' with 'butt'? EASY. Want a book recommender for your local used book store? CHILD'S PLAY. There are so many silly, stupid, and joyful pieces of software that never got made because the effort was juuust a little too much. But no longer! There's so much pressure to build the commercially viable thing, and yeah, I love that game too. But there's a whole other world sitting right next to it: building purely for yourself, for the fun of it, and almost nobody talks about it. That's not like magic. That IS magic. It's why I vibe code. So get out there and build your thing. The one only you want. Don't make it scale, don't call it a startup, just build it and see how good it feels. If this is your kind of thing, I write about all things vibe coding about once a week over at vibeblog.net. Happy Sunday and keep on vibin'!",
"url": "https://reddit.com/r/ClaudeAI/comments/1tzf5jl/im_a_software_engineer_with_a_decade_of/",
"author": "u/thelocalnative",
"published": "2026-06-07T11:28:33",
"source": "r/ClaudeAI",
"source_type": "reddit",
"tags": [],
"summary": "A veteran software engineer makes the case for building tiny, personal, single-user apps with Claude Code for fun rather than chasing startups.",
"importance_score": 65,
"reasoning": "High engagement, well-written reflective essay resonating with the community about the joy of personal coding projects.",
"themes": [
"vibe coding",
"developer philosophy",
"personal projects"
],
"continuation": null,
"summary_html": "<p>A veteran software engineer makes the case for building tiny, personal, single-user apps with Claude Code for fun rather than chasing startups.</p>",
"content_html": "<p>I've vibe coded a pile of side projects with Claude Code. With AI and LLMs there is tons of emphasis on trying to build the next big thing. A startup, a SaaS, the app that finally takes off. I love that too. But I want to make the case for the opposite, because it's where I've had the most fun by a mile: Build the tiny, silly, perfect app that only you will ever use. Hear me out. Coding has always felt like magic to me. You type some letters into a file, run a couple commands, and the computer just does whatever you want. It's insane! It's why I got into coding in the first place. The catch used to be that the magic was SLOW. Building something real took a weekend, or weeks. So you only built the things that were worth all that time. That's over. You already know it's over, it's why you're here. An idea doesn't take days to validate anymore. It takes minutes. MINUTES. The bottleneck isn't the code, or the UI, or the design. It's your imagination now (jesus, did I really just say that?). And when building gets that cheap, the whole math on what's \"worth building\" flips: The most valuable thing you can build is no longer the thing a million people will pay for. It's the thing exactly one person will love. You. Like what? Anything you want, no matter how silly. Useful or not. Want a service that texts your parents and siblings reasons you're the favorite child, every single hour? DONE. Want a chrome extension that replaces every 'but' with 'butt'? EASY. Want a book recommender for your local used book store? CHILD'S PLAY. There are so many silly, stupid, and joyful pieces of software that never got made because the effort was juuust a little too much. But no longer! There's so much pressure to build the commercially viable thing, and yeah, I love that game too. But there's a whole other world sitting right next to it: building purely for yourself, for the fun of it, and almost nobody talks about it. That's not like magic. That IS magic. It's why I vibe code. So get out there and build your thing. The one only you want. Don't make it scale, don't call it a startup, just build it and see how good it feels. If this is your kind of thing, I write about all things vibe coding about once a week over at vibeblog.net. Happy Sunday and keep on vibin'!</p>"
},
{
"id": "17c5c9d873a0",
"title": "Ideogram 4: a solution for removing the annoying censorship has been found.",
"content": "Actually, there are two working methods. In both cases, the block message almost never appears. Method 1Shift only the first sigma step by +0.005.In some cases, +0.01 may be needed.This slightly moves the starting point of the trajectory away from what the model expects. All other sigma steps should remain exactly the same as in the default settings. Method 2 — preferred Increase the initial noise by x2.In some cases, x3 may be needed.The effect is similar: it pushes the model away from its usual starting trajectory. ImportantBoth methods work properly only with an LCM sampler.Based on the behavior, LCM seems able to correct the trajectory after the initial deviation, so the image still converges properly.Other samplers will break the image. Required nodes: Noise Math from the More Math addon SamplerLCMCustom from Extra Samplers Custom Sigmas from KJ Nodes Sigmas2 Mult from RES4LYF These hacks remove the gray censorship square, but they do not fix all of the model’s behavior issues. Ideogram 4 does not seem to be well-trained for very short prompts. Even if one-word prompts no longer trigger censorship, the model may still fail to follow them accurately. Prompts made of several sentences already work much better, even without JSON formatting. For this part of the workflow, I use two samplers sequentially. The first sampler runs the initial high-sigma stage with LCM and the modified noise/sigma setup. Then its output is passed into the second sampler, which continues the remaining steps with the normal sigma range. The split point in SplitSigmas can be adjusted depending on the situation. In my tests, values between 1 and 3 usually work best. This hack works — at least for me 🙂But I still think we should keep looking for other ways to get rid of this censorship. Happy generating, everyone.",
"url": "https://reddit.com/r/StableDiffusion/comments/1tz4fnf/ideogram_4_a_solution_for_removing_the_annoying/",
"author": "u/TRlG0N",
"published": "2026-06-07T02:27:34",
"source": "r/StableDiffusion",
"source_type": "reddit",
"tags": [],
"summary": "A detailed post claiming two working methods to remove Ideogram 4's censorship: shifting the first sigma step by a small amount, or doubling/tripling the initial noise to push the model away from its expected trajectory. Highly upvoted with many comments.",
"importance_score": 62,
"reasoning": "High-engagement post providing concrete, reproducible technical sampler-level techniques to bypass the safety filter, the standout educational item in this batch.",
"themes": [
"Uncensoring / Safety Filter Bypass",
"Technical Research / Novel Methods",
"Ideogram 4 Experimentation & Showcases"
],
"continuation": null,
"summary_html": "<p>A detailed post claiming two working methods to remove Ideogram 4's censorship: shifting the first sigma step by a small amount, or doubling/tripling the initial noise to push the model away from its expected trajectory. Highly upvoted with many comments.</p>",
"content_html": "<p>Actually, there are two working methods. In both cases, the block message almost never appears. Method 1Shift only the first sigma step by +0.005.In some cases, +0.01 may be needed.This slightly moves the starting point of the trajectory away from what the model expects. All other sigma steps should remain exactly the same as in the default settings. Method 2 — preferred Increase the initial noise by x2.In some cases, x3 may be needed.The effect is similar: it pushes the model away from its usual starting trajectory. ImportantBoth methods work properly only with an LCM sampler.Based on the behavior, LCM seems able to correct the trajectory after the initial deviation, so the image still converges properly.Other samplers will break the image. Required nodes: Noise Math from the More Math addon SamplerLCMCustom from Extra Samplers Custom Sigmas from KJ Nodes Sigmas2 Mult from RES4LYF These hacks remove the gray censorship square, but they do not fix all of the model’s behavior issues. Ideogram 4 does not seem to be well-trained for very short prompts. Even if one-word prompts no longer trigger censorship, the model may still fail to follow them accurately. Prompts made of several sentences already work much better, even without JSON formatting. For this part of the workflow, I use two samplers sequentially. The first sampler runs the initial high-sigma stage with LCM and the modified noise/sigma setup. Then its output is passed into the second sampler, which continues the remaining steps with the normal sigma range. The split point in SplitSigmas can be adjusted depending on the situation. In my tests, values between 1 and 3 usually work best. This hack works — at least for me 🙂But I still think we should keep looking for other ways to get rid of this censorship. Happy generating, everyone.</p>"
},
{
"id": "4037ccf9bbb8",
"title": "Claude's new usage limits are insane.",
"content": "**Top community comments:**\n\n- (▲295) Well.. I mean, yeah. Using the most token-consuming model on the most token-consuming thinking level on the most token-consuming context level is going to lead to consuming a lot of tokens. I wouldn't say it's Anthropic'…\n- (▲53) I think at some point we developers need to start taking some responsibility for the tools and processes we use. Along with their associated costs. If you were trying to till your backyard garden and went to Home Depot t…\n- (▲43) Don't use ultracode unless you have a few grand to spare\n- (▲19) Holy crap. Guy uses ultra code and wonders why so many tokens are being burned. That’s literally how ultra code is designed. It is not token efficient, BY DESIGN, and intended to be used for a narrow range of tasks. For…\n- (▲10) That 1 prompt was over 1M tokens alone.\n- (▲4) \"I told it to do overkill and it did overkill and now I'm mad about it\" What are we doing? What's going on?",
"url": "https://reddit.com/r/ClaudeAI/comments/1tzwrxs/claudes_new_usage_limits_are_insane/",
"author": "u/TheTeddyFlame3",
"published": "2026-06-07T23:42:51",
"source": "r/ClaudeAI",
"source_type": "reddit",
"tags": [],
"summary": "A complaint about Claude's new usage limits, with top comments pushing back that using the most token-heavy model, thinking level, and context naturally consumes tokens, and arguing users should take responsibility for tool costs.",
"importance_score": 62,
"reasoning": "High engagement on a recurring pricing/limits tension with substantive counterarguments; reflects broad user sentiment.",
"themes": [
"Claude pricing",
"Usage limits",
"Cost management"
],
"continuation": null,
"summary_html": "<p>A complaint about Claude's new usage limits, with top comments pushing back that using the most token-heavy model, thinking level, and context naturally consumes tokens, and arguing users should take responsibility for tool costs.</p>",
"content_html": "<p><strong>Top community comments:</strong></p>\n<ul>\n<li>(▲295) Well.. I mean, yeah. Using the most token-consuming model on the most token-consuming thinking level on the most token-consuming context level is going to lead to consuming a lot of tokens. I wouldn't say it's Anthropic'…</li>\n<li>(▲53) I think at some point we developers need to start taking some responsibility for the tools and processes we use. Along with their associated costs. If you were trying to till your backyard garden and went to Home Depot t…</li>\n<li>(▲43) Don't use ultracode unless you have a few grand to spare</li>\n<li>(▲19) Holy crap. Guy uses ultra code and wonders why so many tokens are being burned. That’s literally how ultra code is designed. It is not token efficient, BY DESIGN, and intended to be used for a narrow range of tasks. For…</li>\n<li>(▲10) That 1 prompt was over 1M tokens alone.</li>\n<li>(▲4) \"I told it to do overkill and it did overkill and now I'm mad about it\" What are we doing? What's going on?</li>\n</ul>"
},
{
"id": "c6475c516b0f",
"title": "Gemma4_31b_fp8 keeping up with Sonnet_4.6_medium in my harness.",
"content": "Cypher queries for graph traversal (neo4j) Entity extraction from text chunks (web query, graph query, vectors) Agentic tool calling (Skills selection / successful running in Pi) Code writing (Python) Synthesis/summarization of multi-vector-retrieval Gemma/Qwen in FP8. This brought me joy",
"url": "https://reddit.com/r/LocalLLaMA/comments/1tzw207/gemma4_31b_fp8_keeping_up_with_sonnet_46_medium/",
"author": "u/knob-0u812",
"published": "2026-06-07T23:06:48",
"source": "r/LocalLLaMA",
"source_type": "reddit",
"tags": [],
"summary": "A user reports that Gemma 4 31B in FP8 keeps pace with Sonnet 4.6 in their RAG/agentic harness across Cypher graph queries, entity extraction, tool calling, code, and synthesis.",
"importance_score": 60,
"reasoning": "Concrete hands-on local-model performance comparison across real agentic tasks with solid engagement, valuable to the local-LLM community.",
"themes": [
"local LLMs",
"model comparison",
"agentic RAG"
],
"continuation": null,
"summary_html": "<p>A user reports that Gemma 4 31B in FP8 keeps pace with Sonnet 4.6 in their RAG/agentic harness across Cypher graph queries, entity extraction, tool calling, code, and synthesis.</p>",
"content_html": "<p>Cypher queries for graph traversal (neo4j) Entity extraction from text chunks (web query, graph query, vectors) Agentic tool calling (Skills selection / successful running in Pi) Code writing (Python) Synthesis/summarization of multi-vector-retrieval Gemma/Qwen in FP8. This brought me joy</p>"
}
]
}
}
}Comments
Ready-made automations for this
Workflows from the Neura Market marketplace related to this Perplexity resource
- Automate Daily News Summaries with OpenAI and Perplexity AI via Zalo and Telegramn8n · $14.99 · Related topic
- Automate AI News Video Content Creation with Perplexity and OpenAIn8n · $14.99 · Related topic
- Automate Daily AI News Digest with Perplexity and Gmailn8n · $9.99 · Related topic
- Automate Daily News Collection and Storage with Perplexity AI and MongoDBn8n · $9.99 · Related topic