Generative AI: 22nd April 2025

• OpenAI has introduced new reasoning models o3 and o4-mini, which show increased hallucination rates despite improved performance in coding and math tasks.
• Google's Gemini 2.5 Flash introduces 'thinking budgets' that enable businesses to optimize AI reasoning costs, achieving up to 600% pricing efficiency while maintaining performance across benchmarks.
• Databricks CEO Ali Ghodsi and Anthropic CEO Dario Amodei will hold a virtual fireside chat discussing the future of domain-specific AI agents and their partnership enhancing AI technologies.
• Cohere has released Embed 4, a multimodal AI model that can process 200-page documents with a 128,000 token context window, supporting over 100 languages for enterprise-level search.
• Microsoft Research reveals that longer reasoning chains in AI models don't guarantee better performance, highlighting cost and efficiency issues across models like GPT-4o, Claude 3.7, and DeepSeek.
• AI adoption is set to accelerate as infrastructure enablers like durable cloud workflows, resource management tools, and advanced DevOps methodologies support development of reliable, scalable AI-native applications.
• The reality of agentic AI faces enterprise hurdles including data silos and integration gaps, requiring groundwork in infrastructure and governance that could take a decade for true adoption.

Gemma 3 QAT Models (simonwillison.net, 2025-04-19). Google's Gemma 3 optimized with Quantization-Aware Training reduces memory usage for local deployment on consumer GPUs, enabling efficient execution of models like Gemma 3 27B via Ollama and MLX tools

Beyond the Hype: Should fully autonomous AI agents be developed? by Oliver Cronk (blog.scottlogic.com, 2025-04-15). Scott Logic's team discusses agentic AI's applications and limitations, showcasing InferLLM and InferESG, while addressing the ethical considerations and the balance of human oversight necessary for autonomous AI agents in 2025

These are not the same (tante.cc, 2025-04-15). Generative AI is deemed a costly bubble; unlike traditional digital services, it requires constant expensive updates. Infrastructure may remain post-bubble, yet generative models risk becoming obsolete without continual maintenance

If LLMs Can Code, Why Are We Building More IDEs? (poppastring.com, 2025-04-18). Large language models (LLMs) may not replace coding; instead, tools like Cursor, Windsurf, and Firebase Studio indicate a growth in AI-powered IDEs enhancing how developers interact with code

Llama 4 smells bad (fastml.com, 2025-04-17). Llama 4, Meta's latest model, faces criticism for underperformance, deceptive metrics, and benchmark manipulation, with issues in context handling and parameter reporting amid staff resignations over ethical concerns

#248 Pedro Domingos: How Connectionism Is Reshaping the Future of Machine Learning (aneyeonai.libsyn.com, 2025-04-17). Pedro Domingos discusses Connectionism and its impact on neural networks, highlighting the evolution from 1940s neural networks to transformers, the significance of Backpropagation, and the challenges faced by reinforcement and unsupervised learning

Packing Input Frame Context in Next-Frame Prediction Models for Video Generation (lllyasviel.github.io, 2025-04-19). Next-frame prediction models use FramePack scheduling to optimize GPU resource allocation for video generation, addressing issues like drifting while enabling flexible compression rates across input frames with O(1) complexity

12-factor Agents: Patterns of reliable LLM applications (github.com, 2025-04-15). Explore the principles of building reliable LLM applications through concepts like Directed Graphs (DGs), error recovery, and modular designs, and discuss frameworks to enhance agent capabilities in production environments

Tuning Local LLMs With RAG Using Ollama and Langchain (itsfoss.com, 2025-04-20). Explore how to utilize Retrieval-Augmented Generation (RAG) with Ollama and Langchain to create a PDF-based project enabling users to upload documents and ask questions using a local LLM

Configuring Unsloth on Linux for LLM Fine Tuning (jjfumero.github.io, 2025-04-17). Unsloth is a Python framework for optimizing fine-tuning of LLMs on NVIDIA GPUs, offering utilities for quantization and performance. Installation involves Python setup and using tools like spack and pip for necessary packages

Writing an LLM from scratch, part 12 -- multi-head attention (gilesthomas.com, 2025-04-21). The post explores multi-head attention in LLMs, detailing how it enhances representations by using multiple attention heads simultaneously while implementing it in PyTorch with examples of neural network components

Writing an LLM from scratch, part 11 -- batches (gilesthomas.com, 2025-04-19). Batch processing improves efficiency in training large language models, utilizing higher-order tensors for self-attention and causal attention, implemented in PyTorch with dropout and flexible configuration options

Hosting Your Own Coding LLM (prvn.sh, 2025-04-18). Explore the deployment of private LLMs for code completion, assessing GPU requirements, latency issues, workload estimates, and performance metrics with tools like vLLM and vegeta in a practical scenario

Implementing Function Calling LLMs without Fear (bbengfort.github.io, 2025-04-16). A discussion on implementing Function Calling LLMs for AI agents, addressing security concerns and task validation while ensuring effective external function usage without additional training

Stop Blaming the LLM-as-Judge; Fix Your Process Instead (eugeneyan.com, 2025-04-20). Effective evals for AI products utilize the scientific method, eval-driven development, and AI output monitoring, addressing failures through data analysis, hypothesis testing, experimentation, and automated evaluators while emphasizing human oversight

An LLM Codegen Hero's Journey (harper.blog, 2025-04-17). A developer's journey through LLM code generation tools, exploring AI-assisted autocomplete, Copilot, Claude, AI-enabled IDEs like Cursor, and the evolution towards agentic coding for improved productivity

Augmented Coding: an Experience Report (jessitron.com, 2025-04-20). Auggie, an AI coding assistant, facilitates cross-language implementation and automates scaffolding, but also presents challenges in error management and requires increased oversight and discipline from developers

Of LLMs and Men (log.beshr.com, 2025-04-19). Explores the realistic role of LLMs and AI in workflows, comparing them to CAD's integration into engineering, and emphasizing their function as language tools that enhance tasks without replacing human decisions

Good practices for AI-assisted development from a live protein calculator demo (ericmjl.github.io, 2025-04-19). Eric J. Ma demonstrates a protein mass spectrometry calculator, highlighting the importance of standardization, AI assistance, and effective collaboration to enhance coding practices and maintain human creativity in software development

Testing Will Become More Important, Not Less (filiphric.com, 2025-04-16). As LLMs like Nut.new streamline code generation, testing is increasingly vital. Automated testing is integrated with code creation, alongside the importance of skilled manual reviews and observability tools for maintaining software quality

llm-fragments-github 0.2 (simonwillison.net, 2025-04-20). The llm-fragments-github 0.2 plugin introduces a new 'issue' fragment type for fetching entire GitHub issue threads as Markdown, enhancing feedback capabilities for LLM projects using models like Gemini 2.5 Pro and OpenAI o3

LLM-first Web Framework (blog.mgechev.com, 2025-04-19). An innovative LLM-first framework based on Revolt simplifies UI coding for AI by using reactive functions for both static and dynamic values, enhancing code generation efficiency for web applications

Released a new tool: llm-url-markdown (saeedesmaili.com, 2025-04-16). Saeed Esmaili released 'llm-url-markdown', a new Python plugin for the CLI tool 'llm' that fetches markdown content from web URLs and integrates it as additional context for long context models

“GraphRAG: Because Your Credit Risk Model Deserves a Brain Upgrade” (medium.com/neo4j, 2025-04-16). GraphRAG leverages large language models and graph databases to enhance credit risk analysis by integrating structured and unstructured data, improving decision-making, and enabling real-time insights on market conditions

$1000 Local Ai Home Server on Z440 with 3090 (digitalspaceport.com, 2025-04-18). DIY $1000 local AI home server using HP Z440 workstation, NVIDIA 3090 GPU, exploring performance benchmarks for large language models like Gemma 3 and QwQ

I’ve Felt Like a Hallucinating LLM (4gravitons.com, 2025-04-18). Exploring the mechanics of Large Language Models (LLMs), the article highlights their storytelling capabilities, the paradox of 'hallucinations,' and the need for checks in AI-driven advice and narratives

European AI (jlelse.blog, 2025-04-15). Jan-Lukas Else migrates from OpenAI API to Scaleway’s Generative API for blog functionalities, utilizing llama-3.3-70b-instruct for summaries and pixtral-12b-2409 for image descriptions, aiming for future audio transcription support

LLM-enhanced email archive (nelsonslog.wordpress.com, 2025-04-17). Nelson seeks to develop a private LLM trained on 20 years of email archives, exploring options like RAG, SillyTavern, and cloud machines while ensuring email privacy

The Loom and the Thread: Weaving Intelligence with Pattern Languages and LLMs (medium.com/intuitionmachine, 2025-04-21). The synergy of Large Language Models and pattern languages enhances problem-solving in design, fostering innovation and explainable AI through structured frameworks and generative capabilities for rapid, reliable solutions

autoregressive queens of failure (ghuntley.com, 2025-04-21). Exploring autoregressive failure in AI coding assistants, the importance of context windows, and best practices for optimal tool usage in programming to prevent irrelevant outputs and enhance performance

Liquid: Language models are scalable and unified multi-modal generators (foundationvision.github.io, 2025-04-15). Liquid integrates visual and textual data in a unified model, eliminating the need for external visual embeddings and demonstrating scalable performance in multimodal generation tasks

Sparsely-gated Mixture Of Experts (MoE) (eli.thegreenplace.net, 2025-04-18). Sparsely-gated Mixture of Experts (MoE) architecture enhances transformer efficiency by enabling selective expert usage based on token relevance, utilizing techniques like top-k selection and softmax scoring

Will it Run Llama 2? Now DOS Can (hackaday.com, 2025-04-19). Yeo Kheng Meng successfully runs a stripped-down version of Llama 2 on vintage DOS computers using Andreq Karpathy’s Llama2.c, demonstrating the compatibility of LLMs with retro hardware like 486 and Pentium machines

Mixture of Experts (nonint.com, 2025-04-18). Mixture of Experts (MoE) Transformers replace traditional MLP layers with MoE layers, enhancing compute efficiency through sparsity while facing challenges in low-latency and memory-bound environments during inference

Making LLMs Useful with Function Calls and Embeddings (standard-out.com, 2025-04-16). Large Language Models (LLMs) like Google's Gemini can enhance their capabilities via function calls and external APIs, such as the US Congress API, improving their ability to retrieve and process information

Inner Loop Agents (timkellogg.me, 2025-04-19). Inner loop agents enable LLMs to utilize tools directly, enhancing concurrency in problem solving while fostering optimal tool use. Key concepts include LLM Operating Software like Ollama, reinforcement learning, and Model Context Protocol (MCP)

Introducing HELMET (huggingface.co, 2025-04-16). HELMET offers a new benchmark for evaluating long-context language models, utilizing diverse tasks, controllable parameters, and model-based evaluations, enhancing performance understanding across complex applications

Load-Testing LLMs Using LLMPerf (towardsdatascience.com, 2025-04-18). Learn how to load-test Large Language Models (LLMs) using LLMPerf, focusing on token-based metrics, and apply it in Amazon Bedrock for real-time performance evaluation and benchmarking

ELT-Bench: Evaluating AI Agents on Automating Data Pipelines (medium.com/@danieldkang, 2025-04-16). ELT-Bench evaluates AI agents on automating Extract-Load-Transform (ELT) data pipelines, revealing a low 3.9% success rate in transformation tasks and challenging current frameworks like Spider-Agent and SWE-Agent across six large language models

GPT-4.1 and o4-mini: Is OpenAI Overselling Long-Context? (blog.getzep.com, 2025-04-17). OpenAI's GPT-4.1 and o4-mini models were evaluated against the LongMemEval benchmark, highlighting that context size alone is insufficient for effective memory use, with Zep's knowledge graph approach outperforming in several aspects

Quick notes on the danger of LLM benchmarks (blog.rinesi.com, 2025-04-21). LLM benchmarks inadequately assess model performance due to unbounded failure modes, leading to dangerous misinterpretations of improvements amidst serious risks like misinformation in critical contexts

Exact Learning Dynamics of In-Context Learning in Linear Transformers and Its Application to Non-Linear Transformers (arxiv:cs, 2025-04-17). An analytical characterization of in-context learning emergence in linear transformers, revealing timescale separation, nonlinear learning behaviors, and theoretical tools like spectral rank dynamics to explain sudden ICL and delayed generalization phenomena

Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective (arxiv:cs, 2025-04-18). Transformers can approximate Holder continuous functions with limited layers and widths using self-attention and feedforward layers, demonstrating their strong expressive capability without relying on contextual mapping

It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization (arxiv:cs, 2025-04-17). Miras framework designs deep learning architectures using associative memories, attentional bias, and retention gates, leading to novel sequence models (Moneta, Yaad, Memora) outperforming Transformers in various tasks

The State of Reinforcement Learning for LLM Reasoning (sebastianraschka.com, 2025-04-19). Exploration of recent advancements in reinforcement learning, particularly focusing on reasoning improvements in LLMs using techniques like chain-of-thought reasoning, proximal policy optimization, and the emergence of models with enhanced reasoning capabilities

LLM Unlearning Benchmarks are Weak Measures of Progress (blog.ml.cmu.edu, 2025-04-18). Machine unlearning aims to delete unwanted data from LLMs without full retraining. Current benchmarks like TOFU and WMDP fail to assess this accurately, leading to misleading evaluations of progress in unlearning methods

Deep literature reviews: an application of fine-tuned language models to migration research (arxiv:stat, 2025-04-17). A hybrid framework combining bibliometric methods and fine-tuned large language models enhances literature reviews, revealing trends in climate-induced migration while highlighting overlooked factors like air pollution and infectious diseases

The Memorization Problem: Can We Trust LLMs' Economic Forecasts? (arxiv:q-fin, 2025-04-20). Large language models exhibit perfect recall of past economic data, raising concerns about their reliability for economic forecasts. Their ability to reconstruct masked entities highlights memorization issues that could mislead backtesting strategies

PlanGlow: Personalized Study Planning with an Explainable and Controllable LLM-Driven System (arxiv:cs, 2025-04-16). PlanGlow utilizes LLMs to create personalized study plans, enhancing usability, explainability, and controllability in self-directed learning, validated through user surveys, interviews, and comparisons with GPT-4o and Khan Academy's Khanmigo

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions (arxiv:stat, 2025-04-20). Survey highlights Knowledge Distillation (KD) and Dataset Distillation (DD) techniques, including task-specific alignment and generative synthesis, to compress Large Language Models while maintaining reasoning capabilities and linguistic diversity for efficient deployment in various domains

Teuken-7B-Base and Teuken-7B-Instruct: Towards European LLMs (arxiv.org, 2025-04-15). Teuken-7B-Base and Teuken-7B-Instruct are new European LLMs, supported by the Simons Foundation, aiming to enhance computational language capabilities within European contexts

Inferring the Phylogeny of Large Language Models (arxiv.org, 2025-04-19). Research on PhyloLM introduces methods for inferring the phylogeny of large language models while predicting their performance in benchmarks, utilizing advanced computational techniques in the field of computation and language

Continual training of Llama-3.1-8B for 809B tokens (severelytheoretical.wordpress.com, 2025-04-21). Continual training of Llama-3.1-8B utilized 64-node setups, hybrid sharding data parallelism, tensor parallelism, and bf16 mixed precision to process 809B tokens with a focus on optimizing large-scale training efficiency

Paper Review: M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models (andlukyane.com, 2025-04-21). M1 is a hybrid RNN model that utilizes distillation and reinforcement learning, achieving 3x faster inference and improved accuracy on benchmarks like AIME and MATH while implementing the Mamba architecture

Flowco: Rethinking Data Analysis in the Age of LLMs (arxiv:stat, 2025-04-18). Flowco is a mixed-initiative system that integrates LLMs into visual dataflow programming, aiding analysts, especially those with limited coding skills, in quickly authoring, debugging, and refining data analyses