Turing Post is reader-supported publication. Upgrade to support us. Thank you
Lately, weโve seen many ideas making promising comebacks to further fuel our already unstoppable race toward superintelligent computers and reasoning robots. Today, letโs talk about program synthesis and why it might be a missing piece in the puzzle toward AGI.
The first time program synthesis captured our attention was in 2019, when Franรงois Chollet published his brilliant paper "On the Measure of Intelligence." In it, he introduced the Abstraction and Reasoning Corpus (ARC), a benchmark designed to evaluate human-like general intelligence. There, he emphasized the limitations of deep learning for reasoning and generalization, and argued that program synthesis could serve as a key step toward creating truly intelligent systems. By allowing AI to generate solutions dynamically โ writing small programs tailored to specific tasks โ program synthesis shifts the focus from static task performance to adaptability and reasoning.
Fast forward to 2025, ARC-AGI has become one of the primary benchmarks for evaluating models aspiring to AGI. Franรงois Chollet is taking his ideas even further: launching Ndea, a lab dedicated to advancing AGI by exploring the fascinating hybrid of deep learning and program synthesis. This combination, he believes, could unlock new efficiencies, enabling AI to reason abstractly, learn from minimal data, and solve a broader range of problems than ever before. Letโs see what program synthesis is, where it comes from and how it can be combined with deep learning.
History: of course, we can trace Program Synthesis way backโฆ
to our dearest Alan Turing.
Early Years: In 1945, Alan Turing envisioned machines capable of generating programs autonomously. But the formal roots emerged in 1957 when Alonzo Church proposed synthesizing circuits from mathematical requirements, an idea now called "Church's Problem."
Formal Foundations (1960s - 1980s): The field gained a stronger theoretical footing with contributions like the automata-theoretic approach by Bรผchi and Landweber (1969) and the work of Manna and Waldinger (c. 1980). This period focused on developing formal methods for program synthesis, often based on logical reasoning and deductive techniques. ย
Pragmatic Evolution (1990s-2010s): Program synthesis evolved to incorporate more practical approaches, including sketching (introduced in 2006 with the SKETCH system by Armando Solar-Lezama), where programmers provide partial programs with holes that are automatically filled, and programming-by-examples (PBE) (popularized in the 2010s with tools like Flash Fill in Excel, developed by Sumit Gulwani, which automates data transformations by learning patterns from user-provided input-output examples.)
Modern Resurgence (2010s-2020s): The 21st century witnessed a renewed interest in program synthesis, particularly within the formal verification community. This led to advancements like Syntax-guided synthesis (SyGuS), which combines logical specifications with grammatical constraints to guide the synthesis process.
For many years, program synthesis and machine learning have their own independent trajectories, but now we see their collaboration gaining momentum. And there are a few factors that made their integration more feasible and promising:
Increased Computational Power: GPUs! Providing enough (and more more more) computational resources to handle the complexity of both program synthesis and machine learning algorithms allows researchers to explore more sophisticated techniques and tackle larger problems.
Availability of Large Datasets: The rise of big data and the proliferation of online code repositories provided the raw material for training machine learning models used in program synthesis. These datasets enabled the development of data-driven approaches to guide the search process, learn from examples, and generalize to new situations.
Cross-fertilization of Ideas: Software developers transitioning into the ML world brought their specialized knowledge and passion, applying it to various domains.
As and example, in 2023, MIT launched a course โIntroduction to program synthesisโ describing it as โa new field at the intersection of programming languages, formal methods and AIโ.
Cholletโs Vision: The Case for Program Synthesis
Franรงois Chollet has long argued that program synthesis is a crucial step toward artificial general intelligence (AGI). He critiques the limitations of deep learning โ its dependence on massive datasets, its brittleness, and its struggles with reasoning and generalization. Unlike deep learning, which excels at recognizing patterns but often fails to adapt to novel problems, program synthesis allows AI to generate solutions by reasoning abstractly, offering a more adaptable and scalable approach.
In his landmark work On the Measure of Intelligence, Chollet emphasized separating the process of intelligence (the system that generates solutions) from the output (the specific solutions themselves). He argued that program synthesis โ a method where AI creates small, task-specific programs โ is an ideal way to evaluate intelligence. This approach shifts focus from static task performance to the ability to adapt dynamically to unseen challenges.
Deep Learning Meets Program Synthesis
Chollet envisions program synthesis as a complementary approach to deep learning, rather than a replacement. While deep learning models can guide program synthesis by narrowing the search space and handling large-scale pattern recognition, program synthesis brings reasoning and abstraction to the table. This hybrid approach could unlock efficiencies and tackle problems that are currently beyond AIโs reach.
In pursuit of this vision, Franรงois Chollet and Mike Knoop founded Ndea, an AI research lab focused on advancing AGI through program synthesis. Rooted in Chollet's belief that abstraction is key to intelligence, Ndea aims to develop adaptable AI systems that overcome deep learning's limitations by leveraging symbolic manipulation and code generation for flexible reasoning and generalization.
Weโll be following Ndea closely, as AGI isnโt a challenge that can be solved from a single angle. It requires integration and collaboration across various scientific fields. Itโs exciting to see new aspects of AGI being tackled with fresh approaches.
Curated Collections
Has this email been forwarded to you? We highly recommend signing up!
News from The Usual Suspects ยฉ
AIโs New Best Friend: Journalism
Mistral AI partners with AFP, securing access to 40+ years of archives to power its chatbot, Le Chat. OpenAI, not to be outdone, teams up with Axios, supporting its localโs newsroom expansion in four U.S. cities. Googleโs Gemini app also tries to keep up, leveraging APโs real-time feeds for fresher, trusted content.
From archives to real-time updates, AIโs marriage with journalism is rewriting the rules of news delivery. I feel good about it.
Contextual AI: RAG Comes Home
The creators of Retrieval-Augmented Generation (RAG) are back with Contextual AIโs new platform, designed to tackle the most complex, knowledge-intensive tasks. With its unified RAG 2.0 architecture, it outperforms fragmented systems by delivering higher accuracy, fewer errors, and real-world reliability. Itโs always fascinating to see the originators of a concept bring it to a truly production-ready stage.
Hugging Face Launches Free Course on AI Agents
Hugging Face has introduced a free, certified course designed to demystify AI agents. Participants will learn how to build intelligent agents using frameworks like LangChain and LlamaIndex, explore real-world applications, and earn a certification by completing hands-on tasks. Whether you're a developer or simply curious, the course offers a solid foundation in this fast-evolving field.
Microsoftโs AI Machine Rolls On
MatterGen: AI in the Lab Coat
Microsoft Research unveils MatterGen, a generative AI tool that designs new materials from scratch. By bypassing traditional screening processes, itโs already creating stable compounds with properties like magnetism and durability. Batteries, solar cells, COโ capture โ MatterGen could be the key to breakthroughs in sustainable tech. A while ago, we published an interview with one of MatterGenโs coauthors โ Read it.AutoGen 0.4: Building Smarter Agents
The latest release of AutoGen refines Microsoftโs agentic framework, advancing the tools for developing proactive, task-driven AI systems.New AI Engineering division
Microsoft recruits ex-Meta heavyweight Jay Parikh to lead its new AI engineering division. Tasked with scaling supercomputers and platforms, Parikh embodies Nadellaโs mission: โThirty years of change in three.โ Thatโs a very interesting development!
Googleโs Titans Roar
Google Research unveils Titans, an AI model with dynamic "long-term memory" at test time, claiming linear scaling for long inputs. This could shatter Transformers' quadratic constraints and push AI towards human-like cognition. While skeptics flag computational costs and memory bottlenecks, enthusiasts await benchmarks. Titans may just redefine what "attention" really means in 2025.
OpenAI: Serving New President
Days before inauguration, OpenAI unveils its manifest-like Economic Blueprint, urging investment in chips, energy, and talent to drive AI-powered growth while safeguarding democracy โ dancing along the new presidentโs lines to stay relevant.
Meanwhile, ChatGPT gets smarter with โtasksโ in beta. Now a proactive assistant, it handles everything from reminders to automating recurring actions โ making your to-do list one less thing to think about. Task: โHey chat, wake me up when everyone is done with hypocrisy.โ
We are reading
An Introduction to Adversarial Perturbation by Devansh โย a fascinating read which unpacks adversarial perturbations with clarity, highlighting their disruptive potential and untapped opportunities in AI.
The freshest research papers, categorized for your convenience
There were quite a few TOP research papers this week, we will mark them with ๐ in each section.
Attention and Transformer Innovations
๐ Transformer2: Self-Adaptive LLMs
Adapt dynamically to unseen tasks with fine-tuned weight matrices and task-specific expertise for real-time performance boosts.๐ Tensor Product Attention Is All You Need
Reduce memory usage with tensor factorization in attention, scaling efficiently to improve performance in extended contexts.
Reasoning, Thinking and Knowledge Expansion
Towards Large Reasoning Models: A Survey on Scaling LLM Reasoning Capabilities
Advance reasoning in LLMs with techniques like reinforcement learning and test-time scaling for structured problem-solving.In-situ Graph Reasoning and Knowledge Expansion Using Graph-PReFLexOR
Integrate graph-based reasoning with symbolic abstraction to enhance adaptability and interdisciplinary problem-solving.๐ OmniThink: Expanding Knowledge Boundaries in Machine Writing Through Thinking
Mimic human cognition in machine writing, dynamically retrieving and expanding knowledge for comprehensive content generation.
Scaling Foundation Models
๐ MiniMax-01: Scaling Foundation Models with Lightning Attention
Scale efficiently with lightning attention and Mixture of Experts, enabling long-context processing and multimodal integration for enhanced tasks.Learnings from Scaling Visual Tokenizers for Reconstruction and Generation
Scale Vision Transformers effectively, achieving state-of-the-art results in image and video applications with reduced computational costs.Inference-Time Scaling for Diffusion Models Beyond Scaling Denoising Steps
Improve generative diffusion models by optimizing noise selection, enhancing image quality and diversity.
Best Practices for Datasets
๐ Towards Best Practices for Open Datasets for LLM Training
Establish transparency and diversity principles for dataset creation to democratize AI development ethically and effectively.
Benchmarks and Evaluation
HALOGEN: Fantastic LLM Hallucinations and Where to Find Them
Analyze hallucination patterns in LLMs with detailed benchmarks, categorizing errors to guide mitigation strategies.PokerBench: Training Large Language Models to Become Professional Poker Players
Assess LLM capabilities in strategic gameplay, revealing the complexity of incomplete information tasks.
Enhancing Training and Interpretability
SPAM: Spike-Aware Adam With Momentum Reset for Stable LLM Training
Enhance training stability with spike-aware optimization, addressing gradient spikes for improved efficiency and performance.Enhancing Automated Interpretability with Output-Centric Feature Descriptions
Focus on output-based feature analysis to refine interpretability and revive underutilized neural features.
Unspecified
CityDreamer4D: Compositional Generative Model of Unbounded 4D Cities
Generate unbounded 4D urban environments with compositional design, blending real-world data and scalability for urban simulations.Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography
Enable secure AI-driven collaboration with trusted environments as an alternative to cryptographic solutions.
Thatโs all for today. Thank you for reading! Please send this newsletter to your colleagues if it can help them enhance their understanding of AI and stay ahead of the curve
Leave a review!
โ FOD 83 GAN is back
