Tracing and Correcting Programs: Critic-Guided Synthesis for Visual Reasoning

Abstract

Program synthesis for complex reasoning tasks faces a fundamental challenge: initial attempts often generate flawed programs that fail to capture the underlying problem logic. We introduce Tracing and Correcting Programs (TCP), a critic-guided framework that shifts the paradigm from blind guessing to systematic debugging through iterative refinement. Instead of discarding failed programs, TCP begins by analyzing each task, tracing its execution errors, and generating structured diagnostic feedback through a critic module. Through an iterative validation process, corrected programs are refined and tested until a solution emerges. The key contributions include: (1) A systematic approach that transforms failed programs into improved and correct programs, (2) An adaptive sampling strategy that allocates computational resources based on task complexity, requiring only 7–8 samples per task for complete solutions, and (3) It requires no task-specific training. We evaluate TCP on the challenging Abstraction and Reasoning Corpus (ARC), covering all 800 tasks, where TCP solves 159 tasks and improves the majority of tasks by up to 68.1%. TCP achieves systematic improvements by over two orders of magnitude lower samples (300–400× reduction) than evolutionary or multi-agent methods that require evaluating hundreds or thousands of samples, often with training overhead. These results highlight the importance of feedback-driven refinement and establish a new paradigm for efficient program synthesis in complex reasoning domains.