Lilian Weng's Two Recent Blog Posts: Harness Engineering May Become a New Paradigm for AI Self-Improvement, Foundations of Scaling Law Shaken

Lilian Weng, former VP of Safety Research at OpenAI and co-founder of Thinking Machines Lab, published two technical blog posts within two weeks, discussing the limitations of Scaling Laws and the potential of Harness Engineering for AI self-improvement. The first post, "Scaling Laws, Carefully," points out that the fitting results of classical Scaling Laws are highly sensitive to details such as parameter counting methods and loss function precision. The disagreement between Kaplan and Chinchilla stems from bookkeeping-level differences, and data repetition leads to overfitting, undermining the premise of "infinite data." The second post, "Harness Engineering for Self-Improvement," systematically reviews design patterns for the execution system (Harness) wrapped around the base model, including workflow automation, file system persistent memory, sub-agent parallelism, etc. It argues that recursive self-improvement (RSI) is more likely to occur at the Harness layer in the near term rather than through direct rewriting of model weights. The post also introduces context optimization methods like ACE and Meta Context Engineering, as well as self-optimization frameworks such as Self-Harness and Darwin Gödel Machine, while candidly listing unresolved challenges including evaluation ambiguity, memory management, diversity collapse, and reward hacking. Together, the two posts point to a trend: when the marginal benefit of scaling model parameters diminishes, Harness Engineering may become a second growth curve for AI capability enhancement.