The rule as a line search

The two-shot placement rule as the inner loop of derivative-free optimizers: where it wins, where it loses, and why.

A line search in a derivative-free optimizer is terminal placement: a few evaluations along a direction, then a point is returned. Golden-section and Brent searches assume smooth unimodal slices. The grass rule instead assumes an Ornstein–Uhlenbeck slice and places its probes in closed form.

How rough real objectives are

Roughness is measured by the variogram exponent $p$, with $\mathbb{E}\big[(f(x+\Delta)-f(x))^2\big] \sim \Delta^p$ at small lags. Smooth slices have $p = 2$, and an Ornstein–Uhlenbeck path has $p = 1$.

• Classic test functions along random lines are smooth, with $p$ between 1.72 and 1.98.
• Full-physics simulators are often very rough: pool 0.15, mini golf 0.18, curling 0.07, bowling 0.29.
• Composing with a space-filling curve manufactures roughness that falls with dimension.

Inside a custom outer loop

With one outer loop of iterated random-direction line searches and a budget of 120 evaluations, only the inner loop differs. The rule uses one or two evaluations per line, so the outer loop sees three to five times as many directions as it would with Brent.

• It wins on navigably rough simulators with collision or agent chaos: plinko (20 to 21 of 24 seeds against every rival), bowling, boids and tennis.
• It wins in high dimension, where direction count beats per-line precision. Against Brent its seed wins rise from 6 of 24 at dimension 2 to 24 of 24 at dimension 32 on Rosenbrock.
• It loses on smooth low-dimensional slices, and on needle landscapes where the value hides in a tiny region.

Ablations show the engine is the per-line economy and the adaptive bracket, not the third shot. The two-shot version with a uniform interior flee is the one to carry forward, and it is also the simplest.

Against engineered optimizers

Against nine state-of-the-art optimizers ported into humpday, on 22 physics simulators at a budget of 120, the rule ranks about sixth of eleven by tie-averaged rank. It is just below PRIMA's BOBYQA and above golden section, Nelder–Mead, random search and dual annealing.

It is a few-shot specialist. Its standing is best at a budget of 40, where it leads CMA-ES, and CMA-ES overtakes it from about 100 evaluations on.

Swapped into Powell's method

Replacing only the line search inside Powell's method is not an upgrade: stock Powell with Brent wins 373 of 488 paired seeds. Powell's direction-set updates rely on the line search being a genuine minimizer, and a one- or two-evaluation probe feeds them noise.

The exception is budget-starved high dimension. On a 16-dimensional wind-farm layout, Brent's lines exhaust the budget before one direction cycle, and the swap wins 24 of 24 seeds.

Against one-step expected improvement

The optimizer keeps its incumbent, so the matched one-step acquisition is expected improvement on the Ornstein–Uhlenbeck line. The rule, which carries the wider bracket of the terminal game, wins 143 paired seeds to 116 across 14 problems. The wider bracket buys exploration whose value a one-step criterion cannot see.

On its own model

On Gaussian fields with exponential covariance, which are Ornstein–Uhlenbeck along every line, the rule loses: golden section with six probes wins almost every case. With correlation length $1/\kappa$ the cube holds about $\kappa^d$ nearly independent cells, and with 120 evaluations decorrelated probing wins.

So the rule's wins are not explained by the slices being Ornstein–Uhlenbeck. Its terminal-placement logic says to correlate, while a running maximum over many calls says to spread.

The basin hypothesis

The rule should win when the evaluation budget is comparable to the number of local minima. The direction, membership and rank definition were registered before a holdout sweep of 33 full-physics pages. On that holdout the Spearman correlation between basin count and the rule's rank is $+0.455$, one-sided $p = 0.0039$, in the predicted direction.

Across every experiment the strongest general baseline is golden section with two full-segment probes per line. The rule's distinctive wins over it are the navigably rough simulators and structured high-dimensional landscapes.

Code and logs: linesearch/.