Solver Benchmarks¶

benchmarks/solver/ compares SSQP against a handful of other OCP solvers on the same set of problems. Every solver formulates the identical problem through ProblemWrapper, an abstract interface (costs, constraints, finalize, solve, save) with one subclass per backend translating those calls into that backend's own API, so a given problem stays defined once while every solver attempts the same costs and constraints on it.

Six solvers are compared:

ssqp-admm / ssqp-hpipm: SSQP itself, using the two per-stage QP backends described in QP Solvers
fatrop / ipopt: formulated through CasADi's Opti stack, solved with the respective plugin
aligator: Aligator
mim_solvers: mim_solvers' CSQP, built on Crocoddyl

All six are run to the same nominal tolerance (1e-2 by default, see benchmark_config.py), even though that tolerance bounds a different internal quantity for each (SSQP's per-stage KKT/constraint residual, Fatrop/Ipopt's scaled KKT error, Aligator's primal-dual residual, CSQP's convergence gap), since reimplementing every solver's own termination criterion for an exact match isn't practical.

On fairness

A fully fair, apples-to-apples comparison across six independent solver codebases is close to impossible: each has its own convergence criterion, its own QP backend and warm-starting strategy, its own default tuning, and its own years of separate engineering investment behind it. These benchmarks are not a claim that SSQP is faster or higher-quality than any of the others. The goal is to give a rough sense of where SSQP sits relative to the state of the art on a shared set of problems, not to crown a winner.

Known gaps

On the Go2 problems, we were unable to get JIT compilation working in a timely fashion, so a fair benchmark for Fatrop and Ipopt (which rely on it) wasn't possible for those.
On the contact-switching problems (Go2 Jump, Go2 Wheelie, Go2 Frog), we excluded mim_solvers, since we were unable to get it to converge at all.
On Go2 Frog specifically, we were also unable to get Aligator to converge, leaving only the two SSQP solvers.
On Duffing Local, Fatrop is missing at m = 256 for the two largest horizons (N = 500 and N = 1000): JIT-compiling its formulation at that size requires more memory and time than a fair evaluation could accommodate, so those points were left out.
On Duffing Global, whose all-to-all coupling makes for a much denser formulation, both Fatrop and Ipopt are missing entirely at m = 256 (every horizon) for the same reason.

Using the selector below, you may change the tolerance to view the corresponding plots for each problem.

Tolerance:

Problem sizes¶

The problems range from Pendubot's long, low-dimensional swing-up to Go2 Frog's short but heavily constrained, contact-switching locomotion. Sizes below are per SSQP's own direct-transcription formulation (state = generalized position + velocity, control = generalized torque plus one force variable per active contact); a range means the value changes over the horizon with the contact schedule.

Problem	Horizon	State Dimension	Control Dimension	Constraints per Stage	NLP size (vars / constraints)
Pendubot	2500 / 2.5s	4	2	6 (terminal: 4)	15,004 / 15,004
Go2 Standing	100 / 1s	37	30	72 (terminal: 27)	6,737 / 7,227
Go2 Jump	150 / 1.5s	37	18–27*	51–78 (terminal: 42)	9,367 / 10,932
Go2 Wheelie	200 / 2s	37	24–30*	66–84 (terminal: 39)	12,957 / 15,405
Go2 Frog	300 / 3s	37	18–30*	48–84 (terminal: 30)	18,841 / 21,342
Panda Circle	400 / 4s	18	9	30 (terminal: 21)	10,818 / 12,021

* Varies with the number of feet in contact at each stage: every active contact adds a 3D force variable to that stage's control vector, so the dimension shrinks during flight phases and grows back during stance.

Reading the plots¶

Each plot bars four quantities per solver, side by side:

Runtime, in milliseconds
Iterations, the outer solve loop's iteration count
Final objective, the converged cost
Constraint violation, summed over every stage of that stage's worst bound violation (position, velocity, torque, friction, and, for the direct-transcription solvers, the dynamics-transition defect), against a dashed line marking the shared tolerance

A fast, few-iteration solve is not a meaningful win if the trajectory it returns is infeasible, which is what the constraint violation panel is for. A solver that fails to converge for a given problem is labeled "failed".

Since it's summed across stages rather than reported per stage, the plotted constraint violation is not directly comparable to the problem tolerance, which bounds a per-stage quantity. On a long horizon, this sum can end up above the tolerance even for a solver that has actually converged, since it is adding up many small, individually in-tolerance violations rather than reporting a single one that exceeds it.

Running the benchmarks yourself¶

See the solver/ README for the full setup (a Docker image is provided, since several of these solvers pull in their own heavy dependency chains). Once set up:

python run_benchmark.py [problem] -s [solver]

plot_benchmarks.py then renders one problem's data/<solver>/<problem>.json results into the plots shown below:

python plot_benchmarks.py pendubot

The Coupled Oscillators problems below are swept over problem size instead, via --duffing-m/--duffing-horizon (multiple values run a grid search), and rendered with plot_grid_search.py:

python run_benchmark.py duffing_local --all-solvers --duffing-m 2 4 16 64 256 --duffing-horizon 20 100 200 500 1000
python plot_grid_search.py duffing_local --duffing-m 2 4 16 64 256 --duffing-horizon 20 100 200 500 1000

Pendubot¶

The Pendubot Swing-Up problem: an underactuated double pendulum swinging from hanging to upright, actuated at the shoulder joint only.

Go2 Standing¶

The Go2 quadruped holding a fixed standing configuration, all four feet in contact throughout, with no other motion target. A short, contact-heavy but otherwise static regulation problem.

Go2 Jump¶

The Go2 Jump problem: stance, flight, stance, with all four feet leaving the ground for a scheduled window in the middle and landing back at a specified footprint. In this case, the front left foot has no contact and, therefore, should always remain in air.

Go2 Wheelie¶

The Go2 rearing up on its rear two feet: the rear feet stay in stance for the whole horizon, while the front two lift off for a window in the middle and come back down at the end. Unlike Go2 Jump, there's no target trajectory for the freed feet; the standing-configuration cost is what pulls them (and the rest of the robot) back down once the window closes.

Go2 Frog¶

A longer, four-hop bounding motion 2 meters forward, with staggered per-leg liftoff/landing and brief full-flight windows between hops rather than Go2 Jump's single synchronized flight phase. The same motion also appears, run directly rather than through the benchmark harness, as Go2 Frog in the Gallery.

mim_solvers and Aligator both failed to converge on this problem (see "Known gaps" above), so only the two SSQP solvers are meaningfully compared below.

Panda Circle¶

The Franka Panda tracing the circular (green) end effector path in task space via frame-pose tracking costs, additionally constrained to the rectangular window (red) inscribed in that circle, same as Panda Circle (Trajectory Generation), reformulated here in ForwardDynamics mode (torque-actuated) so every backend solves an equivalent problem.

Coupled Oscillators¶

A scalable benchmark instead of a fixed-size one: m weakly-coupled Duffing oscillators (state x = [q_1, ..., q_m, v_1, ..., v_m], one control force per oscillator), swept over both m and the horizon to see how each solver scales with problem size. Two coupling topologies are benchmarked: duffing_local, where oscillator i only interacts with its immediate neighbors (a discrete Laplacian), and duffing_global, where every oscillator interacts with every other one through a sinusoidal coupling term.

Each plot below lines runtime (left axis, solid) and iterations (right axis, dashed) per solver, faceted by the other swept variable; a missing point means that solver wasn't run at that grid point, while a point where the solver ran but didn't converge is marked with a red X.