Optimization

Optimization is a fundamental skill across multiple engineering disciplines. Here are a few resources that I’ve found helpful for learning and using optimization tools.

Books

• Convex Optimization (Stephen Boyd and Lieven Vandenberghe). This book is dense, but it is a must-read for anyone serious about using optimization in advanced academic or industry settings. It teaches the fundamentals of how to prove a problem is convex, how to identify different classes of convex problems, and the guarantees on convergence to a global optimum that convex problems give.

• Engineering Design Optimization (Free PDF from the author in the “Textbooks” sections of the previous link). This is a great introduction to optimization methods for nonconvex optimization problems. In other words, if your problem isn’t convex, you’ll likely be using the methods from this book. (The book does have a brief chapter about convex problems though).

Software

Many of the solvers I’ve found have come from looking at the JuMP.jl list of supported solvers. It’s worth a read if you’re looking to explore options.

Keep in mind that commercial solvers almost always outperform open source / free solvers. It’s not a close comparison. If you have access to a commercial solver, use it.

Linear Programming

• FOSS:

  • CLP

  • GLPK

• Commercial / Non-FOSS:

  • Gurobi

  • SoPlex

  • MATLAB linprog

Mixed-Integer Linear Programming

• FOSS:

  • CBC

  • GLPK

• Commercial / Non-FOSS:

  • Gurobi

  • SCIP

  • MATLAB intlinprog

Quadratic Programming

• FOSS:

  • OSQP

• Commercial / Non-FOSS:

  • Gurobi

  • MATLAB quadprog

Benchmarks of Optimization Software:

• Hans Mittelmann Benchmarks. Compares performance of LP, MILP, and SDP solvers, along with others. Note how poorly the FOSS solvers perform compared to the commercial options, in terms of both number of problems solved and average solution time.