This is a great introductory on convex optimization! Thank you for putting in the time to write it! I have a couple of comments.
Firstly, you mention that for convex problems we have guaranteed time frames to find the optimal solution. This is true even for nonconvex problems (see branch and bound), but for convex problems the important point is that the runtime is polynomial in the size of the problem rather than exponential as in the case of branch and bound.
Also many convex optimization algorithms do not compute a starting point in the feasible region, for example primal-dual interior point methods which are used in solvers such as CVXGEN and ECOS.
I also think the figure depicting a 1D convex optimization problem is slightly misleading. The two dashed lines should be vertical and the feasible region for the problem is the line segment on the x-axis between the dashed lines.
You also say that you do not need an initial guess to start from to solve convex optimization problems. You do need an initial guess since it is inherently an iterative algorithm, but no matter your initial guess you will find the optimal solution. So the important point is that the quality of the solution is independent of the initial guess.
Finally you mention that SpaceX uses a convex approximation to solve their rocket landing problem. I assume you are referring to LCvx/GFOLD. These are lossless convex relaxations which means that they increase the size of the feasible set and guarantee that a solution to this relaxed convex problem is the optimal solution to the initial nonconvex problem. A convex approximation means that the nonconvex set is approximated by a convex set and the solutions to the two problems do not necessarily match. A lossless convex relaxation as used in LCvx/GFOLD is much stronger than a convex relaxation.
Thanks for the comment! I may append the post with some of your feedback. You’re completely right about the, “no initial guess” point. I was coming at it from a practitioners viewpoint where these solvers are basically now a technology which does initial guess generation for you. As for the SpaceX application, as far as I’m aware the literature has shown that lossless convexification is only really available with control constraints like minimum magnitude bounds, but there could be new stuff out there that broadens the application of LCvx. To deal with the general nonlinear dynamics of the problem, you must enter into some approximate, iterative (successive) convexification scheme. But my knowledge could be outdated!
This is a great introductory on convex optimization! Thank you for putting in the time to write it! I have a couple of comments.
Firstly, you mention that for convex problems we have guaranteed time frames to find the optimal solution. This is true even for nonconvex problems (see branch and bound), but for convex problems the important point is that the runtime is polynomial in the size of the problem rather than exponential as in the case of branch and bound.
Also many convex optimization algorithms do not compute a starting point in the feasible region, for example primal-dual interior point methods which are used in solvers such as CVXGEN and ECOS.
I also think the figure depicting a 1D convex optimization problem is slightly misleading. The two dashed lines should be vertical and the feasible region for the problem is the line segment on the x-axis between the dashed lines.
You also say that you do not need an initial guess to start from to solve convex optimization problems. You do need an initial guess since it is inherently an iterative algorithm, but no matter your initial guess you will find the optimal solution. So the important point is that the quality of the solution is independent of the initial guess.
Finally you mention that SpaceX uses a convex approximation to solve their rocket landing problem. I assume you are referring to LCvx/GFOLD. These are lossless convex relaxations which means that they increase the size of the feasible set and guarantee that a solution to this relaxed convex problem is the optimal solution to the initial nonconvex problem. A convex approximation means that the nonconvex set is approximated by a convex set and the solutions to the two problems do not necessarily match. A lossless convex relaxation as used in LCvx/GFOLD is much stronger than a convex relaxation.
Thanks for the comment! I may append the post with some of your feedback. You’re completely right about the, “no initial guess” point. I was coming at it from a practitioners viewpoint where these solvers are basically now a technology which does initial guess generation for you. As for the SpaceX application, as far as I’m aware the literature has shown that lossless convexification is only really available with control constraints like minimum magnitude bounds, but there could be new stuff out there that broadens the application of LCvx. To deal with the general nonlinear dynamics of the problem, you must enter into some approximate, iterative (successive) convexification scheme. But my knowledge could be outdated!
Thanks again.