Hello Greg,

Thanks for your questions. Please let me try to clarify them further:

- "Furthermore, little information is provided about how far ... ": what is this "information" (even if it is not much) ?

* Response: Without loss of generality, let's take the minimization case. In some cases, we may be able to find a lower bound on the global solution, i.e. LB<=Min. We can compare our current solution to this LB. Such as comparison is an example of information we may use to measure how far are we from the global solution if it exists.

- "In contrast, except some cases on the boundary of what is currently possible, there is little art involved in solving a least-squares problem or a linear program" : what are those "cases on the boundary" ? And what is the little art involved in least-squares ?

* Response: When I said little art, I mean that least-squares is based on a well defined mathematical foundation to be followed under some assumptions. There is so far no room for innovation in how this method works. Cases on the boundary are some special functions with specific features that allows their minimization easily using other methods.

- Is there a general algorithm to solve least-squares in "all cases and scenarios" ?

* Response: I guess you mean "solve any non-convex optimization problem". As I said in the blog, no so far.

If interested, you may check non-convex optimization lectures and courses available on the internet to tune further your understanding. In my blog, I didn't go into the details because I share and try to simplify for a large audience :-).

Best.

Mehdi.