Using Newton Method of Optimization Coursework Example | Topics and Well Written Essays - 250 words - 1

Using Newton Method of Optimization - Coursework Example On the other hand, if a constrained optimization is done (for example, with Lagrange multipliers), the problem may become one of saddle point finding, in which case the Hessian will be symmetric indefinite and the solution of xn+1 will need to be done with a method that will work for such, such as the LDLT variant of Cholesky factorization or the conjugate residual method. There also exist various quasi-Newton methods, where an approximation for the Hessian (or its inverse directly) is built up from changes in the gradient. If the Hessian is close to a non-invertible matrix, the inverted Hessian can be numerically unstable and the solution may diverge. In this case, certain workarounds have been tried in the past, which have varied success with certain problems. One can, for example, modify the Hessian by adding a correction matrix Bn so as to make Hf(in) + Bn positive definite. One approach is to diagonalize H f(xn) and choose Bn so that H f(xn) + Bn has the same eigenvectors as H f(xn), but with each negative eigenvalue replaced by Ï µ > 0.