You can get fast second-order convergence in optimization without solving expensive auxiliary problems—just one linear system per iteration with the right parameter choices.
This paper presents a new accelerated Newton method for convex optimization that achieves cubic convergence (O(1/k³)) using only one linear solve per iteration. Unlike existing approaches, it avoids expensive subproblems like cubic regularization and works with inexact solvers, making it practical for large-scale problems.