AdamX offers a practical alternative to standard optimizers like Adam by using cosine similarity to adaptively scale updates, potentially improving training stability and convergence speed without requiring extensive hyperparameter tuning.
AdamX is a new optimizer that combines cosine similarity with gradient descent to control how much model weights change during training. It includes a variance smoothing technique for early training stages and works across different model types and datasets with minimal setup required.