torch-secorder
maintainedA PyTorch-native library for efficient second-order computations in deep neural networks.
Python PyTorch Deep Learning Optimization
Torch-Secorder provides efficient implementations of second-order optimization utilities for PyTorch.
Features
- Hessian-Vector Products (HVP): Computation of Hv for any vector v
- Jacobian-Vector Products (JVP)
- Vector-Jacobian Products (VJP)
- Gauss-Newton matrix computations
- Hessian trace estimation
These tools are essential for second-order optimization methods, natural gradient descent, curvature-based regularization, and neural network analysis and debugging.