A description of perturbation-based methods developed in machine learning to augment novel optimization methods with strong statistical guarantees.In nearly all machine learning, decisions must be made given current knowledge. Surprisingly, making what is believed to be the best decision is not always the best strategy, even when learning in a supervised learning setting. An emerging body of wo…