Possible concept check for SGD

[davidrosenberg]

For a differentiable function that is Lipschitz continuous with constant L, give a bound on the derivative (for functions mapping R to R). (Write down the limit form of the derivative -- each quotient is bounded by Lipschitz constant, so limit is too. modulo some absolute values ) For functions mapping R^d --> R^d, give a bound on the determinant of the Jacobian. (it's L^d) (use the same strategy but with directional derivatives in coordinate directions -- also https://math.stackexchange.com/questions/1195715/jacobian-determinant-of-lipschitz-function

[brett1479]

This question (for the gradient) already exists. Do a search for Lipschitz in https://github.com/davidrosenberg/mlcourse/blob/gh-pages/ConceptChecks/AllChecks_sol.pdf . It is in the L1/L2 regularization section.