Regularization is a mechanism to prevent overfitting (reduce variance, hence improve the generalizability of the model) by adding an extra term (function of parameters) to the cost function. The added extra term, depending on the regularization method we choose, can be L1 norm of the parameter vector (LASSO), L2 norm (ridge) or linear combination of the two (elastic). In case of neural net, drop out is also a regularization method.