Dropout is a method used by Neural Networks to avoid Overfitting. In simple terms, During training a Neural Networks, we randomly choose certain neurons and set them to zero (ignore) for forward pass or backward pass.

These neurons are chosen by a probability p, i.e; individual nodes are kept with probability (1-p) so that a reduced network is left; incoming and outgoing edges to a dropped-out node are also removed.

1. Dropout forces a neural network to learn more robust features that are useful in conjunction with many different random subsets of the other neurons.
2. Dropout roughly doubles the number of iterations required to converge. However, training time for each epoch is less.
3. With H hidden units, each of which can be dropped, we have 2^H possible models. In testing phase, the entire network is considered and each activation is reduced by a factor p.
   

   
   
   

Implementation

Inverted Dropout Method is commonly used for Drop out Implementation