Information Theory
Last updated
Last updated
• Entropy and Information Gain
• Definition of entropy and its significance in measuring uncertainty
• Information gain and its use in decision trees
• Cross-Entropy and Kullback-Leibler Divergence
• Understanding cross-entropy loss in classification problems
• KL divergence as a measure of difference between probability distributions