Master Deep Learning and Generative AI with PyTorch in Hindi
Build and Deploy AI Models: Learn Neural Networks, Computer Vision, NLP, and More with PyTorch
Master Deep Learning and Generative AI with PyTorch in Hindi udemy course free download
Build and Deploy AI Models: Learn Neural Networks, Computer Vision, NLP, and More with PyTorch
you will learn all these Topics and lot more
1. Core Concepts
1. Perceptron
2. MLP and its Notation
3. Forward Propagation
4. Backpropagation
5. Chain Rule of Derivative in Backpropagation
6. Vanishing Gradient Problem
7. Exploding Gradient
Activation Functions
List of Activation Functions
1. Linear Function
2. Binary Step Function
3. Sigmoid Function (Logistic Function)
4. Tanh (Hyperbolic Tangent Function)
5. ReLU (Rectified Linear Unit)
6. Leaky ReLU
7. Parametric ReLU (PReLU)
8. Exponential Linear Unit (ELU)
9. Scaled Exponential Linear Unit (SELU)
10. Softmax
11. Swish.
12. SoftPlus
13. Mish
14. Maxout
15. GELU (Gaussian Error Linear Unit)
16. SiLU (Sigmoid Linear Unit)
17. Gated Linear Unit (GLU)
18. SwiGLU
19. Mish Activation Function
Derivative of Activation Functions
Properties of Activation Functions
1. Saturating vs Non-Saturating
2. Smooth vs Non-Smooth
3. Generalized vs Specialized
4. Underflow and Overflow
5. Undefined and Defined
6. Computationally Expensive vs Inexpensive.
7. 0-Centered and Non-0-Centered
8. Differentiable vs Non-Differentiable
9. Bounded and Unbounded
10. Monotonicity
11. Linear Vs Non Linear
Ideal Activation Function Characteristics
1. Non-Linearity
2. Differentiability
3. Computational Efficiency
4. Avoids Saturation
5. Non-Sparse (Dense) Gradients
6. Centered Output (0-Centered)
7. Prevents Exploding Gradients
8. Monotonicity (Optional)
9. Sparse Activations (Optional)
10. Resilience to Outliers
11. Noise Robustness
12. Stable Training Dynamics
13. Minimal Parameter Dependency
14. Compatibility with Modern Techniques
15. Efficient in Hardware
16. The Function Must Be Continuous and Infinite in Domain
17. Vanishing Gradient Problem
18. Dynamic Range Adaptation
19. Scalability to Deeper Networks
20. Biological Plausibility (Optional)
21. Simplicity in Implementation
22. Gradient Smoothness
23. Compatibility with Unsupervised Objectives
Loss Functions
1. Mean Squared Error (MSE)
2. Mean Absolute Error (MAE)
3. Root Mean Squared Error (RMSE)
4. Root Mean Squared Log Error (RMSLE)
5. Huber Loss
6. Hinge Loss
7. Binary Cross-Entropy (BCE)
8. Categorical Cross-Entropy
9. Focal Loss
10. Contrastive Loss
11. KL Divergence (Kullback-Leibler Divergence)
12. Triplet Loss
13. Smooth L1 Loss:
14. Dice Loss:
Optimizers
1. Gradient Descent
2. Stochastic Gradient Descent (SGD)
3. Mini-Batch Gradient Descent
4. Exponentially Weighted Moving Average (EWMA)
5. Gradient Descent with Momentum
6. Nesterov Accelerated Gradient
7. AdaGrad (Adaptive Gradient)
8. RMSProp (Root Mean Squared Propagation)
9. AdaDelta
10. Adam (Adaptive Moment Estimation)
11. Nadam (Nesterov-accelerated Adaptive Moment Estimation)
12. LAMB (Layer-wise Adaptive Moments):
13. SGDW/AdamW
Improving Performance of Neural Networks
· Effect of Batch Size on Training
· Memoization
Weight Initialization
1. Zero Initialization
2. Non-Zero Constant Value Initialization
3. Random Initialization (with small values, large values)
4. Xavier (Glorot) Initialization
5. He Initialization
6. LeCun Initialization
7. Uniform Initialization
8. Normal (Gaussian) Initialization
9. Bilinear Initialization
10. Orthogonal Initialization
Regularization
1. L1 Regularization (Lasso)
2. L2 Regularization (Ridge) (weight decay)
3. Elastic Net Regularization
4. Dropout
5. Early Stopping
6. Data Augmentation
7. Batch Normalization
8. Residual Connections
9. Label Smoothing
10. Parameter Sharing
11. Weight Constraint
12. Adversarial Training
Normalization
1. Normalizing Inputs
2. Batch Normalization (BatchNorm)
3. Layer Normalization (LayerNorm)
4. Instance Normalization (InstanceNorm)
5. Group Normalization (GroupNorm)
6. RMSNorm
7. Filter Response Normalization
8. Weight Normalization
Other Techniques
Gradient Clipping and Gradient Checking Hyperparameter Tuning
Learning Rate Scheduling
1. Step Decay
2. Exponential Decay
3. Cosine Annealing
4. Cyclical learning rate
5. OneCycleLR
6. Warmup
Recurrent Neural Networks (RNNs)
1. RNN
2. LSTM
3. GRU
4. Deep Stacked RNN, Bidirectional
Sequence-to-Sequence Models
1. Encoder-Decoder Architecture
2. Attention Mechanism
Natural Language Processing (NLP)
· Tokenization: Sentence tokenization, word tokenization, and subword tokenization (BPE, WordPiece).
Text Preprocessing: Lowercasing, stemming, lemmatization, stopword removal etc...
Text Vectorization:
1. One-Hot Encoding
2. Bag of Words (BoW)
3. TF-IDF
4. Word Embeddings (Word2Vec, GloVe, FastText)
5. Contextual Embeddings (ELMo, BERT, GPT, etc.) Complete NLP Basics
Transformers
1. Vanilla Transformer
2. Vision Transformer
3. Swin Transformer
and lot more
