deep learning keywords
Neural Networks
Neural networks are computational models inspired by the brain, consisting of layers of interconnected nodes. They are widely used in deep learning for pattern recognition, classification, and prediction tasks.
Backpropagation
Backpropagation is an algorithm used to train neural networks by minimizing error. It calculates gradients of the loss function with respect to weights and updates them using gradient descent.
Gradient Descent
Gradient descent is an optimization method that adjusts model parameters by moving in the direction of the negative gradient to minimize the loss function over training data.
Activation Function
Activation functions introduce non-linearity into neural networks, enabling them to learn complex patterns. Common types include ReLU, sigmoid, and tanh, each with specific advantages and drawbacks.
Overfitting
Overfitting occurs when a model learns noise or patterns specific to the training data, reducing its ability to generalize to new, unseen data, often requiring regularization to address.
Underfitting
Underfitting happens when a model is too simple to capture the underlying structure in the data, resulting in poor performance on both training and test datasets, indicating insufficient learning.
Regularization
Regularization techniques such as L1, L2, and dropout are used to prevent overfitting by discouraging overly complex models, thereby improving their generalization capability on unseen data.
Convolutional Neural Networks
Convolutional Neural Networks (CNNs) are specialized neural networks designed for processing structured grid data like images, using convolutional layers to detect hierarchical spatial features effectively.
Recurrent Neural Networks
Recurrent Neural Networks (RNNs) are designed for sequential data by using feedback loops. They maintain a memory of previous inputs, making them useful for time series and language modeling.
Long Short-Term Memory
Long Short-Term Memory (LSTM) networks are a type of RNN designed to learn long-term dependencies by using gated cells to manage information flow, mitigating the vanishing gradient problem.
Dropout
Dropout is a regularization method where random neurons are ignored during training. This prevents co-adaptation of neurons, encouraging the model to learn more robust and generalizable features.
Learning Rate
Learning rate is a hyperparameter that controls how much model weights are adjusted during training. Proper tuning is crucial, as values too high or low hinder convergence.
Loss Function
A loss function quantifies the error between predicted and actual outputs. Minimizing this function is the goal during training. Common choices include MSE, cross-entropy, and hinge loss.
Epoch
An epoch is one complete pass through the training dataset. Multiple epochs allow the model to iterate and refine its internal parameters by learning from the data repeatedly.
Batch Size
Batch size defines the number of training examples processed before the model updates its parameters. It affects convergence speed, memory usage, and overall training dynamics.
Optimizer
An optimizer updates model weights to minimize the loss function. Popular choices like Adam, SGD, and RMSProp offer different trade-offs in terms of speed, memory, and stability.
Feature Extraction
Feature extraction involves identifying informative attributes from raw data. In deep learning, neural networks automatically learn hierarchical feature representations from the input during training.
Transfer Learning
Transfer learning applies knowledge from a pretrained model to a new, related task. It enables faster training, requires less data, and often achieves better performance than training from scratch.
Autoencoder
An autoencoder is a neural network used for unsupervised learning. It compresses input data into a latent representation and reconstructs it, useful for dimensionality reduction and denoising.
Embedding
Embeddings are dense vector representations of discrete data like words or categories. They capture semantic relationships and are widely used in natural language processing and recommender systems.