Short answer
Neural networks are computational models made of layered interconnected units that learn weighted transformations from data—forming the substrate of deep learning.
Why it matters
Almost every modern AI breakthrough in perception and language is a neural network story. The basics unlock CNNs, transformers, and training intuition.
Diagram
Article
Practice
Implement a tiny MLP and visualize decision boundaries.
Examples
Binary classifier as a small network
A two-layer network can classify points that are not linearly separable by learning intermediate features that reshape the representation space. Visualizing decision boundaries on a 2D toy dataset makes the role of nonlinear activations concrete for learners.
Transfer learning with a pretrained backbone
A vision network pretrained on a large image dataset can have its early layers reused as a feature extractor for a smaller medical imaging task. Only later layers (or adapters) are trained extensively, reducing data requirements compared with training from scratch.
FAQ
What is an activation function?
An activation function introduces nonlinearity after a weighted sum, enabling networks to model complex relationships. Common choices include ReLU and its variants, sigmoid/tanh in older architectures, and softmax on output layers for multi-class probabilities.
Why do we need many layers?
Multiple layers allow hierarchical composition of features. Depth can represent certain functions more efficiently than a single wide shallow layer, though depth also complicates optimization and requires architectural support such as residual connections.
Related glossary terms
- Deep learning — Machine learning with multi-layer neural networks that learn hierarchical representations from large datasets.
- Neural network — A model made of layered interconnected units that transform inputs through weighted connections and nonlinear…
- Transformer — A neural network architecture that uses attention mechanisms to model relationships among tokens in a sequence…
Sources
-
Deep Learning (online textbook)
Goodfellow, Bengio, CourvilleFoundational chapters on multilayer perceptrons, backpropagation, and regularization.
-
DeepLearning.AI blog
DeepLearning.AIAccessible educational articles on neural network practice.