Neural Networks

Neural networks are the workhorse behind modern machine learning: layers of simple units that transform inputs into predictions. This mind map is a concept guide, not a textbook chapter. It puts Architecture at the center of how you see the field, then fans out into CNNs and Training and Optimization so you can hold the whole picture before you open a paper or a course.

Start with Architecture. A network is a graph of layers — input, hidden, and output — connected by weighted edges. Fully connected (dense) layers mix every feature; convolutional layers share weights across space; recurrent and attention layers carry context across a sequence. Activation functions such as ReLU and softmax inject the non-linearity that lets a stack of linear maps approximate complex functions. Initialization, residual connections, and normalization keep those stacks trainable as they get deeper. Treat “architecture” as the vocabulary of parts, not a single canonical diagram.

Convolutional neural networks (CNNs) specialize that vocabulary for grids: images, spectrograms, and other local structure. Convolution, padding, and stride control the receptive field. Pooling downsamples; 1×1 convolutions mix channels cheaply. Classic families — from early conv nets through residual networks — show how skip connections and depth changed what was practical to train. Transfer learning from an ImageNet-scale backbone is still the default for many vision tasks: freeze early filters that detect edges and textures, then adapt later layers to your labels.

Training and Optimization is the third pillar. Loss functions (cross-entropy, MSE) measure error; backpropagation assigns credit to each weight. Stochastic gradient descent and its adaptive cousins (Adam, AdamW) take steps on mini-batches. Learning-rate schedules, weight decay, dropout, and data augmentation are the everyday tools that keep overfitting in check. Batch size, mixed precision, and gradient clipping are systems details that show up the moment you leave a toy notebook. Read this branch as a loop: forward pass, loss, backward pass, update — repeated until the map in your head matches the behavior of the model.

Use the outline below as a study checklist. If you already know dense layers, skip to CNNs. If you can sketch a conv block, spend time on regularization and schedules. The interactive canvas is the same tree: zoom to reveal subtopics, and treat sibling nodes as related ideas rather than a strict sequence. Pair this map with the Deep Learning guide when you are ready for transformers, representation learning, and larger training recipes.