ML coding problems

Interesting problems and the solutions I want to revisit.

Credit: many problems are adapted from Deep-ML.

Convolutional 2D Layer

Implementing a simple 2D convolution layer with padding and stride, including output shape formulas.

27-01-2026convolution · deep-learning

Implementing residual blocks with shortcut connections to enable gradient flow in deep networks and solve the degradation problem.

25-01-2026resnet · residual-networks · deep-learning

Overview of common activation functions with their mathematical equations, derivatives, and Python implementations.

22-01-2026activation-functions · neural-networks · deep-learning

Understanding LSTM architecture with gate mechanisms, forward pass implementation, and how it solves vanishing gradients compared to vanilla RNNs.

21-01-2026lstm · rnn · deep-learning · memory · gates · gradient-flow · sequence-modeling

Implementing self-attention and multi-head attention mechanisms as part of the Transformer architecture

21-01-2026transformers · attention · self-attention · multi-head-attention · neural-networks · deep-learning · nlp

Train a single sigmoid neuron end-to-end with manual gradients.

20-01-2026backpropagation · sigmoid · gradient-descent · mse-loss