@machinelearningtogo: ➡️ Machine Learning from Scratch, part 43: hyperparameters. A network learns its weights by itself, but a few numbers you set yourself before training, and those are the hyperparameters. The first and most important one is the learning rate, the size of each step downhill: new weight equals old weight minus learning rate times gradient. Too small, and the network has barely moved when training ends. Too large, and every step overshoots the valley. The batch size says how many examples the network sees per step, the number of epochs how often it goes through all the data, and the depth and the width say how many layers and how many neurons per layer the network has. You choose all of them by comparing different values on a validation set, data the network never trained on. The code (hyper.py, 16 lines of numpy, seed 43) trains a network with layers 4, 64, 64 and 1 to learn sin(x) from 4,000 examples, with a learning rate of 0.1, batches of 32 and 20 epochs. On the 1,000 validation examples the error falls to 2 % of the variance of the target. With only the learning rate changed, 0.00001 leaves the error almost where it started, and 10 makes it jump around. A second run with 40 noisy examples and 1,500 epochs shows the validation error rising again after epoch 15 while the training error keeps falling. #machinelearning #ai #neuralnetworks #deeplearning #coding

MachineLearningToGo

Open In TikTok:

Region: DE

Sunday 25 October 2026 11:36:17 GMT