Let me start by saying this, I am relatively new to neural networks and Rust. But here it is: A compact neural network library written in Rust. I know that the best way to learn something is to get your hands dirty. That’s what inspired me to create runn: a beginner-friendly, end-to-end Rust API that takes you from raw data to a trained network in a handful of lines.

In deep learning, optimizers are the secret heroes that guide models toward better performance. Whether you’re training a small neural network or a massive transformer, selecting the right optimizer is very crucial. This post explores four widely-used optimizers SGD, Momentum, RMSprop, and Adam and how they adapt parameter updates for optimal learning..

Gradient descent is one of the main tools that is used in many machine learning and neural network algorithms. It acts as a guide to finding the minimum of a function. But what’s happening under the hood, and why do we need it?

Neural networks are systems that mimic certain aspects of human cognition, aiming to identify and learn patterns within complex data. One crucial element that is hard to grasp in these networks is bias. Bias might seem like just another parameter, but it plays a significant role in the learning process.

I consider time as one of the most important assets of our lives, something we should use wisely. Sometimes, I’m surprised by how much time we waste when we could have used it better. One of the most interesting forms of wasted time is waiting for something or someone. In most cases, it’s up to us to speed things up or eliminate unnecessary waits altogether, yet we still don’t do much about it. Maybe it’s because we don’t care enough or don’t even realize it. I’m not saying we should rush everything, but I do believe we should question the need for certain waits.{{ double-space-with-newline }}

To us humans, life does not come equally. Some are born rich, some are born poor. While some have great families and wonderful childhoods, for others it is not so. Even if we were born in the same family, but in a different country or in a different culture, we wouldn’t be the same person. These are the facts that we cannot change most of the time.

I think the only thing that treats people equally is time. Leaving the relativity theorem aside, time passes at the same speed for all of us. That is, a unit of time has the same value for everyone. What we do in this unit of time is also largely up to us. And I believe, how we use time determines the biggest difference between us.