Hebbian-LMS: What it may mean for Natural Learning

For those who do not know, Bernard Widrow is one of the all time greats in Electrical Engineering and one of the founding founding fathers of AI. He was one of the two main inventors in the Least Means Squares (LMS) and one of the core founders in the field of adaptive filter theory.

In a practical sense, he is one of the main reasons you can see this post on steemit right now. If it was not for his LMS algorithm to perform highly robust channel equalization, we would suffer from significant bit error and we would not be able to enjoy the high speed internet that we currently have. Similarly, his algorithm is highly used in noise-cancelling headphones for its ability to filter out ambient noise. Without going on about this for what could be a technical paper, Widrow's innovations have an impact on everything we do in our lives and this innovation came in the 1960s (yes, over 50 years ago).

Well, why should we care about him now? Widrow has combined his concepts implemented in the LMS algorithm with those commonly used in Hebbian learning to form the Hebbian-LMS algorithm. This algorithm, which is fairly robust for clustering and classification problems in machine learning, might be the closest representation to the way the brain actually learns a new process. If this algorithm is accurate, this would have major implications not only for the way we model natural systems, but also the way we understand intellectual development (perhaps providing opportunities for us to enhance it moving forward). He might be wrong but as he says in his paper: “if it walks like a duck,
quacks like a duck, and looks like a duck, maybe it is a duck.”

I recommend giving it a read, I think it is on a level for the general public to understand it without knowing too much about Hebb, neuroscience, or the LMS algorithm:

http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7296723