TroostplekThe internet resting place

In memoriam · book of condolence

Bernard Widrow

also known as Bernard Widrow

Born
24 December 1929
Died
30 September 2025
Age
95 years
Resting places
Scientists & inventors Famous people, United States Countries

Biography

Bernard Widrow (December 24, 1929 – September 30, 2025) was an American professor of electrical engineering at Stanford University known for his work on artificial neural networks. He co-invented the Widrow–Hoff least mean squares filter (LMS) adaptive algorithm with his doctoral student Ted Hoff. The LMS algorithm led to the ADALINE and MADALINE artificial neural networks, and to the backpropagation technique. He made fundamental contributions to the development of digital signal processing in the fields of geophysics, adaptive antennas, and adaptive filtering.

Widrow was the namesake of "Uncle Bernie's Rule": the training sample size should be ten times the number of weights in a network.

Biography

This section is based on.

Early life and education

Widrow was born in Norwich, Connecticut. While young, he was interested in electronics. During WWII, he found an entry on "Radios" in the World Book Encyclopedia, and built a one-tube radio.

He entered MIT in 1947, studied electrical engineering and electronics, and graduated in 1951. After that, he got a research assistantship in the MIT Digital Computer Laboratory, in the magnetic core memory group. The DCL was a division of the Servomechanisms Laboratory, which was building the Whirlwind I computer. The experience of building magnetic core memory shaped his understanding of computers into a "memory's eye view", that is, he "look for the memory and see what you have to connect around it".

For his masters thesis (1953, advised by William Linvill), he worked on raising the signal-to-noise ratio of the sensing signal of magnetic core memory. Back then, the hysteresis loops for magnetic core memory was not square enough, making sensing signal noisy.

For his PhD (1956, advised by William Linvill), he worked on the statistical theory of quantization noise, inspired by work by William Linvill and David Middleton.

During PhD, he learned the Wiener filter from Lee Yuk-wing. To design a Wiener filter, one must know the statistics of the noiseless signal that one wants to recover. However, if the statistics of the noiseless signal is unknown, this cannot be designed. Widrow thus designed an adaptive filter that uses gradient descent to minimize the mean square error. He also attended the Dartmouth workshop in 1956 and was inspired to work on artificial intelligence

Work on AI

In 1959, he got his first graduate student, Ted Hoff. They improved the previous adaptive filter so that it made a gradient descent for each datapoint, resulting in the delta rule and the ADALINE. To avoid having to hand-tune the weights in ADALINE, they invented the memistor, with conductance (ADALINE weights) being the thickness of the copper on the graphite.

During a meeting with Frank Rosenblatt, Widrow argued that the S-units in the perceptron machine should not be connected randomly to the A-units. Instead, the S-units should be removed, so that the photocell inputs would be directly inputted into the A-units. Rosenblatt objected that "the human retina is built that way".

Despite many attempts, they never succeeded in developing a training algorithm for a multilayered neural network. The furthest they got was with Madaline Rule I (1962), which had two weight layers. The first was trainable, but the second was fixed. Widrow stated their problem would have been solved by the backpropagation algorithm. "This was long before Paul Werbos. Backprop to me is almost miraculous."

Adaptive filtering

Unable to train multilayered neural networks, Widrow turned to adaptive filtering and adaptive signal processing, using techniques based on the LMS filter for applications such as adaptive antenna, adaptive noise canceling, and applications to medicine.

At a 1985 conference in Snowbird, Utah, he noticed that neural network research was returning, and he also learned of the backpropagation algorithm. After that, he returned to neural network research.

Publications

1965 "A critical comparison of two kinds of adaptive classification networks", K. Steinbuch and B. Widrow, IEEE Transactions on Electronic Computers, pp. 737–740.

1985 B. Widrow and S. D. Stearns. Adaptive Signal Processing. New Jersey: Prentice-Hall, Inc., 1985.

1994 B. Widrow and E. Walach. Adaptive Inverse Control. New Jersey: Prentice-Hall, Inc., 1994.

2008 B. Widrow and I. Kollar. Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications. Cambridge University Press, 2008.

Honors

He was one of the Board of Governors of the International Neural Network Society (INNIS) in 2003.

Source: Wikipedia, available under the licence CC BY-SA 4.0.

In Memoriam web link

en.wikipedia.org/wiki/Bernard_Widrow

Photos

Condolences and flowers 0

The book of condolence for Bernard Widrow: offer your condolences to the family below, lay a flower or light a candle.

  • TroostplekOur sympathy to all who are left behind.

Be the first to offer your condolences or lay a flower.

Your condolences or a flower

What would you like to place?
How long is your message?
Choose a flower

All 27

Wreaths 4

Funeral bouquets 3

Classic 20

How long should your flower stay?

Lay flowers more often? Become a member: lay as many flowers as you like, and they all stay forever. €6.00 a month or €60.00 a year. Become a member (with a free account)

0 / 500 characters

Troostplek is a place to remember. Messages are read first and then placed; messages that do not fit here are removed without explanation.

Let your flower stay foreverPost a longer condolence

€3.95€3.45 once

  • Your flower is laid now and can be seen straight away.
  • As soon as your payment has arrived, it stays forever.
  • If the payment does not go through, it stays for a month, like a free flower.
  • Your condolence is posted now. Until your payment has arrived, only the first 500 characters can be read.
  • As soon as your payment has arrived, the whole text can be read.
  • Rather free? Go back and make your message shorter than 501 characters.
How would you like to pay?

With crypto you pay the network's transaction costs yourself, on top of the amount. With iDEAL and a bank transfer there are no extra costs.

With a bank transfer or crypto you will see how to pay after placing it. Fill in your email address to also receive it by email.

Lay flowers more often? As a member you lay as many flowers as you like, forever, from €6.00 a month.Writing longer messages more often? As a member you write as much as you like, for free, from €6.00 per month. Become a member

Report

Does this message or flower not belong on Troostplek? After your report it is hidden until we have looked at it.

Report a mistake or addition

About the memorial of Bernard Widrow. Fill in what should be different; the rest may stay empty.

Now: Bernard Widrow

Now: 24 December 1929

Now: 30 September 2025

Now: Scientists & inventors