Machine Learning and People

Last week I attended a fascinating meeting led by Dr. Lex Fridman of MIT reporting about research on machine learning as applied to autonomous cars. His research involved experiments with autonomous  cars and human drivers. The objective was to get solid evidence about how humans and autonomous driving systems interact with each other and with others using the road. At ARC we are more interested in the manufacture of cars than in their operation — whether by human or autonomous means. So I wasn’t expecting to find very much that would be relevant to issues my clients care about.

WRONG!!!

I was surprised that several aspects of the research resonated with issues ARC hears about all the time. Granted the applications and the defined problems were very different. Here are a few research problems that may sound familiar:

Video intensive data

The research captured sensor data from the CAN bus of the vehicles. That CAN data was a drop in the bucket. The data deluge came in the form of video streams. Multiple video cameras, focused on the road and the driver were the most valuable data sources, and by far the most difficult to analyze. How related? In some industries (oil, gas and utilities) owner-operators perform regular aerial video surveys of linear assets like pipeline right-of ways and transmission lines. How much of this HUGE store of data do they effectively analyze and use for decision support? Pretty much none of it, according to my clients.

Sparse data sets

One of the research objectives was to analyze interactions with pedestrians who might walk directly in front of the vehicle. It turns out this was a much rarer occurrence than one would expect. Less than .1% of the video data was useful for making models of this interaction. How related? This is as rare (and as important) as “abnormal situations” in a process plant.

Data identification difficulties

Both in researching pedestrian interactions and self-driving algorithms, identifying the relevant portions of the huge data store was difficult. How do you identify from the data the specific times when self-driving becomes a safety issue? Is it when the steering wheel is turned rapidly? When the vehicle accelerates or decelerates quickly? When another vehicle is in close proximity? Some combination of these? Etc, etc. These conditions are not easy for researchers to identify algorithmically.

Speed vs.safety

As all human drivers know (save in Massachusetts) there is a trade-off between speed and safety. How does an algorithm optimize this choice?

What you “know” that ain’t so

It’s revered folk wisdom in Boston that “the driver with the right of way is the last one to make eye contact.” False, report the researchers. Instead, drivers choose their actions by estimating the ability of other drivers to slow down or stop.

Culture matters much

Consider one single data point; Is the car’s horn blowing? It is or it isn’t, but what does that imply? If you don’t know the culture where the data came from, you can’t say. Recall there are probably 1-10 million car horn honks in Bangalore for every 1 honk in Tokyo. The message conveyed by the same signal is very different.