so do we trust the programmers?
Here's an interesting thing... years ago when the Airbus A320 had just came out there was a demo flight organised for a bunch of journalists by Air France over Mulhouse. The aircraft was do do a low pass over the runway and then pull up and continue on its way. As the aircraft followed the ILS approach to the runway the computers (5 independent flight computers) decided the aircraft was going to land and configured the aircraft thus, which was okay for the Pilot (lad called Scanlon I thnk..??) anyway he was happy with the initial approach speed height etc and proceeded on the ILS. As he approached the heisght he wanted to do his fly past he tried to level out the aircraft and applied thrust. The aircraft's computers decided..non! We want to land.! The pilot kept pulling back on the controls and the aircraft compensated by still trying to put the aircraft on the ground so the aircraft became unstable in a stall attitude. Scanlon tried to power out and increased thrust the aircraft still tried to go lower until it got itself into a ridiculous stall attitude and went into what is called a compressor stall - effectively the angle of the aircraft is so high that the airflow into the engines is disrupted and the compressors are effectively starved of air. Upshot is the aircraft crashed.
There was an investigation and on the behest of Airbus and Air France it was determind that the aircraft crashed because of pilot error...too low too slow etc.. Those of us in the know...wink wink... have another theory in that the flight computers were supposed to be independed of each other. 4 computers were to analyse and apply the instruction of the pilot with the fifth to "vote" that the other four were doing the correct thing. The issue was that fly-by wire for civil aircraft was not that advanced and that there were not enough programmers qualified at Airbus to undertake the such a monumetal program in such a short space of time. Each flight computer was supposed to be programmed by a completely seperate team with no communication or cribbing between teams...effectively they were to be entirely independent. The only problem was that all the programmers underwent training through one team and that training team had flawed applications so each team was taugh the same flaw. The computers all contained the same programming algorythm that decided that if the aircraft was appraoching at a certain speed, height attitude and under ILS control the aircraft would land. The pilot was the victim of a flawed programme which due to political and business reason could bot be made public. The pilot was hung out to dry. Anyway that's my conspiracy theory of the day..
Upshot...don't trust AI
Sounds like the pilot needed an off switch.
Reminds me of the computer in 2o01, A Space Odyssey that took over the space craft.
It's also a bit like a feature on my new car, that keeps it in the center of the lane. If I start to wander too close to the line, it automatically steers the car back to the center. That's all well and good until there is a vehicle, usually a semi, hugging the line and the dumb thing doesn't want me to move over and give it room. Lucky for me, I do have an off switch.
It also has what's called "adaptive cruise control." If I have the cruise set, it will slow the car when traffic ahead slows. It will stop if the car ahead stops. If I have enough faith in the machine, which is rare, I don't have to do a thing. Problem is, the computer doesn't recognize when the 'object" ahead that is stopped is not a car. It tried to stop for a big sheet of plastic that had become airborne in the middle of the freeway, really a good way to get run over. Fortunately for me, I have an off switch.
Now, if I'm going to send an AI drone to kill someone I've decided needs to be killed, I can't give that person an off switch.
