Sounds like the pilot needed an off switch.
Reminds me of the computer in 2o01, A Space Odyssey that took over the space craft.
It's also a bit like a feature on my new car, that keeps it in the center of the lane. If I start to wander too close to the line, it automatically steers the car back to the center. That's all well and good until there is a vehicle, usually a semi, hugging the line and the dumb thing doesn't want me to move over and give it room. Lucky for me, I do have an off switch.
It also has what's called "adaptive cruise control." If I have the cruise set, it will slow the car when traffic ahead slows. It will stop if the car ahead stops. If I have enough faith in the machine, which is rare, I don't have to do a thing. Problem is, the computer doesn't recognize when the 'object" ahead that is stopped is not a car. It tried to stop for a big sheet of plastic that had become airborne in the middle of the freeway, really a good way to get run over. Fortunately for me, I have an off switch.
Now, if I'm going to send an AI drone to kill someone I've decided needs to be killed, I can't give that person an off switch.
