AI got the blame for the Iran school bombing. The truth is far more worrying
Lots of interesting and potentially controversial points depending on your leanings, but I think there is at least one potentially universal truth included:
Personnel became disinclined to ask whether some targets were potential allies, or not actually bad guys at all, because producing targets meant participating in the hunt. The targeting guide had warned about this too. “If targeteers don’t provide full targeting service,” it read, “then other well meaning but undertrained and ill-experienced groups will step in.”
Unless your users are locked into your system by higher powers with no alternatives allowed, if you don't give them enough of what they want - no matter how objectively bad or risky it is, they will just ignore your system and/or exploit loopholes.