AI and in particular machine learning has made great progress in the last decade. Yet I am deeply concerned about the hype associated with AI, and the risks to society stemming from premature use of the software. We are particularly vulnerable in domains such as medical diagnosis, criminal justice, seniors care, driving, and warfare. Here AI applications have begun or are imminent. Yet much current AIs are unreliable and inconsistent, without common sense; deceptive in hiding that they are algorithms and not people; mute and unable to explain decisions and actions; unfair and unjust; free from accountability and responsibility; and used but not trusted. 

Patient safety and peace of mind is aided in medical contexts because doctors, nurses, and technicians disclose their status, e.g., specialist, resident, intern, or medical student. This helps guide our expectations and actions. Most current chatbots are not open and transparent. They do not disclose that they are algorithms. They do not indicate their degree of competence and their limitations. This leads to user confusion, frustration, and distrust. This must change before the drive towards increasing use of algorithmic medical diagnosis and advice goes too far. The dangers have been illustrated by the exaggerated claims about the oncology expertise of IBM’s Watson. 

Ai algorithms are not yet competent and reliable in many of the domains anticipated by enthusiasts. They are brittle — they often break when confronted with situations only trivially different from those on which they have been trained.  Good examples are self-driving anomalies such as strange lighting and reflections, or unexpected objects such as kangaroos, or bicycles built for 2 carrying a child on the front handlebars. Ultimately, algorithms will do most of the driving that people now do, but they are not yet ready for this task. AIs are also shallow, possessing little innate knowledge, no model of the world or common sense, which researcher Doug Lenat, creator of the CYC system, has been striving to automate for four decades. 

But we expect even more of good agents beyond competence.  Consider a medical diagnosis or procedure.  We expect a physician to be open to discussing a decision or planned action.  We expect the logic of the decision or action to be transparent, so that, within the limits of our medical knowledge, we understand what is being recommended or what will be done to us.  We expect a decision or an action by an agent to be explainable. Despite vigorous recent research on explainable AI, most advanced AI algorithms are still inscrutable. 

We should also expect actions and decisions to be fair, not favoring one person or another, and to be just in terms of generally accepted norms of justice. Yet we have seen repeatedly recently how poor training data causes machine learning algorithms to exhibit patterns of discrimination in areas as diverse as recommending bonds, bail, and sentencing; approving mortgage applications; deciding on ride-hailing fares; and recognizing faces. 

If an algorithm rejects a résumé unfairly, or does a medical diagnosis incorrectly, or through a drone miscalculation injures an innocent person or takes a life, who is responsible?  Who may be held accountable? We have just begun to think about and develop the technology, the ethics, and the laws to deal with algorithmic accountability and responsibility. A recent example is an investor suing an AI company peddling super-computer AI hedge fund software after its automated trader cost him $20 million, thereby trying to hold the firm responsible and accountable. 

The good news is that many conscientious and ethical scientists and humanists are working on these issues, but citizen awareness, vigorous research, and government oversight are required before we will be able to trust AI for a wide variety of jobs These topics are discussed at far greater length in Chapter 11 of . Computers and Society: Modern Perspectives, Chapters 12 and 17 of  Digital Dreams Have Become Nightmares: What We Must Do, and also in The Oxford Handbook of Ethics of AI. 

FOR THINKING AND WRITING AND DISCUSSING 

What do you think? Are my expectations unreasonable? What issues concern you beyond those I have discussed?