Why doesn't AI dig in its heels?

7.1 AI is not just about knowing more, but about continuing to update

We ended Chapter 6 with one word: updating. Reality keeps producing outcomes. The hard part is allowing those outcomes to put our original judgments to the test. In this chapter, we turn to machines. Machines make mistakes, of course. But the most interesting thing about machine learning is not how much a machine knows. It is that new samples, errors, and evaluation can continue to change its state. Training data, computing power, and model size all matter. But if storing more information were enough to constitute intelligence, hard drives would have taken over our companies long ago. They may contain millions of documents; we rarely invite them to be CEO. The more revealing questions are these: Does incoming information produce new capabilities? And when errors appear, can the system still change?

A model that initially could barely recognize digits

One of TensorFlow's public introductory tutorials presents a small task: recognizing handwritten digits. For a person, this means looking at a picture and deciding which number it shows. For a newly created, untrained model, the initial probabilities are close to random. The tutorial uses the images' correct labels to calculate error and adjust the model's parameters. Performance improves as training proceeds. When checked against test images that were not used in training, the recorded example reaches an accuracy of nearly 98 percent. The difference between almost random output and recognizing most test images is not a few extra files on a hard drive. The internal state involved in producing the model's judgments has changed. This remains a narrowly defined digit-recognition task. It does not show that the model understands the world, or guarantee similar performance in another setting. It does make updating tangible: faced with digit images, the trained system is no longer the same system it was before training.