So-called “AI” is already being used in several US states to predict whether candidates for parole will reoffend, with critics claiming that the data the algorithms are trained on reflects historical bias in policing. Recently, Facebook’s news feed algorithm asked users who saw a news video featuring black men if they wanted to “ keep seeing videos about primates”. But datasets are not simply neutral repositories of information they often encode human biases in unforeseen ways. Machine learning works by training the machine on vast quantities of data – pictures for image-recognition systems, or terabytes of prose taken from the internet for bots that generate semi-plausible essays, such as GPT2. More recently, Deepmind’s AlphaFold 2 was greeted as an important milestone in the biological field of “protein-folding”, or predicting the exact shapes of molecular structures, which might help to design better drugs. In this way, AlphaZero, the chess-playing entity created by the British firm Deepmind (now part of Google), played millions of training matches against itself and then trounced its top competitor. How do we get there from here, assuming we want to? Modern AI employs machine learning (or deep learning): rather than programming rules into the machine directly we allow it to learn by itself.

That is sometimes called AGI, for artificial general intelligence. The second sense, from which the first borrows its glamour, points to a future that does not yet exist, of machines with superhuman intellects. The first sense is, essentially, a marketing one: anything computer software does that seems clever or usefully responsive – like Siri – is said to use “AI”. Some confusion is caused by two very different uses of the phrase artificial intelligence.

Nick bostrom books