Digital infrastructure exceeding limits of human control, industry experts warn

Juliette Garside theguardian.com A series of system crashes affecting Google, Amazon, Apple andMicrosoft in the past fortnight has brought warnings that governments, banks and big business are over-reliant on computer networks that have become too complex. The alarm was sounded by industry experts in the aftermath of a three-hour network shutdown that paralysed the operation of the Nasdaq stock market in New York on Thursday, on what should have been a quiet day of routine share trading on the exchange. Jaron Lanier, the author and inventor of the concept of virtual reality, warned that digital infrastructure was moving beyond human control. He said: "When you try to achieve great scale with automation and the automation exceeds the boundaries of human oversight, there is going to be failure. That goes for governments, for consumer companies, for Google, or a big insurance company. It is infuriating because it is driven by unreasonable greed. In many cases, the systems that tend to fail, fail because of an attempt to make them run automatically with a minimal amount of human oversight." The Nasdaq collapse was caused by a communication failure between its platform for processing quotes and trades and that of another party – reportedly the New York Stock Exchange. So serious was the fallout that it resulted in a third fewer shares being traded in the US on that day. "These outages are absolutely going to continue," said Neil MacDonald, a fellow at technology research firm Gartner. "There has been an explosion in data across all types of enterprises. The complexity of the systems created to support big data is beyond the understanding of a single person and they also fail in ways that are beyond the comprehension of a single person." From high volume securities trading to the explosion in social media and the online consumption of entertainment, the amount of data being carried globally over the private networks, such as stock exchanges, and the public internet is placing unprecedented strain on websites and on the networks that connect them. By 2017, an amount of data equivalent to all the films ever produced will be transmitted over the internet in a three-minute period, according to Cisco, a manufacturer of communications equipment. Internet traffic today per person is measured in gigabytes, with six gigabytes of information exchanged per human per year. In 2017, that number will have risen to 16. By then, global data will be counted in zettabytes – roughly one trillion gigabytes. to read more click here: theguardian.com