Dr. Shewhart's principal question was how AT&T, Western Electric's parent company, could be assured that their wires, telephone poles, and indeed, their telephones would function as desired. They were particularly interested in consistency. Each phone needed to be the same, which is to say, predictable and “perfect”. Inspired by a philosopher, C.I. Lewis, Dr. Shewhart published his work publicly with the encouragement of a newly-minted PhD that had worked a summer there, a man named W. Edwards Deming.
Dr. Deming was a fan of the technique, known as statistical process control (SPC), but he worked as a statistical sampler for the federal government, the Department of Agriculture and the Census. He encouraged teaching of SPC during World War II to build up dependable materiel, with admirable, historic results. Over 31,000 production managers had been trained.
After the war, U.S. producers abandoned the practice, to Deming's open disgust. He went to Japan under MacArthur as a census taker after the war. When he saw the devastation, he asked if Japanese government and business leaders were interested in learning the methods. They were. They mastered it. For example, when faced with the need to meet a 4% error rate to meet the requirements of a U.S. customer, a Japanese firm famously delivered the product in two groups, the 96% perfect products and the 4% defective ones, wondering aloud why the Americans oddly wanted the defective units.
The world balance of economic power shifted.
Though many organizations in the West and elsewhere also embraced the methods and extensions added by the Japanese, known as “lean production”, continual improvement was spotty. For one thing, the founding principles faded with time. For another, the benefit of SPC was never really understood as an overriding corporate model; it was relegated to the production floor for the most part.
Another significant problem was the advent of more widespread computerization when SPC and lean production came together, for the most part in the early 1990s. This was also the time of “reengineering”, which took in some of the SPC concepts, but without a substantive base. In the maze of “process owners” and “process customers” and a sea of other proto-concepts, the kernels of truth represented by SPC were lost.
The biggest problem was computerization, though. Complexity on top of complexity on top of complexity. The Year 2000 Problem took its toll as well. It served as a Trojan Horse, reducing all activities in the organization to what the technologists either agreed with, understood, felt like doing, or knew how to computerize in some fashion, using some tool that itself brought in other issues.
That era, from the 1990s to the earliest decade of the 2000s, involved wholesale dis-empowerment of mostly everyone within organizations – particularly those who had creative ideas and who espoused new ways of doing things. Such people, rather than being seen as “improving and perfecting the system” were viewed as “introducing new risks” and “wasting peoples' time” and “never being satisfied”. Most people developed very jaded attitudes during this period. Many gave up on rationality, let alone perfection.
The result was very different from stories that had previously come from Japan in the era of SPC and lean production. Loss of hope. Huge, failed information processing projects. People just giving up and leaving just out of frustration.