I just finished reading Henry Petroski’s To Engineer Is Human: the Role of Failure in Successful Design. I liked it.
One of the central themes of the book is that there is much more to learn from the failure of a design than from the success of one. When a structure collapses, the wreckage can be studied, and the cause identified, providing future designers with a definite mode of failure that they can plan to avoid and check for in their own designs.
Another major theme is an analogy of the design process as a scientific method. In science, a hypothesis can be disproved by experimentation, but it cannot be proven. If an experiment fails to disprove an hypothesis, it merely adds supporting evidence to it. The hypothesis may be incorrect, but no experiment has yet exposed its failure. Similarly, a design can be found lacking by analysis, but there is no guarantee that one which passes all the known analyses will, in fact, stand.
There is an oft-quoted saying among computer scientists from Dijkstra that makes a similar point: “Program testing can best show the presence of errors but never their absence.”