High Reliability Organizations (HROs) are organizations that operate in high-risk domains but reliably avoid catastrophic failures. Examples include nuclear plants, air traffic control, and aircraft carriers. Research into HROs aims to determine how they achieve extreme reliability and whether these lessons apply to AI companies working on dangerous technologies.
Key HRO insights include: tracking failures to learn; avoiding oversimplification; staying operationally sensitive; committing to resilience; deferring to expertise; and an "informed culture" that reports issues, avoids blame, and fosters flexibility and learning.
HRO literature may provide useful principles for AI companies, but different feedback loops and job functions limit applications. Research into fields like biotech may also be relevant....