COEN 286: Software Quality Assurance and Testing -- fall, 2001

Bugs: Failures, Faults, Mistakes, and Error.

Prof. John Noll

Santa Clara University

1. Quality

• A function of:
  • the features that make the customer want to use the program, and
  • the flaws that impede that use.
• The number of bugs per (loc, statement, staff-hour, ...).
• Uptime.

2. Definitions

Failure

• A mismatch between the program and its specification
• A mismatch between the program's behavior and the user's expectation of that behavior.

Error

The degree to which a failure deviates from expected behavior.

Fault

The incorrect design element, implementation item, or other product fragment that caused a failure.

Mistake

The human act that injected a fault.

Summary:

Programmers make mistakes. Mistakes result in faults. Faults can cause failures. Failures are observed as errors. (Or not; is it possible for a failure to occur without observable error?)

3. Failure Categories

• UI
  • Correctness
  • Completeness
  • Usability
• Error Handling
• Boundary Conditions
• Calculation (Algorithmic)
• Initial State
• Control Flow
• Data Interpretation/Handling
• Race Conditions
• Load Conditions
• Hardware
• Configuration Mgt
• Documentation
• Testing