The OT Stability Doctrine

1 · Validated configurations are sacred

When a system configuration has been tested, validated, and proven stable in production, changing it requires extraordinary justification. Every firmware update represents weeks-to-months of compatibility testing, risk of new bugs, disruption to proven procedures, and revalidation of all connected systems.

2 · Downtime costs are catastrophic

• Automotive manufacturing — $22,000 per minute of downtime
• Oil & gas production — $100,000+ per hour of unplanned outage
• Pharmaceutical batch loss — millions in destroyed product
• Utility disruption — public safety and regulatory penalties

3 · Interoperability is complex

OT environments include equipment from dozens of vendors, systems installed across multiple decades, proprietary protocols and interfaces, custom integrations and modifications. A single firmware change can cascade through these interdependencies in unpredictable ways.

4 · "Security" updates can reduce security

New firmware may change communication patterns; updates can break existing security monitoring; changed behaviors may bypass safety systems; untested code is inherently less secure than proven code.

Comprehensive change control

• Business justification for any update
• Full impact analysis across all systems
• Rollback procedures and testing
• Documentation of all changes and rationales

Lifecycle planning

• Firmware versions matched to equipment lifecycle
• Spare parts inventory includes matching firmware
• Knowledge retention for legacy versions
• Vendor support agreements for extended lifecycles

Risk-based decision making

• Updates only when risk of not updating exceeds risk of change
• Operational requirements always supersede IT policies
• Production stability is the primary metric
• Safety implications override all other considerations

• Year 1 — three production outages, $2.4M in downtime
• Year 2 — compatibility issues forced hardware replacement, $8M cost
• Year 3 — return to OT-managed firmware policy
• Years 4–10 — zero firmware-related outages

The lesson. The "cost" of running old firmware was $0. The cost of updating it was $10.4M.

For OT operators

1. Document current firmware versions and why they were chosen
2. Establish clear criteria for when updates are justified
3. Maintain vendor relationships that respect stability requirements
4. Train IT partners on OT operational requirements
5. Create and enforce firmware management policies

For technology vendors

1. Design products for 15+ year lifecycles
2. Never force updates through artificial means
3. Maintain compatibility with ancient protocols
4. Price support realistically for long lifecycles
5. Respect that stability is a feature customers pay for

For IT/OT convergence

1. Establish separate policies for IT and OT systems
2. Recognize that OT stability requirements aren't negotiable
3. Design security controls that don't require frequent changes
4. Accept that OT and IT have different, valid approaches
5. Focus on outcomes (uptime, safety) not compliance checkboxes

1. Stability IS security in OT environments.
2. Firmware updates must be justified by operational need, not vendor schedules.
3. Professional OT management means knowing when NOT to change things.
4. Vendors must support old versions as a core business requirement.
5. IT and OT require different approaches — both are valid in their domains.