Crowdstrike Hubris

(CNN) Global tech outage disrupts airlines, banks, hospitals, 911 services.

Frank Sprague,  an early developer and manufacturer of electrical equipment has been credited with the dead man’s switch, a safety device for electric trolley cars, rail locomotives , elevators, and buses. The operator is required to keep his foot on a pedal or hand on a grip.  If the operator is incapacitated or dies, his foot or grip relaxes, which causes the vehicle to automatically stop. Analogous hardware would have prevented global meltdown.

Crowdstrike undoubtedly has a software analog of the above, detecting loss of functionality.  If the analog runs on the same machine it protects, it cannot report all kinds of failure. Sometimes the murder victim manages to leave a note;  usually not. So this massive update kept running without informing Crowdstrike it was killing machines.

There used to be a simple way of checking whether a remote machine was still alive. The ping network utility causes a dialog with a remote machine a little like the  dialog  of harmoniums in Kurt Vonnegut’s The Sirens of Titan:

Query; “Here I am, Here I am, Here I am.”

Response: “Yes you are, Yes you are, Yes you are.”

Now days, with computers protectively hidden behind routers with complex routing rules, it is not advisable to expose to pinging from outside connections. Given the large size of most Crowdstrike clients, it is entirely feasible for an organization to have a dedicated hardware box that reports to Crowdstrike if their automatically pushed update is causing mass death.

The absence of dedicated dead-man hardware  is technological hubris.