I think this is why “nothing seems wrong” has never felt very comforting to me at work. Some of the worst situations I’ve seen were quiet for quite a while. Not healthy quiet. Just the kind of quiet you get when the system has stopped noticing, and people confuse that with control.
A loud problem at least gives you something real to react to. A silent one gets protected by dashboards, routines, and the general human desire to believe that no alarm means no danger. That is what makes it so expensive.
Inga - have you ever heard the phrase "you're yelling so loud I can't hear you!"? This is the situation here where a 'display' or even a "sense" of something wrong is being innocently overlooked, sidestepped or worse, deliberately built-up aside from without proper 'purple-teaming' (red-team and blue-teams collaborating together to "share insights in real-time to immediately improve defenses") And can AI truly address this?
I wonder if there's really something to the (sometimes labeled as "suspicious") phrase "something's not right here.." Hmmm🤔, or as Mike Fisher would say "that's a beard scratcher..."
As AI continues to be a trusted "partner" across organizations, I believe this message becomes even more important. AI will do tasks you ask, work will be produced, engines will run. How as a individual or company are you confirming AI is turning the right gears, answering the prompt correctly, and most importantly, not creating a false sense of control over a process.
Fantastic article , Fish. After reading this I immediately thought about the game of football and it's zillions of strategies for offense-defense applications. Here's something you wrote that really caught my attention:
"The problem was that their alarm system, the very mechanism designed to surface danger, had failed silently. The absence of alerts was interpreted as the absence of problems. Silence became reassurance. Product teams fall into the same trap." Yes, and so do NFL teams it seems.
An example would be let's say, at the start of a game a team comes crashing out of the gate and totally obliterates the other team through all four quarters to the last 2 minutes (or in some cases the last 20 seconds https://www.youtube.com/watch?v=GP4Xr44LP98 ) until... "what's that, no way, we're way ahead", people leaving the stadium thinking ok, time to head home, etc. And then, oopsie-daisy, the gob smacked losing team heads to the locker room tail-between-legs, long faces and a coach not being able to say much yet trying to remain composed..
Regarding power grid and high security Corporate or Government businesses, I'm thinking, never assume a task is complete without it's Fail-Safe programs being monitored in real time. This eliminates the possibility of no-alarms, what alarms? "quiet-time" deceptive security. Usher in the combination of Red-Teaming and Blue-Teaming - and I'm sure that "Purple-Teaming" is something the AI community is feverishly working on..
In the case of football however, the upstairs in-the-booth coaches also need to be monitored and advised in real time. Red-teaming is simulating an "outside" what-if team testing umpteen scenarios for the Offense to consider and implement, as well as Blue-teaming (Offense protection) likewise for it's Defense, BUT, they both are typically performed before or after the games so as being reviewable.
..So, what if there was a system running that had an Official 'Purple-Teams' (like Special Teams) Director(s) simultaneously (from AI input) overseeing the game in real time, with direct communication to the Head Coach, Offensive & Defensive Coordinators and almost (except for ONLY the Head Coach) being given final game-call authority before each play? Whhaaaat??
Sounds preposterous, doesn't it? - Yeah, me too!
Fish, thank you so much for your thought-provoking, informative (and might I add, creative) articles. They are truly inspiring, and always get the "little gray cells" moving!
I think this is why “nothing seems wrong” has never felt very comforting to me at work. Some of the worst situations I’ve seen were quiet for quite a while. Not healthy quiet. Just the kind of quiet you get when the system has stopped noticing, and people confuse that with control.
A loud problem at least gives you something real to react to. A silent one gets protected by dashboards, routines, and the general human desire to believe that no alarm means no danger. That is what makes it so expensive.
Inga - have you ever heard the phrase "you're yelling so loud I can't hear you!"? This is the situation here where a 'display' or even a "sense" of something wrong is being innocently overlooked, sidestepped or worse, deliberately built-up aside from without proper 'purple-teaming' (red-team and blue-teams collaborating together to "share insights in real-time to immediately improve defenses") And can AI truly address this?
I wonder if there's really something to the (sometimes labeled as "suspicious") phrase "something's not right here.." Hmmm🤔, or as Mike Fisher would say "that's a beard scratcher..."
As AI continues to be a trusted "partner" across organizations, I believe this message becomes even more important. AI will do tasks you ask, work will be produced, engines will run. How as a individual or company are you confirming AI is turning the right gears, answering the prompt correctly, and most importantly, not creating a false sense of control over a process.
Fantastic article , Fish. After reading this I immediately thought about the game of football and it's zillions of strategies for offense-defense applications. Here's something you wrote that really caught my attention:
"The problem was that their alarm system, the very mechanism designed to surface danger, had failed silently. The absence of alerts was interpreted as the absence of problems. Silence became reassurance. Product teams fall into the same trap." Yes, and so do NFL teams it seems.
An example would be let's say, at the start of a game a team comes crashing out of the gate and totally obliterates the other team through all four quarters to the last 2 minutes (or in some cases the last 20 seconds https://www.youtube.com/watch?v=GP4Xr44LP98 ) until... "what's that, no way, we're way ahead", people leaving the stadium thinking ok, time to head home, etc. And then, oopsie-daisy, the gob smacked losing team heads to the locker room tail-between-legs, long faces and a coach not being able to say much yet trying to remain composed..
Regarding power grid and high security Corporate or Government businesses, I'm thinking, never assume a task is complete without it's Fail-Safe programs being monitored in real time. This eliminates the possibility of no-alarms, what alarms? "quiet-time" deceptive security. Usher in the combination of Red-Teaming and Blue-Teaming - and I'm sure that "Purple-Teaming" is something the AI community is feverishly working on..
In the case of football however, the upstairs in-the-booth coaches also need to be monitored and advised in real time. Red-teaming is simulating an "outside" what-if team testing umpteen scenarios for the Offense to consider and implement, as well as Blue-teaming (Offense protection) likewise for it's Defense, BUT, they both are typically performed before or after the games so as being reviewable.
..So, what if there was a system running that had an Official 'Purple-Teams' (like Special Teams) Director(s) simultaneously (from AI input) overseeing the game in real time, with direct communication to the Head Coach, Offensive & Defensive Coordinators and almost (except for ONLY the Head Coach) being given final game-call authority before each play? Whhaaaat??
Sounds preposterous, doesn't it? - Yeah, me too!
Fish, thank you so much for your thought-provoking, informative (and might I add, creative) articles. They are truly inspiring, and always get the "little gray cells" moving!