Great show this morning! Whatever forces in the universe that caused me to be aware of your work, I am deeply grateful. Thank you for the work you put in and I appreciate your approach and take on things.
Listen, I know you are busy (I have 5 kids myself) and I don’t want to take too much of your time, but I am finally passionate about something lately. This Facebook outage has me very intrigued. I have been a network engineer/architect for a long time (20+ years) and you are correct in thinking there is more.
The FB explanation is plausible, however very unlikely and I just wanted to share some thoughts in more detail; something chat does not allow sometimes.
My experience in this industry coupled with the fact I have two former colleagues working at FB in the network group leads me to think I have a good handle on what MIGHT be going on. Here is what I believe we can say we some degree of comfort is known:
• Facebook’s network is comprised primarily of Cisco technology.
• Facebooks network investment is measured in the billions of dollars
• Facebooks network has at least 3 environments: test, dev or staging, and production
• Facebook has a rigorous change control management system
• Door scanners/magnetic locks failed
• Internal applications and services failed (remember this)
• Zuck takes a at least a modest amount of pride in the network
• Outage lasted 5- 6 hours
• The recently resigned CTO is a good friend of Zucks
• Zuck lost a lot of personal wealth
• FB lost millions
• Media is barely covering the story
What you said about contradictions not existing is true. Let’s take some plausible theories and see how they stack up to what we know (ranked from least likely to most):
1. Premise - This was an accident made by some network engineer who was making changes to the BGP core on Monday morning (Offical FB press release)
a. Contradiction – no way they make this change without first running it through the change board
b. Contradiction – this change would have had to been run through test environment successfully prior to change board approval.
c. Contradiction – a reasonable backout plan would have to be detailed to the change board.
d. Contradiction – Cisco DNA Center would have been used (or some version such as Prime) to ensure quick recovery and regression
e. Contradiction – billions of dollars spent on a network that a single network engineer can take down in minutes…accidentily
It is safe to say that this premise is flawed and not to be believed
2. Premise – Blackhat did this to delete some FB data or some other reason because of the “whistleblower” testimony.
a. Contradiction – Zuckerberg lost billions
b. Contradiction – CTO resigned
c. Contradiction – FB lost millions
d. Contradiction – they could have done this at any time off hours. This was done in prime time (for maximum impact maybe)
It is highly unlikely that black hats would cause so much damage for some singular objective
3. Premise – Internal sabotage by some whitehat within FB
a. Contradiction – it would NOT have taken 5 to 6 hours to recover (esp with the use of DNA Center)
b. Contradiction – BGP is a dynamic protocol that relies on passive network discovery of adjacencies. It broadcasts adjacencies very quickly and often.
c. Contradiction – if the saboteur was physically on the inside they would have risked being locked in (magnetic door lock/key cards failed)…not likely.
d. Contradiction – if the saboteur had remote access (ie VPN, Citrix, or RDP) that would have been lost as well and would not be likely method.
e. Contradiction – if the saboteur was a whitehat the media would have run with the story of a Trump supporting domestic terrorist much like the vehicle outside the SCOTUS. Instead, we hear crickets.
It is plausible that this is still the case and FB is covering this up. It would be embarrassing to have an internal saboteur. However, this still does not explain the 5 to 6 hour downtime during the day.
4. Premise – Warning/Dry run by Whitehats to signal 10 day blackout (X22 report)
a. Contradiction – Again, media would have run with the narrative of a Trump saboteur
b. Contradiction – no need for a dry run of predicted blackout; the Deepstate Media might do this anyway in any number ways
c. Contradiction – A warning to FB by whitehats against further censorship is just that…a warning. They will still censor and we know that.
It is still very plausible that this is a warning from whitehats against further censorship and I like aggressive nature of it
5. Premise – A dry run by some “no such agency” to penetrate, compromise, and damage FB
NO CONTRADICTIONS
a. The fact that this was done on a Monday for maximum impact screams of a Trump tactic
b. An external penetration by some agency to appear internal and cause damage is pretty slick
c. Is it possible that FB and others could be compromised by Space Force in the future? EO13848 gives them the legal standing to do so…domains and urls are assets listed in that EO.
d. I do believe that at some point soon, social media assets will be confiscated. Could we have witnessed a preview?
The point is that however weak or insignificant the contradictions might be, each has a contradiction except one in my opinion. As of this writing my FB colleague has held the company line and said the official Facebook press release is the truth and that they are “learning from it.” He knows I am skeptical but cannot/will not divulge more at this time.
One final thought: when I read EO13848 and it listed assets like domain and urls of those who worked with foreign entities I thought of a scenario very similar to this where some agency takes the BGP ASN offline, changes A records in DNS to point urls to some govt block of IP addresses, turns the ASN back up, and uses forwarders back to FB….just saying.
If you read this far, thank you. I hope something I said might help us all try and give structure to this black hole we see in front of us.
UPDATE: I have just learned that Peggy Hubbard (conservative politician in Illinois) who I followed on FB was suddenly reinstated on Monday after a year of no FB access.
Written 3 days after FB outage on October 8th.
I remember this from Kyle's stream. Extremely impressive take. NOTABLE.