You are viewing a single comment's thread. View all
0
MrBaptist on scored.co
1 year ago0 points(+0/-0)1 child
Incompetence/budget/etc. is always the safer bet, no nefarious fed scheming required.
Look at the CrowdStrike outage earlier this year. They broke every customer simultaneously - they must have been YOLOing updates for a while now, and they got away with it until one bad update took it all down. Compare with Microsoft using previews, staging, etc. - even if a bad update affects some customers, the rollout halts automatically well before everyone breaks.
Another fun one I remember is [MGM Resorts](https://ww.reddit.com/r/sysadmin/comments/16owgsl/anyone_wanna_help_rebuild_mgm_resorts/), where they went down so badly they decided to start over:
> Anyone Wanna Help Rebuild MGM Resorts? They're looking for a Red Hat admin who will work every day until Humpty Dumpty is rebuilt from scratch.
> 10 hour shifts 7 days a week with people swaping in and out [...] For $100/hr
Look at the CrowdStrike outage earlier this year. They broke every customer simultaneously - they must have been YOLOing updates for a while now, and they got away with it until one bad update took it all down. Compare with Microsoft using previews, staging, etc. - even if a bad update affects some customers, the rollout halts automatically well before everyone breaks.
Another fun one I remember is [MGM Resorts](https://ww.reddit.com/r/sysadmin/comments/16owgsl/anyone_wanna_help_rebuild_mgm_resorts/), where they went down so badly they decided to start over:
> Anyone Wanna Help Rebuild MGM Resorts? They're looking for a Red Hat admin who will work every day until Humpty Dumpty is rebuilt from scratch.
> 10 hour shifts 7 days a week with people swaping in and out [...] For $100/hr
>5 years exp
>$100/hr, 10hr/7d, on-call extra, 3 weeks
Sounds like they were looking for exactly what got them in that troube to begin with: poojeetz