Since this got such a surprising response, I feel compelled to write this as I find myself in an unexpectedly euphoric state of vindication today…
A year ago when my partner was planning her new business and asked me to do the IT I agreed on the sole condition that it’ll be a Linux network, my sanity be damned. It took months to piece together from scraps of painfully obtuse documentation, and after some last minute panics and sleepless nights(weeks??) it worked. They have a full AD DC samba network, linux samba servers (DC & NASs all dual redundant) with offsite rsync backup. All Debian workstations all configured via deployed policy from the DCs. The scope got so out of hand it nearly broke me a couple times, but, ADHD… Everything is theirs, and everything is onsite.
I got a message today when she read the news which was just ‘Everything working here’ I took today off because I couldn’t work out how to feel about it all, after last few weeks thinking I’d wasted a year of my life because as everyone in IT knows - no one cares what you did - but in just a day, it became all worth it.
FUCK MICROSOFT lol
I’m happy. The emails I get suck anyway and maybe we’ll get some digital sovereignty out of it. Seems like a win.
No one remembers the Anna Kornikova, Melissa, or I love you viruses.
It made me think about how we are fearing the potential of a Carrington event hitting us in the modern era, how bad it would be and how much it will take to build back.
It occurs to me that we could just have a ‘software calamity’ of similar scale, since we love putting so many eggs in so few baskets controlled by people that want to ditch all their expensive intelligent caretakers and slop things up into oblivion instead.
Oh you can already feel a new normal coming on.
Start saving bottlecaps everyone.
So glad I’m not doing user-facing IT support anymore. Last time something like this happened, my boss asked me to send an all-staff email explaining that email wasn’t working.
I got a text message alert
When it came back up today, the first message in there was from corporate IT support telling us that Outlook was down…
War never changes.
Sure thing boss man, I will get right on that.
For my group, basically everything came to a grinding halt. The more fun one is now when LLM’s go down. Oh, Claude not working today, guess no one knows how to get anything done any longer. FFFFFFF.
Fun fact: f in musical notation means “play it loud”
fff means “play it really, really, REALLY loud”
FFFFFFF would be “play it so loud the deaf lady in the back row shits herself. No one on stage should survive this noise”
Fortissimissimissimissimissimissimo
Last time I was the guy fixing exchange servers internally and for customers, while drunk/hangover newyear night and day
So what do we think it is this time, DNS or SSL?
Edit: According to the BBC it’s SSL:
What Microsoft has told us is that the part of its servers that sets up secure connections is getting swamped, and once that happens the server just stops answering,
A family I know was recently laid off from micro$oft. I imagin3 their schadenfreude feels pretty good right now.
The whole family?!
Mom and dad, yes!
And the children too?!
And the dog and the cat and two guinea pigs, both named Claire.
I’m in IT, previously this would have been something that gave me heartburn; wondering when it’ll be back up, trying to figure out work arounds, assuring my users. Now? Oh well. It’s down folks, it’ll be back up when it’s back up, and there’s nothing much we can do, so no need to get your panties in a twist.
Something changed recently that I just don’t care as much anymore. It’s a job, not a passion.
At least at my work there was pretty genuinely no workaround. The service desk group chat was mostly just sharing memes and making jokes about it cause wtf else you gonna do.
As a general rule though, if it’s something legitimately out of your control, don’t stress. The bosses don’t gaf about you, so you have to gaf about you.
I’m almost convinced moving to cloud services was a way for IT to deflect blame
Deferred liability is a HUGE benefit to paying someone else to handle things.
Which is probably why management loves AI so much - not only are they deferring liability to the AI, the AI tells them they did a great job even when things go wrong!
Answer the phone, blame Microsoft. Just like any other day.
Email doesn’t have an outage. Centralized email has.
I mean email is already decentralized
Spec-wise, yes. Practically, no.
Gotta find an alternative to Microsoft where at all possible.
Nah, let’s all use Microslop, that way we get random time off every now and then 🙃
It’s like a snow day of sorts 🌨️🌐🌨️💻🌨️
Other than a trusted outgoing IP, you can run a <10 user mailserver on a raspberry pi. The cost of setting up a trusted outgoing mailserver is also incredibly cheap (actually free if the volume is low enough).
The only things stopping businesses from doing their own mailserver is tech illiteracy, deferred liability and/or an unwillingness to maintain systems (my org has a 3rd party IT service that refuses to do anything in house, even after selling us a very decent in-house server that can definitely handle more than SSO and file storage).
The only things stopping businesses from doing their own mailserver is tech illiteracy, deferred liability and/or an unwillingness to maintain systems
That’s very true. My wife is having a rough Friday the 13th at work because of this outage. I think they could really benefit from doing more in house and being less reliant on 3rd parties, but their IT department is about as useless as a box of rocks.
I’m not exactly a fan of MS, but other mail providers also sometimes have outages.
Also btw Yahoo just added something to their terms about AI scanning all mails for CSAM, which sounds creepy af.
“other mail providers” should be decentralized smaller businesses / ISPs, not some megacorp. Then outages happen but they never affect millions at the same time.
“were protecting the kids” - where rights go to die
Boy, don’t gun owners know it.
As a gun owner, I’m going to disagree here. While there are always extremist pushing for the strictest regulations, for the most part what most want is basic logical control.
People with history of violence or mental disorders shouldn’t have weapons. Period. Does anyone really need an AR-15 for “hunting”? No.
The tired old argument of the 2nd amendment keeping the citizens armed for a militia to stand against a corrupt government is laughably absurd. The local police are practically armed like the army now. And no ones fully automatic AR-15 is going to do jack shit against an Abrams tank or a F-35 or a predator drone.
Gun control makes sense in the same way seatbelts do, will it save everyone? No. Can it still do a lot of good? Very much so.
People with mental illness and a history of violence aren’t allowed to have weapons in my country. What undeveloped nation do you live in?
Ha! You really have to ask? The only nation where mass and school shootings are a regular occurrence.
Apple, Google, all of them are doing it, ‘protect the kids’ right? Or just target ads and those who dissent against the government.
IMO doing stuff like this is the job of the police and should be defined by law and not vigilante email providers with their ToS or privacy policy. In a capitalist economy, you gotta ask yourself: “why are email providers doing it?” Doing so costs money and doesn’t benefit them; so obviously there’s something they don’t tell you, that makes them money and that’s just not okay.
And all we had to do, was not giving this communication to a central authority like Microslop. Will our CTOs ever learn?
Spoiler: I don’t think you should hold your breath waiting for them to learn from mistakes. Like ever…
Maybe we should assist them at holding their breath.
Now there’s a thought…
I had such a quiet afternoon at work. Can’t wait for them to go down again. It’s Friday, come on MS! You can fail again, I believe in you!
Honestly been great for me. Had such a quiet work day courtesy of Microsoft. They should kill their email servers more often haha
