1 sat \ 3 replies \ @hugo 21 Feb 2022 \ on: Daily discussion thread
Out of interest, what caused the outage today?
AWS attempted to update my database's minor version but it failed.
This should've been fine (it just should've rolled back and kept trucking), but I have a custom nginx config and whenever there's a configuration change without code being deployed, aws fails to apply the nginx config and nginx doesn't start.
All I had to do was ssh into the machines and start nginx.
What I'll be working on today:
- figuring out why the database couldn't update
- how to avoid this nginx issue (it was just an inconvenience before ... never took the site down)
reply
Turns out when I upgraded my database a few weeks ago, the aws service I was using didn't update it's config - they're out of sync and no upgrades can be performed.
I'm going to try to decouple the database from the environment and try to not take down the site again.
reply
Oh boy this is a mess ... I'm going to have to build a totally fresh environment ... All the hacks I tried have failed.
reply