pull down to refresh
Yep, we expected a backwards incompatible rolling deployment (db gets migrated, but client doesn't know yet), but this migration was also non-atomic (spread across two migrations) and the second one failed unexpectedly requiring manual intervention.
I woke up at 5a this morning and ek and I were in a video call anticipating something like this. By far, this is the most complicated migration we've done to date. We migrated 'polymorphic' relations (ie a hack with triggers for fast polymorphic joins) and encrypted columns, nearly doubling the number of tables we store wallet related stuff in.
5am...migrated polymorphic relations...backwards incompatible rolling deployment...failed unexpectedly
this is what I imagine my nightmares would be like if I was a developer.
This was a nightmare. When these migrations fail, you have no idea how long it might take to resolve. Even with tons of QA, we experienced a bunch of problems.
Modeling data for long term storage and millisecond access is hard. Getting the design wrong, like we did with wallets v1, is very expensive.
actually awesome omg
ah that's what that was
yes
Also, I noticed a brief 500 when loading SN earlier this morning. I assume it was during this deployment?