Interesting read. The most important for me here was:

we're in a slightly weird state today where the models are good, but not great. They can work for minutes at a time, but not yet hours.

Thanks fren for confirming all the discussions we've had about reducing long contexts and breaking up to smaller and more specifically tuned instances.