It seems like, with the internet and tools like AI, the ability to come up with well-researched, fantastically weird and interesting plots should be at an all time high. Access to media is unprecedented: any writer can read or watch the greatest stories from all of humanity (any language, any time). There's almost no story you can't get your hands on.

And yet all we do is produce shallow plots and sequels.

I'm pretty sure it has to do with funding: a story that has already been successful is a safe bet. Doing a sequel is more likely to make a return than some un-tested new story.