"I have never let my schooling interfere with my education"
-- Mark Twain

As someone who fooled themself into years of upper education in pursuit of nothing but my own interests, I can say that I found far more education outside that system since abandoning it

Why would I spend a year of my time and LOTS of money to learn about Descartes first, when all I wanted to study were the Classical Greeks?

Modern schooling erects many roadblocks to learning, masquerading as a form of meritocracy, but I would argue it has actually resulted in far less meaningful education overall

Don't take my word for it, either; look at literacy alone - the founding documents of America were considered to be at the reading level of children of the time, as was the pamphlet 'Common Sense'

Another example: what was Alice doing before she went on her Adventures in Wonderland? The young girl was studying Latin, and that was a perfectly normal scene at the time of writing

I ended up doing almost exactly as you are considering, taking only the work offering skills of interest while putting my education into my own hands during "off-time"

Nobody is more qualified than you are to determine your direction, but I will add this:

I regret nothing.