i figured out a work around.

i know people hate spark, but i tried it out and it's pretty smooth , given al the leagal and infrastructure limitations