I got my hands on the entirety of the FOIA dump mentioned here, and am pushing it through my custom (and sovereign) OCR => vectorize => embed pipeline on a 16G Mac M1 right now. To see what needs updates in that pipeline, and if I can improve quality further. Takes forever but it's like an ETL pipeline that you can just run on cheap, near-EOL hardware.
Anyone has some inspiring research questions to try and find answers for from that dataset?
I got my hands on the entirety of the FOIA dump mentioned here, and am pushing it through my custom (and sovereign) OCR => vectorize => embed pipeline on a 16G Mac M1 right now. To see what needs updates in that pipeline, and if I can improve quality further. Takes forever but it's like an ETL pipeline that you can just run on cheap, near-EOL hardware.
Anyone has some inspiring research questions to try and find answers for from that dataset?