0 sats \ 1 reply \ @davidw OP 28 Feb \ parent \ on: The data to help you become a better Stacker ⚡ meta
I could certainly improve the workflow and automate a lot more if I wanted a live dashboard or to do this regularly, but without any API this is what was used to get the job done for now:
- visit stacker.news front page as I mentioned above under 'notes'
- tap 'more' button to see more than 21 posts at a time. repeat 390 times
- right-click on page and save .html file locally
- created a python script that recursively spits out and renders all the front-end post content (title, author, territories, sats, cowboy hat, comments etc) into a csv
- copy that CSV into a spreadsheet with new columns with formulas to sort and categorise content better
- create a bunch of pivot tables & graphs in said spreadsheet
- screenshot those graphs & upload to SN
If I was to make this a continued process, I would automate Steps 2 and 5 - making sure the data got pushed to Tableau or something as capable. For a one-off exercise like this however, it's more than sufficient.
And next time I want to click a button 390+ times to get an updated view, I can simply paste the data over the existing spreadsheet data and all the pivot tables automatically update.