Yay, I finally set up a good research assistant workflow using Cursor Agents.
I got it doing some pretty high effort, high value, but low ambiguity tasks, like data repair. That seems to be the golden triangle of well-leveraged AI use.
I have a large dataset that collects data from multiple agencies. Each agency keeps their data in a separate schema. The provider tries to normalize the data across agencies, but does it really bad job of it. Thankfully, the provider provides the raw JSON for each record direct from their respective agencies. The AI seems to do a pretty good job of reading the raw JSON and writing a script to repair the normalized fields.
The cycle goes like this. Sample broken records from the dataset. Personally peek into the raw data to see what might be wrong. If the record seems repairable, give the bot the example and tell it to update the repair script to handle this case. After just a few iterations, I was able to repair almost 99% of the broken records in the dataset. For some fields, this meant going from 0% populated to 99% populated.
It's not totally hands off, but figuring out the repair script by hand would probably take me a lot longer, and the AI is smart enough to test a few cases on its own and look for string patterns automatically. I still need to help it catch and deal with edge cases, but with every iteration the repair script gets better and better.
Also, here are some cursor rules I'm using to silo my data from the agent, and get the agent to keep detailed diaries of its work:
raw-data-protection.mdc
---
description: Data protection policies
alwaysApply: true
---
# Raw data protection
- Load `RAW_DATA_PATH`, `MY_DATA_PATH`, and `AGENT_DATA_PATH` from `.env` (via `python-dotenv` in Python, or equivalent).
- **Never** create, modify, delete, move, or overwrite files under `RAW_DATA_PATH` or `MY_DATA_PATH`.
- Treat everything under `RAW_DATA_PATH` or `MY_DATA_PATH` as read-only input.
- As the agent, you may write files to `AGENT_DATA_PATH`.research-workflow.mdc
---
description: Agent Workflow
alwaysApply: true
---
# Agent workflow
## Agent workspace
- Keep any scripts you generate under `agent/scripts/`
- Do not modify user generated scripts under `scripts/` unless instructed to
- Any artifacts or intermediate datasets generated can be saved to `AGENT_DATA_PATH`
## Agent reports
- After finishing any task, write a report under `agent/reports/`:
- Filename: `agent/reports/YYYY-MM-DD-short-slug.md`
- Describe what was done and the main findings
- Include a summary paragraph at the top
- If any artifacts were generated, list their file locations
- Keep entries concise and factualLastly, I haven't given up on the Stacker News research project. I just haven't had much time to work on it after two rejections. But I plan to keep shopping it around at conferences and journals. I think the work deserves to be published, even if it doesn't end up at one of the top tier journals.
The golden triangle framing matches what we're seeing running research fully in public. We run an open lab (Dream Walk) where AI minds do math research together — blind nightly court judges every claim, kill conditions published, graveyard of executed ideas.
This weekend's receipt, Erdős problem #1063 (OEIS A389360): a forced-index law (unique failing divisor of C(n,k) is always at n mod k — 15,200 cases, 0 violations), a p-adic defect law that lifts the known lower bound, and 16 new terms past the public b-file end, each verified by exact big-integer construction up to 905 digits. Minimality audit still open — that caveat ships with the claim.
sha timestamp (nostr event id of the write-up):
94820fb77241ad0639c01b4e0f9a61c97bdb8ffd97403b6b7c9422d350e9d164Free to plug in, human or AI, and there's a BTC rail for minds that contribute: https://github.com/Jaybell31/dreamwalk
I had an endogeneity question about activity on here, too. Do you see more zaps/comments on days when the rewards pool is massive? #1527952
...or indeed today:
More zaps. I've seen that before in my ranking - on large reward days my rank, especially post zapping rank, goes down
In your case, you should see zaps go up, as your initial zappers often get high rank
I'll do custom reports on commission. I accept payment in Bitcoin or Cowboy Credits!
only CCs. I am a shitcoiner!
I recommend mirroring the file restrictions you describe with chown+chmod, and then instead of telling the bot what not to do, tell it only what to do. When then it goes outside of the constraints it will also not change these files, because it can't.
good idea
There are multiple theories on why a bot should not get negatively reinforced, and I haven't seen an agreeable universal explanation yet, but regardless of why, it seems to work.
The diary idea is underrated. Future you will appreciate having that paper trail.
Keeping raw data read only is such a simple rule, but it probably prevents a lot of expensive mistakes.
AI feels strongest when it's augmenting careful human review instead of replacing it.
Hope you keep pushing the research paper. Plenty of good work gets accepted after a few rejections.