a very large prompt would make each pass require more compute.

Yes. Feature not bug. More tokens = more moneys. So you better clean it up and reset that context window sometimes.

The alternative is searchable instructions, see also #1250028 from yesterday, which shows a clear way forward.

All you need is a well-tuned LLM that implements a process, not a persona.