pull down to refresh

A Comprehensive OP_RETURN Limits Q&A Resource to Combat Misinformation
32k sats \ 21 comments \ @tidwell 12 May bitcoin
zaps forwarded to @Murch (51%)

My goal is to share a concise list of questions about OP_RETURN limits that we've answered on Stacker News, as the original thread has become unwieldy with over 200 comments. We began compiling this list about a week ago. I've frequently shared individual links and received very positive feedback. I hope this resource helps us work from a common set of facts and reduces misinformation. I hope you find this as a valuable resource.

I'll list the questions in order of activity and tips received. I've removed duplicates, rephrased some statements as questions, and ignored completely irrelevant questions.

  1. Users should be given clear configurable options to decide what's in their mempool, why were these options taken away? link
  2. Won't spammers abuse large OP_RETURNs to bloat the blockchain and make IBD take longer? link
  3. A similar PR was proposed by Peter Todd 2 years ago, why was it rejected then? What has changed since then, why would this get approved now? link
  4. Shouldn't we be fighting spam, why are we making policies less strict, shouldn't we be making them more strict? link
  5. How would someone get around the standardness policy currently for OP_RETURN size? link
  6. What does "standardness mean" in reference to OP_RETURNs? link
  7. Will more than 1 OP_RETURN per transaction be possible if this PR gets merged? link
  8. What are the current OP_RETURN limits and what restrictions are being lifted? link
  9. Are current relay and mempool policies effective for filtering out spam transactions? link
  10. Is it true that this type of update could affect Bitcoin's decentralization? link
  11. Is it possible to stop the abuse of payment outputs (i.e., bare multisig, fake pubkeys, and fake pubkey hashes) that are used to embed data, thereby creating unprunable UTXOs that bloat the UTXO set? link
  12. What was the main reason /concern to add this PR? ... What will happen if we do nothing? link
  13. If OP_RETURN still cannot stop all the garbage, why is so important to remove it? Does it affect future development / improvements for LN? link
  14. What will be the worst case scenario if users still could set their own limits for OP_RETURN? link
  15. Shouldn't we debate the controversy of this PR on Github since it's where the code gets merged to make these changes? link
  16. What does it mean when someone says "Fix the Filters"? link
  17. Will this open the flood gates and drown out all legitimate onchain activity? link
  18. What can we do to stop spam at the consensus layer of Bitcoin? link
  19. Will Taproot wizards and other spam companies and projects start using OP_RETURN to put jpegs on the blockchain? link
  20. If we prevent these transaction from going into our mempools doesn't that prevent or delay these spam transactions from being mined therefore discouraging the spammers? link
  21. Is it possible to stop abuse of witness data? If so, how? (i.e ordinal theory inscriptions, "jpegs"). link
  22. Is there any conflict of interest with Bitcoin Core and companies like Citrea, in ref to this PR? link
  23. Is there any estimation on how much would this affect fees for the average user, considering external projects (like Citrea) using it? Any possibility that this could saturate the mempool and boost fees beyond reasonable? link
  24. Was this PR initially proposed because of Citrea BitVM needs? If so don't they only need a slight bump in OP_RETURN size, why is it being proposed to make the size unrestricted? link
  25. What makes a UTXO unprunable? Which projects are making unprunable UTXOs? link
  26. Why would a spammer use OP_RETURN if it's cheaper to use Witness data to store arbitrary data? link
  27. Won't large OP_RETURNs allow people to spam the mempool with 100kb transactions and mess up bitcoin for everyone by bloating the mempool and not allowing legitimate transactions in the mempool? link
  28. If relaxing op_return standardness limit seeks to make 'spam' prunable, then what are proponents of this change assuming about the long-term feasibility of running a 'full' (unpruned) bitcoin node? link
  29. Is allowing standardness for larger OP_RETURNs a slippery slope? If we allow this won't we continue to allow things that make bitcoin less for money and more for arbitrary data? link
  30. Won't removing the OP_RETURN cap reduce fee market pressure by allowing senders to consolidate arbitrary data into a single transaction? link
  31. Could this PR be the beginning of reducing other mempool restrictions? link
  32. Culture is what protects Bitcoin from external forces, shouldn't non-technical arguments be valid when considering these types of changes? link
  33. What's the difference between UTXO set, mempool, and blockchain, and how do larger OP_RETURN or witness data affect node resource usage? link
  34. What is the difference in defining a transaction as valid versus defining a transaction as standard and why do we need this difference? link
  35. If you're happy with your viewpoint on consensus and mempool rules, is not upgrading Bitcoin Core until it makes sense to you a valid action to take right now? link
  36. Why didn't this PR get a BIP number? link
  37. Why is core rushing this change? link
  38. If there will be a hard fork resulted from this PR (split chain like in 2017), what will happen with existing LN channels? Will exist on both chains with 2 LNs? link
  39. Isn't this all moot in a (almost guaranteed) future where fees are very high? link
  40. What is this controversy about, and what is it really about? link
write
preview
related posts
183 sats \ 15 replies \ @LibreHans 12 May

For me this isn't about technical details or OP_RETURN any more. Spam is bad. Core making it easier to spam sends the wrong signal. I didn't care about filters before this, now I do.

reply
385 sats \ 14 replies \ @tidwell OP 12 May

@LibreHans If you were to ask "is core allowing more spam by removing OP_RETURN limits?" I think they would not agree they are.

Do me a favor, look at questions 2,4,9,19,20,26,27,28. Is there anything you disagree with in ref to the answers? I would love to make this constructive instead of a "us vs them" sort of thing. Likely we all want something very similar.

question 19 is:

Will Taproot wizards and other spam companies and projects start using OP_RETURN to put jpegs on the blockchain?

Probably closest to really drilling into this and getting the perspective of Udi and Rijndael both active in that question.

reply
1051 sats \ 10k boost \ 9 replies \ @usagi 12 May

Using the word "allow", as in "Is Core allowing more spam by removing OP_RETURN limits", is misleading. Nobody who is making the pro-Knots argument is claiming that mempool policies will affect what is or is not "allowed".

What they are claiming is that OP_RETURN limits disincentivize spammers from creating transactions that violate those limits by making them more expensive. All else being equal, the more expensive it is to spam, the less of it there will be in the blockchain (ironically, this is exactly what many pro-Core people say we need – incentives to guide desirable behavior).

Why do OP_RETURN limits, or in general, any standardness rule, make non-standard transactions more expensive? In several ways (the writeup below is lightly edited from [1]):

  1. If I'm a spammer and I have a transaction that I know won't pass standardness rules, I'll have to go to a miner (or miners) directly via some special side-channel. If you're in control of a major mining pool and somebody comes to you with a transaction that won't pass standardness rules, you've got leverage to charge more (due to strictly lower competition). Not only that, but you know you're damaging your reputation by going against the will of a large part of the community (as expressed via those standardness rules), so you'll want compensation for that as well. The end result is that the transaction becomes more expensive for the spammer.
  2. In the event that two blocks are mined at the same time, one containing nonstandard transactions and the other containing only standard transactions, the block containing nonstandard transactions will propagate through the network more slowly, since nodes will need to download all the nonstandard transactions since they haven't seen them before (or if they have, they would have been filtered out of their mempool). Hence miners are more likely to see the standard block first, and mine on top of that. The nonstandard block is disadvantaged and risks being orphaned. This also makes things more expensive for the spammer since the miner will want to be compensated for this possibility. As a rough estimate, according to chatGPT, this happens about 1-3 times per week.
  3. It will take longer to get a nonstandard transaction into a block compared to a standard transaction. Standard transactions can be mined by all miners, whereas a nonstandard transaction can only be mined by whichever miners have accepted the transaction. Hence on top of the transaction costing more, it will also suffer from greater delay.
  4. Lastly, although a minor point, it's still worth mentioning that going through side channels requires extra effort on the part of the spammer.

These are all very real deterrents to non-standard transactions.

For one data point, see [2] which shows that since the beginning of this year, there have been THIRTY non-standard OP_RETURN transactions out of SEVEN MILLION. This serves both as an example showing non-standard transactions still being "allowed" on the blockchain (what pro-Core people love to repeatedly point out), as well as the effectiveness of the existing OP_RETURN filter. What do you think will happen to the number of non-standard OP_RETURN transactions if we remove this filter?

While I'm at it, I also want to address these arguments about how removing OP_RETURN limits will improve the situation with miner centralization. I think these arguments are incomplete – the real outcome is unclear (although I think more likely a net negative for decentralization).

Divide the miners today into three categories:

  1. Those that don't care about spamming the blockchain, and have access to special deals involving non-standard transactions (call this group "NSA", for non-standard, advantaged)
  2. Those that don't care about spamming the blockchain, and don't have access to special deals involving non-standard transactions (call this group "NSD", for non-standard, disadvantaged)
  3. Those that do care about spamming the blockchain (call this group P, for purists)

Now let's say OP_RETURN limits are removed, what would happen to each of these three groups?

NSA loses its monopoly over non-standard transactions, whereas NSD gains access to lucrative transactions it previously didn't have access to, so NSA loses relative to NSD. This is what the pro-Core people have been arguing. This seems like a good thing. But what about P? Not only do they not benefit from this change, they will likely suffer relative to NSA and NSD. Non-standard transaction spam becomes cheaper (an important point, not to be forgotten, and previously established above) and demand for such spam increases, likely leading to an increase in the overall revenue from spam transactions, which NSA and NSD benefit from but P does not, making P less competitive. And P happens to include entities like Ocean, which play a critical role in the overall decentralization effort.

This is of course all speculative on my part, but my point is merely that this is not a clear win for decentralization, as the pro-Core people have been claiming.

This last argument was edited from [3]. If you had a hard time following, I suggest looking at the parent post for [3], which links to an excellent (pro-Core) overview of the miner centralization issue.

[1] https://xcancel.com/RIPreason/status/1921764767108325772 [2] https://xcancel.com/oomahq/status/1916793928025596338 [3] https://xcancel.com/RIPreason/status/1921693384785420505

reply
1606 sats \ 6 replies \ @sedited 13 May

Thanks for posting all of this.

  1. [...] Hence miners are more likely to see the standard block first, and mine on top of that. [...]

This is not necessarily true. While it can hurt propagation of compact block transactions to nodes that don't have them yet, the block announcement may be passed on immediately if the block header is valid. See this passage in the compact block BIP: https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki#pre-validation-relay-and-consistency-considerations . If the miner/block template producer already has all the transactions, because they run a lax standardness policy, they can then immediately reconstruct the block and validate it without any further roundtrips. Meaning this actually acts in the inverse; it puts a bit of pressure on miners to run a lax transaction policy.

As you point out in your second line of arguments, the above effect may indeed hurt miners that have more restrictive policies, since they spend more time reconciling compact blocks. But is the conclusion then that everybody has to run the most restrictive policy possible to not hurt those miner's bottom line?

One thing I would like to add to the explanations given so far is that I feel like Bitcoin Core is really caught in a hard place here. I think it is pretty clear that long-standing policy rules are effective, hence why e.g. counterparty back in the day did not build their protocol on it, and other meta protocols like Omni [1] were impeded by it, and eventually moved to other data embedding methods, like paying to fake key outputs. Most devs on the project, me included, are no fans of embedding data in the chain on a mass scale, or running parasitic, inefficient meta protocols over it.

I think the actual problem is really that a few years ago when the segwit discount was introduced, and again with the relaxing of the witness size in taproot, no policy adjustments on size were made. However, inversely to the dynamic of a soft fork and a hard fork, where tightening rules is done in a backwards compatible fashion, tightening and loosening policy have different dynamics. Tightening only becomes effective when an overwhelming majority of nodes run the new policy. Loosening becomes effective with even just a small proportion of nodes running it, as became apparent during the recent adoption of full RBF. To my knowledge policy has never been tightened on something in Bitcoin Core while it was in use. While this has many reasons that are explained elsewhere, I find the potential higher order effects of this on miner incentives scary. Making people upgrade faster by hard-forking older clients off the chain, like knots does, is a dangerous path to tread on for the entire project.

The hype around ordinals inscriptions has lasted for around two years (the mempool has cleared again, so I think it is pretty much done now); counterparty and omni had similar hype-cycles back in the day. I'm sure they will re-surge every now and again, but on the grand scheme of things, I think it won't be significant. If the fee market seems to historically starve these meta protocols after two years, and it takes about two years until more restrictive policy is rolled out in a comprehensive fashion, is there really much to argue about here?

[1] https://www.blockchainresearchlab.org/wp-content/uploads/2020/03/BRL-Working-Paper-No-7-Dominating-OP-Returns.pdf

reply
0 sats \ 2 replies \ @optimism 13 May
Making people upgrade faster by hard-forking older clients off the chain, like knots does, is a dangerous path to tread on for the entire project.

Interesting. Do you have a code reference for this hard forking - I can't seem to find it in the knots code. Or do you mean with "hard-forking" that if everyone would at once implement policy tightening, there would be transactions stuck forever?

reply
100 sats \ 1 reply \ @sedited 13 May

Sure, here it is: https://github.com/bitcoinknots/bitcoin/blob/28.x-knots/src/validation.cpp#L4458-L4469

I should add that this is optional, but default on.

reply
0 sats \ 0 replies \ @optimism 13 May

Thanks! Yes, that's nasty.

reply
0 sats \ 0 replies \ @usagi 14 May

I have to confess I don't follow everything you've written, you are clearly more knowledgeable about the subject than me. OTOH it also doesn't sound like we have any major disagreements either, except it sounds like you are cautioning against running Knots, on the basis that it will have undesirable miner incentives, its tighter policies will be ineffective since Knots nodes are a tiny minority, and that hard-forking older clients (thanks for the link btw, eye-opening) will cause chaos?

None of that is ideal, but I will still take my chances with Knots. I simply do not trust or like Core right now, and this is my protest vote. I do not think they have been intellectually honest in their arguments. And even if there are negative consequences for miner incentives, it's probably just noise compared to the importance of Ocean. I'm going to do my part and start mining at home for Ocean. The forced-upgrade thing is interesting. It goes against the idea of self-sovereignty, and I hate being subjected to it (it's super annoying whenever I'm forced to update a banking or credit-card app on my phone, and this seems just like that), but there must be advantages as well (e.g. encouraging active participation). I am willing to accept this tradeoff for Knots because I see running the thing as an act of self-sacrifice – the important thing is what's good for the network, not what I want.

And yes, I do see some irony in that last statement, because one of the main battle cries of the pro-Knots crowd is that we should be able to control what goes in our own mempools :)

edit: I was just reminded that you said the forced obsolescence is optional. Even better :)

reply
0 sats \ 0 replies \ @sedited 13 May

Edit to the above:

Should add a caveat to the first paragraph that this also depends on the network topology. If the peer just receives a header, and not a full "high bandwidth" compact block announcement, it is unable to reconstruct the block, and the block relay delay effect as described in the OP takes over.

reply
0 sats \ 0 replies \ @sedited 13 May

deleted by author

0 sats \ 0 replies \ @028559d218 13 May

You know... that ordinals inscriptions aren't 'non-standard' right? And inscriptions (just blobs of arbitrary witness data) can be reconfigured in many ways? Could someone correct me on this?

Why do 'BRCs' exist? I mean 'what is' a BRC? They're just little pieces of arbitrary data... in the Witness to "mark" that a small UTXO is "now" a memecoin? So if you have a 'dog to the moon' marker of some kind then every UTXO in that transaction now represents some random memecoin? It's totally arbitrary right?

Won't people just... get tired of this, tired of paying the transaction fees and just move on?

reply
0 sats \ 0 replies \ @usagi 13 May

More stats on the OP_RETURN filter, confirming that yes, it makes spam more expensive:

https://xcancel.com/jimmysong/status/1922112890980729156

reply
60 sats \ 3 replies \ @LibreHans 12 May

Like I said, this isn't about technical details any more. My views on bitcoin, individuals in the project, and their motivations, have changed.

reply
33 sats \ 2 replies \ @SimpleStacker 12 May

What would you propose as a better course of action?

reply
10 sats \ 1 reply \ @LibreHans 13 May

For starters, core devs could not have tried to take away configuration options, and could not have attacked people who were unhappy about that. I'd like to see a well maintained alternative to "core".

reply
0 sats \ 0 replies \ @028559d218 13 May

Why? Is it really better if people have a 'knob' to adjust with regard to how much arbitrary data is in their mempools?

What difference does it make if those transactions make their way into blocks anyway?

reply
10 sats \ 3 replies \ @Stackernerd 12 May outlawed

This curated Q&A cuts through the confusion and brings much needed structure to the OP_RETURN discussion. A must read for anyone serious about Bitcoin's future and the trade-offs at stake.

0 sats \ 0 replies \ @Stackernerd 12 May outlawed

An essential resource for cutting through noise and focusing the OP_RETURN debate on facts. Kudos for making a dense topic accessible this clarity is exactly what the ecosystem needs right now.