pull down to refresh

1635 sats \ 18 replies \ @optimism 22 Oct \ on: Stacker Saloon
Can report that I'm able to reproduce @SimpleStacker's reported issues with IBD on a Raspberry Pi 4 w/ Core 29.1. Eg block 796776:
2025-10-22T13:41:33Z [bench]   - Load block from disk: 240.23ms
2025-10-22T13:41:33Z [bench]     - Sanity checks: 11.34ms [2958.27s (3.71ms/blk)]
2025-10-22T13:41:33Z [bench]     - Fork checks: 0.31ms [135.27s (0.17ms/blk)]
2025-10-22T13:42:08Z [bench]       - Connect 704 transactions: 34679.36ms (49.260ms/tx, 3.470ms/txin) [51837.88s (65.06ms/blk)]
2025-10-22T13:42:08Z [bench]     - Verify 9995 txins: 34716.35ms (3.473ms/txin) [51906.08s (65.15ms/blk)]
2025-10-22T13:42:08Z [bench]     - Write undo data: 14.97ms [3908.57s (4.91ms/blk)]
2025-10-22T13:42:08Z [bench]     - Index writing: 0.09ms [39.59s (0.05ms/blk)]
2025-10-22T13:42:08Z [bench]   - Connect total: 34744.00ms [59248.99s (74.36ms/blk)]
Going to let it run (may take up to 20 days to finish lol). Almost 35 seconds to verify 10k txins is a lot.
The process is (logically) CPU-bound on b-msghandler because that's where ConnectBlock is ultimately called from, and high soft interrupt; which I expect to be from using wifi instead of wired (but I figured that since raspibolt is explicitly configuring for usage of that, that most operators would run it like that):
write
preview
100 sats \ 17 replies \ @SimpleStacker 22 Oct
Dude, thank you so much! Mind if I share it here? https://github.com/raspibolt/raspibolt/issues/1482
Can you help me understand further? Is CPU the bottleneck, not hard disk speed or bandwidth? Would connecting from an ethernet cable have helped? And lastly, is it due to UTXO set bloat, or simply due to larger blocks?
Sorry for all the questions, haha
reply
30 sats \ 16 replies \ @optimism 22 Oct
Feel free to share it with anyone, though note that it's right now only a confirmation that it can be reproduced. Next steps for me:
  1. Let it finish so that I can graph and analyze the full IBD end-to-end for patterns.
  2. Start comparing BenchConnectBlock across different ARM hardware. 1
  3. If there are visible differences in bench, profile it. Otherwise make a new bench until it shows the differences, and profile that.
  4. Attempt to optimize the code
  5. Do a PR if 4 has outcome
I aim to do 2 and 3 in parallel to the IBD whenever I find time.
Is CPU the bottleneck, not hard disk speed or bandwidth?
It is for my test, so I'd say "yes". It's also what I expected.
Would connecting from an ethernet cable have helped?
I don't expect that, because txin verification doesn't depend on network - it's all done between disk, cache and cpu.
And lastly, is it due to UTXO set bloat, or simply due to larger blocks?
The block I gave as an example above contains mostly consolidation of p2sh utxo. So in this case, we're actually seeing bad performance on utxoset unbloat.

Footnotes

  1. because for example the IBD I did on an RK3588 earlier this year went much faster. There are some cpu feature differences between that and a Pi4. This would also suggest that a Pi5 would perform much better because it has some more CPU features built in. ↩
reply
100 sats \ 1 reply \ @SimpleStacker 22 Oct
Ok, I'll wait for more concrete results before sharing anything. That Github issue is closed anyway so unlikely to get any real attention.
The block I gave as an example above contains mostly consolidation of p2sh utxo. So in this case, we're actually seeing bad performance on utxoset unbloat.
Interesting. But we wouldn't need to do this much validation if the UTXO set wasn't so bloated to begin with, right? So I think it's fair to chalk it up to the sphere of problems caused by UTXO bloat?
reply
30 sats \ 0 replies \ @optimism 22 Oct
I'd argue that bearer instruments (metals, coin/cash, bitcoin) have consolidation as a part of the natural cycle, especially for businesses. So I'd argue that it's fair to chalk it up to having a utxo system, but not to bloat.
reply
0 sats \ 13 replies \ @88b0c423eb 31 Oct
I synced on an odroid m1 with debian 13 and it was much faster, at expected speed for that hardware...not a huge difference from a rpi4, it's a quad core 2.0ghz arm board.
reply
0 sats \ 12 replies \ @optimism 31 Oct
Yeah my rk3588 is an Odroid M2 - the difference seems to be the crypto extensions.
reply
100 sats \ 11 replies \ @88b0c423eb 31 Oct
very sad the m2 has lower nvme speed than the m1...I really like those arm boards with native nvme support, do you use orangepi also?
reply
10 sats \ 10 replies \ @optimism 31 Oct
I don't own one of these right now, because I'm roaming. I have 4 boards on me and customs fucks with me every time, no matter what country I enter. They all end up thinking I'm just some virgin nerd tho so it's fine. lol
reply
100 sats \ 9 replies \ @88b0c423eb 31 Oct
lol, I was thinking about buying one of those: http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/Orange-Pi-5-plus-32GB.html But the debian support seems to lack
reply
0 sats \ 8 replies \ @optimism 31 Oct
dayummm 1
The M2 seems to be positioned a bit odd on the board versus odroid ones, other than that, looks fine. For chipset, RK3588 is very well supported in ARM firmware 2
You may need to bake your own image. Basically take an ubuntu ARM, shift it 512MB to the right, then inject uboot and the firmware and fuck a bit with fdisk, to describe it correctly.
I can build an image building script for the world but I cannot test it rn. Would have to get rid of one of the boards to replace it with this, let me be pensive for a week on that.

Footnotes

  1. I literally had to use archive dot org for that link! What's happening to net neutrality? Or is it net neuterality now, haha? ↩
  2. In fact I was a bit grumpy that they focused on this versus RK3328 - which is perfect if you don't need fast storage for things that you can just run from 4GB memory. ↩
view all 8 replies