Did some calculations and you are correct indeed about the difference in returns being substantial.

Using the numbers from
https://www.investorsfriend.com/what-warren-buffett-accomplished-for-share-owners/ and https://www.nasdaq.com/articles/if-you-invested-$100-in-berkshire-hathaway-in-1965-this-is-how-much-you-would-have-today

share price in 1965: 18 dollars
share price in 2025: 460 000 dollars

That's roughly a 25 000x compared to gold's 100x over the time period.
Impressive indeed.

That's a 250x over 60 years adjusted to inflation as measured by gold,
or about 9.7 percent per year annual compound growth rate above inflation rate as measured by gold price changes.

Not taking into account inflation:
1960s to 2020s: 25 000x over 60 years, or 18.4 percent annual compound growth

Taking into account inflation as measured by gold:
1960s to 2020s: 250x over 60 years, or 9.7 percent annual compound growth rate

Pretty big difference