Hey guys!
Want to thank Kevin Rooke for inviting me here for an AMA.
There's loads of things happening in Bitcoin and of course the AI space. And with that, comes alot of noise and misconceptions. The team and I have been working hard building an open source, trained-from-scratch, "Bitcoiner" LLM - NOT a "wrapper on ChatGPT", but a real Bitcoin model.
With that being said, I'll be around for the next hour to answer any questions any of y'all might have. So hit me up and I'll do my best to answer (it will be me answering, not our model..lol)
Olá @Svetski ! Here is a slightly provocative question, but without any malice. I have a question about the potential bias of a large language model trained on a dataset of Bitcoin-related text.
reply
yes - it will definitely have a bias.
Language models are a mirror of the data they're trained on. ALL data has an inherent Bias.
A bias is simply a "model of the world", or an "opinion" or a "worldview". Everything that's ever been written, has a worldview (or bias).
Therefore, every LLM ever built will have....you guessed it...a Bias!
The key is not to try and remove bias (impossible) but to have MNAY models, each with their own biases.
As Bitcoiners, our bias, and model of the world is different to the mainstream, fiat model of the world.
As a result, as the model is trained on more and more Bitcoin data, it will represent, more and more, some "aggregation" of the bitcoin model of the world.
Does that make sense? Let me know if I can clarify further :)
reply
The key is not to try and remove bias (impossible) but to have MNAY models, each with their own biases. As Bitcoiners, our bias, and model of the world is different to the mainstream, fiat model of the world.
👍 💯
reply
What do you recommend as the ultimate samurai/ancient warrior book list to read (including order) leading up to Bushido?
reply
ooh.. Good question.. haha
  1. Shogun: James Clavell
  2. Musashi, by Eiji Yoshikawa
  3. The Book of Five Rings: Miyamoto Musashi
  4. The Virtues of War: Steven Pressfield
reply
The Unfettered Mind.
reply
Seems like AI using bitcoin for automated payments is going to be a big deal, do you have any insight there? What would that world look like - why will AI be paying each other for stuff and why use bitcoin instead of fiat??? (I have my own answers to this but would love an insider perspective!)
reply
Mmm.
I actually have a contrarian viewpoint on this. I think it will take a while for this to really happen at scale, because all of the current accounting system (especially with the big companies) is denominated in USD.
Doing machine to machine payments in sats will create an accounting nightmare for them - especially in the near term.
But...that doesn't mean we're not going there. We are for sure.
I just think it's going to take alot longer than what some of us Bitcoiners think..
reply
In terms of what the world looks like.. Probably largely similar.
As I said to someone earlier - AI is a tool, like a computer. It exists to automate away some tasks and if use properly, create leverage for the user.
If you can further automate some of that leverage (by having the machine execute some economic tasks) you can increase the leverage, and ideally do more with less.
It's the been the same store with automation, tools and technology since the beginning of time
reply
reply
reply
deleted by author
reply
I am reading first... but not really interested. I am more interested in creating content for https://planb.network/ (via https://github.com/DecouvreBitcoin/sovereign-university-data)
reply
Spoke with Rogsy about this. I think we'll definitely work together!
reply
I am listed in the professors page https://planb.network/professors I just want to update the tutorials and add more ones. I have a lot of content to add. That project is huge!
reply
Awesome - will check it out and connect :)
reply
is there a SkywalkerCoin anywhere to give a brother a hand?
reply
deleted by author
reply
Totally agree with him. See my comments at the bottom of this thread. Outsourcing has a price, which is a lack of muscle.
That's why less of us can do the things our forefathers did. LLMs will likely lead people (who become dependant on them) toward diminishing faculties of thought.
I 100% agree with @DarthCoin
reply
can you elaborate on the distinction between GPT wrappers and a trained-from-scratch LLM?
reply
ok..
So a GPT wrapper is simply taking a bunch of data (say, your docs, a few books, whatever), chunking them up and then storing them in what's called a "Vector Database". Once you've done that, you have ChatGPT (or other LLM) reference the most relevant "chunk" from your vector database when someone asks a question.
This is kind of like inserting an example when you're using chatGPT. What you're essentially doing is inserting relevant context so when the "model" responds, it's more relevant.
It's a great, cheap, quick, "hacky" way to get a model to give more bitcoin-like responses, but it's not really a "Bitcoin model", nor has the model actually been "trained".
Where it falls over is when you ask anything that's not specifically in the vector store. Then you're basically back to normal chatGPT.
On the other hands, a model that's been trained (like Spirit of Satoshi), has actually had a TON of data transformed and formatted for training, then a bunch of GPU cycles spent on actually CHANGING the weights and biases of either a pre-existing open source model, or (the harder alternative) using all that data to do a ground-up "pre-training".
These are both very different processes to a "ChatGPT wrapper". The latter actually involve training a model, whereas the former uses another model with some reference material.
Training from scratch ultimately delivers the best possible result, but it takes WAY longer and is orders magnitude more expensive.
Hope that makes sense man!
reply
got it, makes sense.
how do you think about building a moat around Spirit of Satoshi? how easy/hard do you think it will be for someone to replicate/compete with your model?
reply
The moat is in the quality of the model.
Honestly - it's been over 6 months of tinkering and fucking around with data structures, data styles, data mixes, training formats.
This whole process is more art than science. And unless you know Bitcoin very intimately, you're not going to compete.
And if you do, you need to really have a good grasp of training models. Which honestly - not many people do. We have 4 Data scientists who happen to be Bitcoiners too, working with us - and we've all been pulling our hair out.
So I'd say perhaps the outward facing "moat" will be the quality of the model, and the inward moat is the experience we've had (trial, error, experimentation).
That's hard to replicate.
reply
Many people seem to think that ChatGPT will be the google of AI, that there will be an AI model monopoly. Implicitly these people think there isn't much need for narrow intelligence if you have sufficiently powerful general intelligence. Spirit of Satoshi seems like a bet against this.
Can you share how you arrived at the conclusion that there won't be an AI model monopoly? What does the future of AI look like to you?
reply
Just basic physics. The more general you make something, the less specialised it can be.
Google is a good example, but so are reddit and forums. Google might give you a good high level overview on something, but when you want to go deep into a topic or subject, you go to reddit or a forum.
Same thing will happen with models. You'll have mainstream general ones that are "good enough" with general stuff.
Then when you want to go deep on a topic (especially those outside of mainstream, ie; Bitcoin, self defence, home schooling, alternative health, etc) you will want to seek out a domain-specific language model.
We will look to make the method we've used to train Satoshi and the training app (train.spiritofsatoshi.ai) available for other domains.
reply
"The more general you make something, the less specialised it can be."
... unless it has no memory or processing constraints... right?
Why couldn't a highly specialized self-defense LLM be given a new dataset for home-schooling so that it becomes highly specialized in both? And after that, why wouldn't it be the same with infinite new specialized datasets on the same LLM?
I would think that being highly specialized in (N+1) datasets would make it even more specialized in each specialization overall, because it has multiple domains with which to draw conclusions.... what humans call "wisdom".
This is different for humans because it takes us so long to become specialized in one area, and we have short lifespans. Also, our limited memory and processing power. We seem to forget things that aren't used.
But LLM's don't have any of these limitations. So why can't they be highly specialized in seemingly everything? (srsly trying to learn, not trying to be condescending. I'm actually very curious about this)
reply
many bitcoiners are skeptical about AI, some have already closed their minds off to the possible upsides from it.
what do you see as the biggest benefits AI can provide for Bitcoiners?
reply
Honestly - it's like having a computer. Like any other technology, "AI" is just a tool.
In fact, it should probably called "IA" (Intelligence Amplification), because when used intelligently, that's what it does (once again, like any other tool, or technology).
There's no real magic here. There's no "sentient being" that will come out of the circuits. All that "AGI" talk is just fairy tales from wierdos who wish they could create their own god. It's silly.
At the end of the day, it's a tool. Use it where necessary and use it wisely, and you can do more work, in less time.
You can also use it stupidly (like how some people might use a hammer - hitting themselves in the head - or social media - doom scrolling all day) and you will make yourself more dumb (ie; never think for yourself, and just ask chatGPT everything)
reply
So to clarify, in terms of benefit: "Automation and Leverage". Same story, as with al other tools and tech over the millennia of mankind's existence
reply
deleted by author
Interested to know how it will conclude highly debatable topics amongst bitcoiners like covenants or drive chains. Will you personally sway it in your desired direction?
reply
haha - yeah this is a tough one. How the model answers it will be a little emergent from what it's been feed. We will see.
reply
deleted by author
is there any way we could use hashpower to train LLMs? Like instead of mine we use it for AI stuffs?
reply
reply
reply
Very good bitcoin discussion
deleted by author
reply
🤣🤣🤣
One day...when there's enough data about my dumb ass, and I have nothing better to do with my life, I'll build a Ghost of Svetski.
Or maybe my kids will. Until then, I'll try do more useful things..
reply
deleted by author
reply
yeah - that's a RAG model, kind of like what I described to @kr below. It's just a general base model that references his data, maybe with a slight LoRa fine-tune.
I reached out to do something similar with my data, so I can see what they're doing in greater detail.
reply