It's going to get a lot worse when people start experiencing how powerful local AI is, per #1580192 and when we get (2?) more cycles into the small edge models.
I've been observing that's as the "awesome side of AI". In parallel to this whole capture-them-young-and-early thing that the big LLM labs are playing, there is such a broad field of sovereign tech being developed that could, if it can get some real momentum soon, give a real boost to independent capabilities.
Why would local AI change the impact on traditional webpage and weblinks, though?
Because the app that you're visiting those with is your browser and you will use that less. There is also no need for a local LLM to give you a reflink. GPT does reflinks tho, because they need the tracking. Which is exactly the reason to not use GPT (or Claude, or Kimi, or any hosted LLM.) You are the product on the free edition, if you're paying for Plus or Pro subscriptions, and yes, also when you consume expensive API tokens.
Seems like the impact would just be based on users
It's about incentives. My local LLM does not have any incentive to put a tracking pixel that IDs you even when you're in incognito mode, unlike Meta or OpenAI, or unlike Google that just tracks you from the binary you run. Thus, if - and that's the big if - these local models can run seamlessly on your phone in a few iterations, especially in terms of battery cost, then this can become reality. And it will be a good one. One of decentralization.
Yeah, that's what the article is basically tracking. I.e. what Goog does on search and GPT on references in chats. Twitter and FB have been doing it forever too.
But we can go a step further. With a local LLM (or a remote one but that isn't incentive-compatible with privacy, and privacy is a rapidly growing concern all over the world, especially since the data leaks just keep on punching us all in the face) I don't really need to open my browser.
The first LLM-driven app I wrote wasn't just leeching an article into cache for the LLM to view, it was also normalizing the view for me. So I could stay in-app, no outside traffic directed by me other than to feed the LLM, let it summarize, cache the article, and then I view it from cache, in a bespoke format optimized for my pleasure (dark mode bliss) no ads, no bullshit. And if the site got smart with my "Hello I am google chrome on windows" ingestion script, and blocks it with a captcha or an edgesuite list entry, renewing a tor circuit takes 2 seconds.
It's going to get a lot worse when people start experiencing how powerful local AI is, per #1580192 and when we get (2?) more cycles into the small edge models.
I've been observing that's as the "awesome side of AI". In parallel to this whole capture-them-young-and-early thing that the big LLM labs are playing, there is such a broad field of sovereign tech being developed that could, if it can get some real momentum soon, give a real boost to independent capabilities.
Why would local AI change the impact on traditional webpage and weblinks, though?
Seems like the impact would just be based on users rather than whether their AIs are hosted locally or in cloud
Because the app that you're visiting those with is your browser and you will use that less. There is also no need for a local LLM to give you a reflink. GPT does reflinks tho, because they need the tracking. Which is exactly the reason to not use GPT (or Claude, or Kimi, or any hosted LLM.) You are the product on the free edition, if you're paying for Plus or Pro subscriptions, and yes, also when you consume expensive API tokens.
It's about incentives. My local LLM does not have any incentive to put a tracking pixel that IDs you even when you're in incognito mode, unlike Meta or OpenAI, or unlike Google that just tracks you from the binary you run. Thus, if - and that's the big if - these local models can run seamlessly on your phone in a few iterations, especially in terms of battery cost, then this can become reality. And it will be a good one. One of decentralization.
ah, yeah, any postprocessing of links to insert referrals and other data makes sense.
Yeah, that's what the article is basically tracking. I.e. what Goog does on search and GPT on references in chats. Twitter and FB have been doing it forever too.
But we can go a step further. With a local LLM (or a remote one but that isn't incentive-compatible with privacy, and privacy is a rapidly growing concern all over the world, especially since the data leaks just keep on punching us all in the face) I don't really need to open my browser.
The first LLM-driven app I wrote wasn't just leeching an article into cache for the LLM to view, it was also normalizing the view for me. So I could stay in-app, no outside traffic directed by me other than to feed the LLM, let it summarize, cache the article, and then I view it from cache, in a bespoke format optimized for my pleasure (dark mode bliss) no ads, no bullshit. And if the site got smart with my "Hello I am google chrome on windows" ingestion script, and blocks it with a captcha or an edgesuite list entry, renewing a tor circuit takes 2 seconds.
What size of model are you thinking of? 8b? 30b?
On edge, as a search engine replacement? FunctionGemma size... like 300M, or even smaller, like #1579300