WorkingLemmy@lemmy.world to Open Source@lemmy.ml · 13 days agoFOSS infrastructure is under attack by AI companiesthelibre.newsexternal-linkmessage-square28fedilinkarrow-up1354arrow-down11cross-posted to: linux@programming.devopensource@programming.devtechnology@lemmy.worldtechnology@lemmy.world
arrow-up1353arrow-down1external-linkFOSS infrastructure is under attack by AI companiesthelibre.newsWorkingLemmy@lemmy.world to Open Source@lemmy.ml · 13 days agomessage-square28fedilinkcross-posted to: linux@programming.devopensource@programming.devtechnology@lemmy.worldtechnology@lemmy.world
minus-squarebeeng@discuss.tchncs.delinkfedilinkarrow-up14·13 days agoYou’d think these centralised LLM search providers would be caching a lot of this stuff, eg perplexity or claude.
minus-squaredroplet6585@lemmy.mllinkfedilinkEnglisharrow-up39arrow-down1·13 days agoThere’s two prongs to this Caching is an optimization strategy used by legitimate software engineers. AI dorks are anything but. Crippling information sources outside of service means information is more easily “found” inside the service. So if it was ever a bug, it’s now a feature.
minus-squarejacksilver@lemmy.worldlinkfedilinkarrow-up16·13 days agoThird prong, looking constantly for new information. Yeah, most of these sites may be basically static, but it’s probably cheaper and easier to just constantly recrawl things.
You’d think these centralised LLM search providers would be caching a lot of this stuff, eg perplexity or claude.
There’s two prongs to this
Caching is an optimization strategy used by legitimate software engineers. AI dorks are anything but.
Crippling information sources outside of service means information is more easily “found” inside the service.
So if it was ever a bug, it’s now a feature.
Third prong, looking constantly for new information. Yeah, most of these sites may be basically static, but it’s probably cheaper and easier to just constantly recrawl things.