Terrasque

Or a few more gb of LLM?

https://y.yarn.co/2a4fe37e-ed9d-448b-af12-48a7a3202fa5_text.gif

Careful, if you spend 8 hours playing with your deck you might go blind

Hah. Snake oil vendors will still sell snake oil, CEO will still be dazzled by fancy dinners and fast talking salesmen, and IT will still be tasked with keeping the crap running.

This has a lot of “I can use the bus perfectly fine for my needs, so we should outlaw cars” energy to it.

There are several systems, like firewalls , switches, routers, proprietary systems and so on that only has a manual process for updating, that can’t be easily automated.

That’s because they don’t see the letters, but tokens instead. A token can be one letter, but is usually bigger. So what the llm sees might be something like

• st
• raw
• be
• r
• r
• y

When seeing it like that it’s more obvious why the llm’s are struggling with it

In many cases the key exchange (kex) for symmetric ciphers are done using slower asymmetric ciphers. Many of which are vulnerable to quantum algos to various degrees.

So even when attacking AES you’d ideally do it indirectly by targeting the kex.

I generally agree with your comment, but not on this part:

parroting the responses to questions that already existed in their input.

They’re quite capable of following instructions over data where neither the instruction nor the data was anywhere in the training data.

They’re completely incapable of critical thought or even basic reasoning.

Critical thought, generally no. Basic reasoning, that they’re somewhat capable of. And chain of thought amplifies what little is there.

No, all sizes of llama 3.1 should be able to handle the same size context. The difference would be in the “smarts” of the model. Bigger models are better at reading between the lines and higher level understanding and reasoning.

Wow, that’s an old model. Great that it works for you, but have you tried some more modern ones? They’re generally considered a lot more capable at the same size

Increase context length, probably enable flash attention in ollama too. Llama3.1 support up to 128k context length, for example. That’s in tokens and a token is on average a bit under 4 letters.

Note that higher context length requires more ram and it’s slower, so you ideally want to find a sweet spot for your use and hardware. Flash attention makes this more efficient

Oh, and the model needs to have been trained at larger contexts, otherwise it tends to handle it poorly. So you should check what max length the model you want to use was trained to handle

Temperature 0 is never used

It is in some cases, where you want a deterministic / “best” response. Seen it used in benchmarks, or when doing some “Is this comment X?” where X is positive, negative, spam, and so on. You don’t want the model to get creative there, but rather answer consistently and always the most likely path.

https://learnprompting.org/docs/intermediate/chain_of_thought

It’s suspected to be one of the reasons why Claude and OpenAI’s new o1 model is so good at reasoning compared to other llm’s.

It can sometimes notice hallucinations and adjust itself, but there’s also been examples where the CoT reasoning itself introduce hallucinations and makes it throw away correct answers. So it’s not perfect. Overall a big improvement though.

Microsoft’s Dolphin and phi models have used this successfully, and there’s some evidence that all newer models use big LLM’s to produce synthetic data (Like when asked, answering it’s ChatGPT or Claude, hinting that at least some of the dataset comes from those models).

randomly sampled.

Semi-randomly. There’s a lot of sampling strategies. For example temperature, top-K, top-p, min-p, mirostat, repetition penalty, greedy…

It’s an inherent negative property of the way they work. It’s a problem, but not a bug any more than the result of a car hitting a tree at high speed is a bug.

Calling it a bug indicates that it’s something unexpected that can be fixed, and as far as we know it can’t be fixed, and is expected behavior. Same as the car analogy.

The only thing we can do is raise awareness and mitigate.

Well, It’s not lying because the AI doesn’t know right or wrong. It doesn’t know that it’s wrong. It doesn’t have the concept of right or wrong or true or false.

For the llm’s the hallucinations are just a result of combining statistics and producing the next word, as you say. From the llm’s “pov” it’s as real as everything else it knows.

So what else can it be called? The closest concept we have is when the mind hallucinates.

This is a very simple one, but someone lower down apparently had issue with a script like this:

https://i.imgur.com/wD9XXYt.png

I tested the code, it works. If I was gonna change anything, probably move matplotlib import to after else so it’s only imported when needed to display the image.

I have a lot more complex generations in my history, but all of them have personal or business details, and have much more back and forth. But try it yourself, claude have a free tier. Just try to be clear in the prompt what you want. It might surprise you.

What llm did you use, and how long ago was it? Claude sonnet usually writes pretty good python for smaller scripts (a few hundred lines)

The only issue I see with targeting Linux is the sheer variety of Desktop setups. Finding one keyboard shortcut and payload that will work on even just the majority of distros would be a challenge.