Yeah, that’s the technical lingo meaning. Attach a debugger means that you have a running process without a debugger running with it, and then you attach the debugger to it to get debug output from that process.
But I still don’t quite get the intended process.
So you run the code the LLM outputs with a debugger and let the LLM interact with the debugger? Not really sure if that helps, because for the LLM would need to know how to operate the debugger and would need to understand what problems it should be looking for.
Current systems (e.g. Github Copilot) combine the LLM with static code analysis and compiler outputs to find and fix errors. They can also execute the code and run tests and compare outputs with expected outputs.
Current systems (e.g. Github Copilot) combine the LLM with static code analysis and compiler outputs to find and fix errors. They can also execute the code and run tests and compare outputs with expected outputs.
Am I misunderstanding something or does this sound like programmers making themselves unnecessary?
It’s not a “themselves”. It’s large corporations trying to make programmers unnecessary.
It’s like blaming Tesla test drivers for making taxi drivers unnecessary.
But it’s very far from the point where programmers would be unnecessary.
LLMs lack any understanding of the bigger picture. They work for tiny throwaway tasks that you won’t use in the future. The stuff they make is hardly runnable, let alone maintainable. It usually needs multiple rounds of asking it to do the right thing before you get anything remotely worth your time.
The feedback loop with static code analysis and so on generally doesn’t work. Once an LLM gets lost enough to produce compiler errors (which happens quite fast) it gets completely lost, since it then only focusses on fixing the compiler errors while totally forgetting what the actual task was. So it will change the program so that it compiles, but then it will not do what it was supposed to do.
It’s a hot mess.
Studies say that using LLM while programming reduces measured performance of the programmer by 20-30% as soon as the task isn’t something entirely trivial.
Edit: When an LLM gets into that compiler error loop, it’s like taking a non-compiling program written by a first-year student and handing it and the compiler output to the rest of the class and everyone sequentially gets a shot at fixing it while completely ignoring anything the others have done befora apart from taking their code as input.
Only the very junior ones who are writing code without understanding why they’ve been asked to write it. Anyone with more than about 18 months experience will be able to start deciding what to actually build, and I haven’t seen LLMs be particularly helpful with that yet.
Yeah, that’s the technical lingo meaning. Attach a debugger means that you have a running process without a debugger running with it, and then you attach the debugger to it to get debug output from that process.
But I still don’t quite get the intended process.
So you run the code the LLM outputs with a debugger and let the LLM interact with the debugger? Not really sure if that helps, because for the LLM would need to know how to operate the debugger and would need to understand what problems it should be looking for.
Current systems (e.g. Github Copilot) combine the LLM with static code analysis and compiler outputs to find and fix errors. They can also execute the code and run tests and compare outputs with expected outputs.
Am I misunderstanding something or does this sound like programmers making themselves unnecessary?
It’s not a “themselves”. It’s large corporations trying to make programmers unnecessary.
It’s like blaming Tesla test drivers for making taxi drivers unnecessary.
But it’s very far from the point where programmers would be unnecessary.
LLMs lack any understanding of the bigger picture. They work for tiny throwaway tasks that you won’t use in the future. The stuff they make is hardly runnable, let alone maintainable. It usually needs multiple rounds of asking it to do the right thing before you get anything remotely worth your time.
The feedback loop with static code analysis and so on generally doesn’t work. Once an LLM gets lost enough to produce compiler errors (which happens quite fast) it gets completely lost, since it then only focusses on fixing the compiler errors while totally forgetting what the actual task was. So it will change the program so that it compiles, but then it will not do what it was supposed to do.
It’s a hot mess.
Studies say that using LLM while programming reduces measured performance of the programmer by 20-30% as soon as the task isn’t something entirely trivial.
Edit: When an LLM gets into that compiler error loop, it’s like taking a non-compiling program written by a first-year student and handing it and the compiler output to the rest of the class and everyone sequentially gets a shot at fixing it while completely ignoring anything the others have done befora apart from taking their code as input.
Only the very junior ones who are writing code without understanding why they’ve been asked to write it. Anyone with more than about 18 months experience will be able to start deciding what to actually build, and I haven’t seen LLMs be particularly helpful with that yet.