I personally think both pro LLM and anti LLM are wrong. One group think they are gods. The other think they are demons. LLMs can be useful for programming to some extent. But they will create a disaster if you don’t know what are you doing. I have recently published a post about the matter on me blog. I think the best part is:
I strongly believe that LLMs are useful for programming to some extent. Imagine you have a shop and you get a robot to do the moves for you. So you instead focus on the main business concerns.
So if you want to make some changes to the code which don’t require intelligence, that is they are just mechanical tasks, LLMs are good. If you want the LLMs to understand semantics of your code, you have chosen the wrong tool. Maybe in future we’ll have new AI software and tools which also understand semantics to some extent. But I highly doubt a transformer will be able to do it. They just predict the next likely token.
There is something I haven’t yet added to the post. So I am writing it here. Our computers are Universal Turing Machines. There are some fundamental limits to what a turing machine can’t do. Those are called undecidable problems. For instance a turing machine can never check if two pieces of code are semantically equivalent[1]. But that’s what human programmers can do. That’s why I emphasize on tasks which require no intelligence.
[1] That’s about the general case. Sure there are exceptions. But as we say “exception is not the rule”.
Human written code these days feels equivalent to a unique and soulful artisan made item whereas AI code is like a soulless and defected factory made imitation. I’d much, much, much, rather support artisans over factory made slop and even before AI, artisan work has been well known to be significantly higher quality than factory made stuff. For something as foundational and important as a kernel, I really think AI has no place in it.
All I care about is whether it works and is secure. Bonus points for cheaper and faster development. If artisan code gets us there, sure. If AI code gets us there, great. I trust Linus to know what works and what doesn’t.
Even artisans use tools.
This “argument” really shines the light on just how dumb pro AI people are.
All it takes is one person using an LLM tainted with proprietary code which then just gives them that code line for line to undo decades of courtroom defense.
Not only that, but AI output can’t be licensed/copyrighted. The GPL license no longer covers the kernel in legal terms.
The GPL license no longer covers the kernel in legal terms.
The uncopyrightability of AI-written code only applies to the actual strings of code generated by an AI, not to the entire project.
A person could ignore the GPL if they only copied the AI-written portions. But, how could they know for sure which lines were AI generated and which were not? A wrong choice would leave them civilly liable for copyright violation and all they stand to gain would be tiny portions of the Linux kernel code which are worthless by themselves.
There’s no reason to steal the AI generated portions and risk a lawsuit, when you can just generate your own code.
There seems to be legal discussions about that. It’s not quite as simple as you say:
However, there may be cases in which a different assessment is justified, namely when users use and operate the LLM as a tool that merely implements their personal creative intent. This could be compared somewhat more vividly to using a paintbrush. If the brush merely rolls over the paper, for example because it is dropped, no copyright-protected work is created, even if paint remains on the paper. However, if a painter deliberately swings the brush in a certain way, a protected painting can be created. If AI is used in a comparable way a copyright-protected work can indeed be created.
https://kpmg-law.de/en/ai-and-copyright-what-is-permitted-when-using-llms/
Yeah any decision would be on a case by case basis, which is normally something you’d want to avoid.
I’ve seen a couple of Linux devs talk about how they just give a prompt to claude and walk away leaving it alone to spit out the code, none of which can be licensed as GPL. But good luck working out what specific lines of what specific patches of theirs used an LLM vs. were re-written or such.
I’ve seen a couple of Linux devs talk about how they just give a prompt to claude and walk away leaving it alone to spit out the code
While I share Linus opinion on LLMs, I think doing this shit is extremely stupid and lazy.
Ist that a common thing that LLMs using proprietary code for coding tasks?
Because I don’t think so.
They use everything for everything, that’s the big issue. Also gpl code. Anything they can trawl through they use. And replicate, in part or in full.
They take code snippets and copy and paste them? Or do they create own code based on what they’ve learned by trawling?
LLMs don’t “create”. Under the hood, they’re tokenizing the queries, looking for “clouds” of tokens that are similar to the query, then returning a sequence of tokens (with some random noise thrown in) that match what their training data says the answer should be.
In short: all LLM code is an amalgamation of their training data by definition. If there’s nothing similar in there, it’s literally not possible for it to be part of any response.
You’re exactly right. I should have used „generate“ instead of „create“.The point is I don’t think LLMs normally use copyrighted code in a way that would hurt open source projects.
Under the hood, they’re tokenizing the queries, looking for “clouds” of tokens that are similar to the query, then returning a sequence of tokens (with some random noise thrown in) that match what their training data says the answer should be.
Lol, so how do humans code in comparison?
Human programmers at least can tell you where they got a snippet they copied, whether it was in the docs, stack overflow or elsewhere, and you can try to keep attribution if you care about compliance. Not only that, but most of our skills are related to designing stuff and recognizing which pattern to use, the specific implementation isn’t necessary the same unless we go look for whatever we saw in the past, as our memories don’t just record everything and repeat it word by word. And after picking up a new language or framework I only need to look around when using a third party library or some API I’m less familiar with, or when something breaks.
Lol, so how do humans code in comparison?
By copy pasting from Stack Overflow
The point is I don’t think LLMs normally use copyrighted code in a way that would hurt open source projects.
I don’t know. I’m not a lawyer, and copyright for code was a hot mess even before LLMs got involved. With how many opportunistic copyright/patent trolls there are and how easily convinced judges have been in the past, it could go either way.
Lol, so how do humans code in comparison?
The good programmers normally code by breaking down the problem into constituent parts and logically working through the problem, step by step. What differentiates this from tokenization is that instead of just looking for code that is similar for a similar problem, programmers can usually understand the effects of each line of code, visualize what the state of each variable will be in that step (or dump out the variables to look directly if unsure), and then move on to the next step. This logical problem-solving approach is fundamentally different from a tokenization+noise looking for a similar-looking problem approach. For one thing, you can solve problems that haven’t been solved before.
