But if you go to the repo, and search all pull requests by author, nothing comes up for claude
I’m no fan of AI generally, but “AI Vulnerable” as a term just doesn’t make much sense to me. Code reviewing should be filtering out bad code whether it originates from an AI or a human.
PR spamming with the usage of AI is another problem which is very serious and harmful for OSS, but that’s not due to some unique danger that only AI code has and human contributors don’t.
Code reviewing should be filtering out bad code whether it originates from an AI or a human.
But studies are showing it doesn’t work.
A human makes a mental model of the entire system, does some testing, and submits code that works, passes tests, and fits their unstanding of what is need.
A present day AI makes an educated guess which existing source code snippets best match the request, does some testing, and submits code that it judges is most likely to pass code review.
And yes, plenty of human coders fall into the second bracket, as well.
But AI is very good at writing code that looks right. Code review is a good and necessary tool, but the data tells us code review isn’t solving the problem of bugs introduced by AI generated code.
I don’t have an answer, but “just use code review” probably isn’t it. In my opinion, “never use AI code assist” also isn’t the answer. There’s just more to learn about it, and we should proceed with drastically more caution.
Here’s an example I ran into, since work wants us to use AI to produce work stuff, whatever, they get to deal with the result.
But I had asked it to add some debug code to verify that a process was working by saving the in memory result of that process to a file, so I could ensure the next step was even possible to do based on the output of the first step (because the second step was failing). Get the file output and it looks fine, other than missing some whitespace, but that’s ok.
And then while debugging, it says the issue is the data for step 1 isn’t being passed to the function the calls if all. Wait, how can this be, the file looks fine? Oh when it added the debug code, it added a new code path that just calls the step 1 code (properly). Which does work for verifying step 1 on its own but not for verifying the actual code path.
The code for this task is full of examples like that, almost as if it is intelligent but it’s using the genie model of being helpful where it tries to technically follow directions while subverting expectations anywhere it isn’t specified.
Thinking about my overall task, I’m not sure using AI has saved time. It produces code that looks more like final code, but adds a lot of subtle unexpected issues on the way.
It produces code that looks more like final code, but adds a lot of subtle unexpected issues on the way.
That is an excellent summary of the challenge. The code looks high quality sooner in the debug lifecycle, which actually makes debugging a little bit slower, at least with our current tools.
Yeah, it’s good enough that it even had me fooled, despite all my “it just correlates words” comments. It was getting to the desired result, so I was starting to think that the framework around the agentic coding AIs was able to give it enough useful context to make the correlations useful, even if it wasn’t really thinking.
But it’s really just a bunch of duct tape slapped over cracks in a leaky tank they want to put more water in. While it’s impressive how far it has come, the fundamental issues will always be there because it’s still accurate to call LLMs massive text predictors.
The people who believe LLMs have achieved AGI are either just lying to try to prolong the bubble in the hopes of actually getting it to the singularity before it pops or are revealing their own lack of expertise because they either haven’t noticed the fundamental issues or think they are minor things that can be solved because any instance can be patched.
But a) they can only be patched by people who know the correction (so the patches won’t happen in the bleeding edge until humans solve the problem they wanted AI to solve), and b) it will require an infinite number of these patches even to just cover all permutations of everything we do know.
A present day AI makes an educated guess which existing source code snippets best match the request, does some testing, and submits code that it judges is most likely to pass code review.
That’s still on the human that opened the PR without doing the slightest effort of testing the AI changes though.
I agree there should be a lot of caution overall, I just think that the problem is a bit mischaracterized. The problem is the newfound ability to spam PRs that look legit but are actually crap, but the root here is humans doing this for Github rep or whatever, not AI inherently making codebases vulnerable. There need to be ways to detect such users that repeatedly do zero effort contributions like that and ban them.
Yes, it is their fault, and also, that fault is a widespread problem
I don’t code so correct me if I’m wrong, but wouldn’t the code have to be generally accepted, reviewed, and verified by other members of the project? Ai can fuck right off as far as I’m concerned, but this isn’t a situation where a CEO just unilaterally decides vibe coding is the move. Unless I’m mistaken.
Yeah. The real problem with AI generated code in open source projects is people flooding projects with slop merge requests.
why is cpython on github? I thought they had their own forge like GNOME and KDE
The only link listed on wikipedia is on github
They moved to GitHub a few years ago, mostly for the benefits of issue tracking, which previously was not integrated in the forge IIRC.
Context?
cpython is the reference implementation of the python interpreter. The person who took this screenshot has the Claude user on GitHub blocked so that whenever it contributed to a git repo you see this warning. The Claude user is an AI agent. AI code is garbage.
That was as to the point as humanly possible. Thank you.
What should be done?
Waybe it’s save if the maintainer reviews PRs before merging
The problem is they get overwhelmed with these PRs. Godot has been talking about not being able to manage the workload lately, people just task AIs to vibecode fixes to perceived bugs and half of them don’t even do what they were prompted to do.
You can block those users but they just make new accounts
It honestly feels like a DDoS on do it yourself computing, by corporations who want total control over our thoughts.
Slop PRs are submitted by users, not by a Claude bot like this screenshot refers to
It may be good to allow repo owners reputation-based PR filters based on account age, history, rejected PRs etc.
I can’t wait for the money to dry up. It’s insane to me just how stupid people have been, trusting LLMs with anything whatsoever. These things cost so much money to run and they seem to fucking hypnotize investors into burning their money. Sooner or later the fact that they’re not making money has to catch up with them, right?
Thank for the explanation! The user in the image is claude itself, not a random anonymuous user. I see the problem of the ddos with issues, tickets etc. that is a real problem! But I don’t get the rigid denial of generative ai. As long as I review the code it generates, it can save me lots of time. I would hate the actions you described as well but the image depicts nothing fishy. Am I wrong about this?
And maybe the janitor should sift through that river of diarrhea for the couple of pennies someone might have swallowed.
I mean it’s Python. This is what we get for having been overly reliant on it.
All kidding aside, I am a more than a bit confused by this.
Maybe I don’t understand what this warning means, but I don’t see anything here: https://github.com/python/cpython/issues?q=is%3Apr+author%3Aclaude
Does this mean something else?
If you block Claude, or any user really, and then visit a repo they’ve contributed to you will see this message.
Maybe Claude didn’t open the PR but contributed commits.
I tried a
git log --grep=claudebut it doesn’t net much, basically just this PR (which in fairness does look vibecoded).Maybe there’s some development branch in the repository that has a commit authored by Claude but if so it’s not on main.
sadge
CC is actually really good if you know what you’re doing. The only issue imo would be PR spamming
I do not allow it committing
How hard can it be to have an AI take PR’s from other AI’s and clean out the worst + plus hardening PR protocols ? It could even assist/guide AI contributors via a special AI-contributor forum or whatever. AI are currently high-lighting a lot of ‘holes’ in systems where we expect a certain behavior. Just coping/complaining and closing things off is a bad decision, and we should accept these flaws in our systems and adapt them to a new world. The sooner the better.
The projects that get it right, now have an army of managed AI contributors, and a filtered/educational AI PR pipeline where project maintainers cherry-pick the top creme de la creme…
This is a fantasy. LLMs will just produce the same errors over and over a lot of the time.
We’ve got several months of evidence as to how hard.
What is “AI vulnerable”? What is the problem here? Claude isn’t reverse-Midas, it’s not like everything they touch turns to shit.
Studies continue to show that AI routinely generates unsafe code and even human code reviews often don’t catch major problems. AI generated code should not be trusted or accepted and projects that accept them should be treated as compromised.
Alright, well I use Claude in my code and it produced a better library than anything that was publicly available on Github from me just feeding a PDF of a datasheet for the module into an LLM.
I’m all for not blindly trusting AI, give it limits, review and test everything it makes, but flat out rejecting any AI generated code as “compromised” feels reactionary to me.
I understand the problems, but I don’t think they amount to something as simple and close-minded as “all LLM generated code bad and evil”, unless thinking critically takes too much time and energy I guess? Some people just have to make blanket decisions because it’s easier for them.
Humans can barely write safe C code, so I definitely don’t trust AI to. I’m not even blanket against AI assistance in programming, but there are way too many hidden landmines in C for an LLM to be reliable with.
I use it in C++ and it has been very helpful. The OP appears to be just blanket against AI assistance in programming? There’s no indication of what degree Claude was involved here, or what amount of blind trust the human reviewers gave to it.
I agree with you. More to the point…why accept code from anyone (clanker or meatbag) without provenance?
If I don’t know you, and you can’t explain what it does? Straight into the garbage it goes.
The issue isn’t AI contamination. It’s accepting code from any source without provenance and accountable review.
I suspect the anti-AI push is coming from Russia or China, probably because the AI products that are in such high demand right now are of Western origin.
