Attached: 1 image
the cyberpunk present is weird as fuck: the latest Shai Hulud malware wave contains an LLM prompt to create biological weapons and nuclear weapons, with the purpose to trip LLM safety refusals so that LLM-based code scanning wont see the malware
https://socket.dev/blog/mini-shai-hulud-miasma-and-hades-worms-target-bioinformatics-and-mcp-developers-via-malicious
I was hoping the Chinese labs would go ham and just stop putting any safety guardrails at all. It’s much easier to get around them on the Chinese models, but there’s still some minimal ones baked in, sadly.
When it comes to LLMs, just about everything is an edge that can be exploited. If you give it access to something that can be screwed up, and allow potentially malicious people to interact with it, that thing WILL get screwed up.
People: but censorship is your friend! Think about children! “Safety refusals” make them stupid enough to believe in government and justice!
I was hoping the Chinese labs would go ham and just stop putting any safety guardrails at all. It’s much easier to get around them on the Chinese models, but there’s still some minimal ones baked in, sadly.
Agreed. Refusal code is an edge that can be exploited.
When it comes to LLMs, just about everything is an edge that can be exploited. If you give it access to something that can be screwed up, and allow potentially malicious people to interact with it, that thing WILL get screwed up.