that the technology is more or less sticking the landing
Only in competent hands, because everything it generates has to be validated manually. My office uses Copilot, and every competent worker involved in complex projects hates it and only uses it for trivial things, like generating an email response, which you then have to read anyway so you might as well type it yourself in the first place. No one uses it for anything meaningful.
Human validation is propping up the perception of LLM’s.
One cannot trust this technology to do anything overly consequential or precise. It’s like how Theranos’ Edison system could perform maybe four types of blood tests correctly, but the extravagant promises, lies, and outright fraud about the product were contrary to Elizabeth Holmes’ grandiose claims.
We’re only a few years away from similar documentaries about Sam Altman, and if you read the recent Ronan Farrow article about him, maybe not even years.
every competent worker involved in complex projects hates it and only uses it for trivial things, like generating an email response, which you then have to read anyway so you might as well type it yourself in the first place.
Hard disagree. Reading bullshit is painful, but writing it is torture.
Only in competent hands, because everything it generates has to be validated manually.
Same can be said for the output of most interns, and even more senior employees. This is why we have “Quality Systems” safety audits, design controls, and the rest of the regulations which basically set us all up checking each others’ work all day long. No AI required, these systems were shown to improve both safety AND efficiency of industry back in the 1980s, which is why they were rolled out as law for industries like aviation, medical, automotive and finance in the 1990s long before anyone would have claimed that AI of the day was doing anything.
The reason studies say those “measure twice cut once” practices increase efficiency is because mistakes are expensive, extremely expensive when you get to problems like Boeing has, it’s not just the lives lost or cost of crash damage/loss, it’s the reputational impact to the company, public perception diminishing the real value of their products.
ALL those same quality practices apply to AI. People are complaining because AI output is so much faster than people output, leaving people holding the review bag, but… newsflash: AI does quality review too. Imperfectly, incompletely, just like people, except AI does the quality reviews much faster than people. Will AI as author, editor, publisher and critic work? Maybe not as a complete closed loop system today, but the individual functions of AI acting within those systems have all improved dramatically in the past 12 months.
What hasn’t changed? The broad public’s perceptions and growing anxiety. Justifiable concerns about how the powerful owners/directors of AI companies will abuse their influence.
AI does quality review too. Imperfectly,
but the individual functions of AI acting within those systems have all improved dramatically in the past 12 months
I suppose that depends on how we define improvement, because from where I’m sitting, it’s reasonable to be apprehensive about LLM’s and their output when we see spectacular failure after spectacular failure.
Whether it’s bombing a school in Iran because Claude fucked up the targeting, or an AI agent deleting your email inbox or your production database, or creating a court case out of thin air, or stats in a SCOTUS ruling that are fictitious, over and over and over again the extravagant promises they keep telling us are just around the corner appear to be decidedly half-baked.
And if you use Teams or Windows and pieces of functionality that worked for two decades are no longer working as designed in a dependable way, I guess I just don’t know what to tell you.
It makes perfect sense not to trust this technology, and the speed it promises is often mitigated by the fact that you can’t and shouldn’t trust its output, because if you’re the unlucky SOB that doesn’t check a reference, you can literally become national news.
Further, being that it’s already been trained on the entirety of recorded human knowledge, I’m not sure how it gets better either. You can make it faster, but it’s just going to spit out slop at a faster rate.
Whether it’s bombing a school in Iran because Claude fucked up the targeting
I’m going to call user error on that, and I don’t think it matters what system they were using - they were going to make mistakes.
an AI agent deleting your email inbox or your production database
The real error there? Conducting risky operations without backups.
creating a court case out of thin air
That’s just big silicon-brass balls. Interns do it too, but you don’t hear about them. On the other hand, trusting the AI or the intern, that’s disbarment levels of reckless.
It makes perfect sense not to trust this technology
Or any technology, until we have figured out what it is, and isn’t, capable of doing reliably.
But, plenty of people still play Russian Roulette, for one reason or another. Is that the revolver manufacturer’s fault?
being that it’s already been trained on the entirety of recorded human knowledge, I’m not sure how it gets better either
Yes, you can mitigate any risk you can think of with the right planning, but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before. Previous systems had risks, but they were different, more limited in scope, and accounted for.
It may be true that some of these systems are worth those new risks and the planning necessary to find and mitigate them all, but we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.
but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before
I disagree. The risks of arbitrary severity are already there, we have just slowly evolved into handling them how we handle them now, learned (sometimes) from the mistakes either how to handle them better or just accept the consequences as “how things are.”
So, I do agree that rapid switching to letting AI handle things will mis-handle the old risks in new and unanticipated ways. Maybe if AI helps us actually build a space elevator or functional fusion Tokamaks or other things like that then that will be creating significant new risks with their own new issues.
we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.
AI is just the latest shiny in a long list of shinies that enthusiastic entreprenuers have hand-waved all the hard work around the real challenges away about. I think self driving cars are a pretty good example of how this “disaster waiting to happen” will really roll out - things like https://www.youtube.com/watch?v=BYkv6jvTpCchttps://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg etc. Is it regrettable that Elaine died? Yes - right along with the other 6000+ pedestrians killed by human drivers that year in the US alone.
I feel that we must largely agree, because you are making similar points to me, albeit with somewhat different apparent sentiment.
For example, your points illustrate that there are perfectly good examples of the previous systems having horrible, regrettable, issues, as well.
I think the real meat of my concerns with this rapid and, in my opinion, reckless and completely unregulated rollout, is that when something does go spectacularly wrong with an AI product, everyone throws their hands up and shouts “NOT IT!” when its time for responsibility and culpability to get handed out.
Personally, I am totally fine with this technology and really any technology so long as the responsible parties are made responsible for their failures and negligence. Today, big tech is trying to claim all the accolades and money for AI, and zero of the responsibility, and are so far being allowed to do so.
Frankly, it seems that they have so thoroughly captured the levers of government meant to regulate them that we will have to wait for their insurers to force them to take responsibility, since someone has to hold the bag when there’s lawsuits. They’ve got a strong lobby, too.
everyone throws their hands up and shouts “NOT IT!” when its time for responsibility and culpability to get handed out.
Oh, like the financial crisis of 2008-9 - bail us out, we’re too big to fail - it’s not our fault that we rushed into the unregulated territory and ignored all the risk… The taxpayers are the ultimate insurance underwriters.
as the responsible parties are made responsible for their failures and negligence
As such, regardless of technological involvement or not, you must be just about as sad as I am about the current trends regarding responsibility, accountability, enforcement of legal precedents, stability of our systems, etc. In that respect, I’ll say the state of AI is more a symptom than a cause.
wait for their insurers to force them to take responsibility, since someone has to hold the bag when there’s lawsuits.
There was a scene in Star Wars (the original: A New Hope) where “the senate will no longer be a problem, the emperor has disbanded the body…” I’m waiting for them to start talking about doing that with the courts, they talk about every other outrageous thing imaginable.
I think we do, except, AI is just another (big, powerful, broad reaching implications) tech, like the internet, or computers, or nuclear fission, or spaceflight (artificial satellites), or powered flight, or the steam engine, or steel making, or circumnavigation capable sailing ships, or gunpowder…
None of those things switched on all at once in an instant, and AI has been creeping up on us for 60+ years. The past 2 or 3 have been a rather dramatic acceleration, fulfilling much of the promise and expectation of the past 50 years, but it’s still not as great as people imagine it could be - a lot like everything else on the list above. Most people “on the cusp” of those technologies had very different visions for what they would bring to the world compared with what actually happened.
Can we “shoot ourselves in the foot” with this one? Yes, but you could do the same with a rock in a bamboo pole with a little gunpowder, too.
But that’s just one guy’s poor foot, or several volunteers tragically crashing to their deaths, etc. These guys are terraforming the planet to be hostile to ours and many (most?) other species as a side effect of their technology.
Even if this technology is twice as earthshattering as the Internet or gunpowder or you name it, it is hard to see the point if most or even many of us have to suffer and die, similarly condemning our heirs, just to force it to market before its time.
This is not some remote future possibility or granola eating hippie concern. There are permit requests on file all over the North America, among others, for AI data facilities that propose to use more water and power than their entire province/state they reside in does, and are securing sweetheart deals for those resources, passing the costs onto the community. The aquifers are draining, the cities are sinking, and their power bills are rising already, and they’re not even built all built yet.
These technologies have a lot of promise and in some non-LLM applications, have already produced incredible results, but to unleash it for mass consumption is irresponsible on a global scale. I can’t help but see current cloud LLMs as large scale basic answer and structured data generators also capable of the not useless ability to perform hundreds of “bad programmer” man-hours in minutes, but they run on a furnace fueled by metaphorical countless burning orphans, and our business and political leaders are taking the stance that is the non-negotiable cost of progress, and it isn’t.
Only in competent hands, because everything it generates has to be validated manually. My office uses Copilot, and every competent worker involved in complex projects hates it and only uses it for trivial things, like generating an email response, which you then have to read anyway so you might as well type it yourself in the first place. No one uses it for anything meaningful.
Human validation is propping up the perception of LLM’s.
One cannot trust this technology to do anything overly consequential or precise. It’s like how Theranos’ Edison system could perform maybe four types of blood tests correctly, but the extravagant promises, lies, and outright fraud about the product were contrary to Elizabeth Holmes’ grandiose claims.
We’re only a few years away from similar documentaries about Sam Altman, and if you read the recent Ronan Farrow article about him, maybe not even years.
Hard disagree. Reading bullshit is painful, but writing it is torture.
Same can be said for the output of most interns, and even more senior employees. This is why we have “Quality Systems” safety audits, design controls, and the rest of the regulations which basically set us all up checking each others’ work all day long. No AI required, these systems were shown to improve both safety AND efficiency of industry back in the 1980s, which is why they were rolled out as law for industries like aviation, medical, automotive and finance in the 1990s long before anyone would have claimed that AI of the day was doing anything.
The reason studies say those “measure twice cut once” practices increase efficiency is because mistakes are expensive, extremely expensive when you get to problems like Boeing has, it’s not just the lives lost or cost of crash damage/loss, it’s the reputational impact to the company, public perception diminishing the real value of their products.
ALL those same quality practices apply to AI. People are complaining because AI output is so much faster than people output, leaving people holding the review bag, but… newsflash: AI does quality review too. Imperfectly, incompletely, just like people, except AI does the quality reviews much faster than people. Will AI as author, editor, publisher and critic work? Maybe not as a complete closed loop system today, but the individual functions of AI acting within those systems have all improved dramatically in the past 12 months.
What hasn’t changed? The broad public’s perceptions and growing anxiety. Justifiable concerns about how the powerful owners/directors of AI companies will abuse their influence.
I suppose that depends on how we define improvement, because from where I’m sitting, it’s reasonable to be apprehensive about LLM’s and their output when we see spectacular failure after spectacular failure.
Whether it’s bombing a school in Iran because Claude fucked up the targeting, or an AI agent deleting your email inbox or your production database, or creating a court case out of thin air, or stats in a SCOTUS ruling that are fictitious, over and over and over again the extravagant promises they keep telling us are just around the corner appear to be decidedly half-baked.
And if you use Teams or Windows and pieces of functionality that worked for two decades are no longer working as designed in a dependable way, I guess I just don’t know what to tell you.
It makes perfect sense not to trust this technology, and the speed it promises is often mitigated by the fact that you can’t and shouldn’t trust its output, because if you’re the unlucky SOB that doesn’t check a reference, you can literally become national news.
Further, being that it’s already been trained on the entirety of recorded human knowledge, I’m not sure how it gets better either. You can make it faster, but it’s just going to spit out slop at a faster rate.
I’m going to call user error on that, and I don’t think it matters what system they were using - they were going to make mistakes.
The real error there? Conducting risky operations without backups.
That’s just big silicon-brass balls. Interns do it too, but you don’t hear about them. On the other hand, trusting the AI or the intern, that’s disbarment levels of reckless.
Or any technology, until we have figured out what it is, and isn’t, capable of doing reliably.
But, plenty of people still play Russian Roulette, for one reason or another. Is that the revolver manufacturer’s fault?
Better editing.
Yes, you can mitigate any risk you can think of with the right planning, but these are new risks of arbitrary severity, from trivial to devastating, that did not exist at all before. Previous systems had risks, but they were different, more limited in scope, and accounted for.
It may be true that some of these systems are worth those new risks and the planning necessary to find and mitigate them all, but we have to do that hard work and be real with ourselves, rather than hand-wave them away because of the potentially thrilling prospects.
I disagree. The risks of arbitrary severity are already there, we have just slowly evolved into handling them how we handle them now, learned (sometimes) from the mistakes either how to handle them better or just accept the consequences as “how things are.”
So, I do agree that rapid switching to letting AI handle things will mis-handle the old risks in new and unanticipated ways. Maybe if AI helps us actually build a space elevator or functional fusion Tokamaks or other things like that then that will be creating significant new risks with their own new issues.
AI is just the latest shiny in a long list of shinies that enthusiastic entreprenuers have hand-waved all the hard work around the real challenges away about. I think self driving cars are a pretty good example of how this “disaster waiting to happen” will really roll out - things like https://www.youtube.com/watch?v=BYkv6jvTpCc https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg etc. Is it regrettable that Elaine died? Yes - right along with the other 6000+ pedestrians killed by human drivers that year in the US alone.
I feel that we must largely agree, because you are making similar points to me, albeit with somewhat different apparent sentiment.
For example, your points illustrate that there are perfectly good examples of the previous systems having horrible, regrettable, issues, as well.
I think the real meat of my concerns with this rapid and, in my opinion, reckless and completely unregulated rollout, is that when something does go spectacularly wrong with an AI product, everyone throws their hands up and shouts “NOT IT!” when its time for responsibility and culpability to get handed out.
Personally, I am totally fine with this technology and really any technology so long as the responsible parties are made responsible for their failures and negligence. Today, big tech is trying to claim all the accolades and money for AI, and zero of the responsibility, and are so far being allowed to do so.
Frankly, it seems that they have so thoroughly captured the levers of government meant to regulate them that we will have to wait for their insurers to force them to take responsibility, since someone has to hold the bag when there’s lawsuits. They’ve got a strong lobby, too.
Oh, like the financial crisis of 2008-9 - bail us out, we’re too big to fail - it’s not our fault that we rushed into the unregulated territory and ignored all the risk… The taxpayers are the ultimate insurance underwriters.
As such, regardless of technological involvement or not, you must be just about as sad as I am about the current trends regarding responsibility, accountability, enforcement of legal precedents, stability of our systems, etc. In that respect, I’ll say the state of AI is more a symptom than a cause.
There was a scene in Star Wars (the original: A New Hope) where “the senate will no longer be a problem, the emperor has disbanded the body…” I’m waiting for them to start talking about doing that with the courts, they talk about every other outrageous thing imaginable.
I think we do, except, AI is just another (big, powerful, broad reaching implications) tech, like the internet, or computers, or nuclear fission, or spaceflight (artificial satellites), or powered flight, or the steam engine, or steel making, or circumnavigation capable sailing ships, or gunpowder…
None of those things switched on all at once in an instant, and AI has been creeping up on us for 60+ years. The past 2 or 3 have been a rather dramatic acceleration, fulfilling much of the promise and expectation of the past 50 years, but it’s still not as great as people imagine it could be - a lot like everything else on the list above. Most people “on the cusp” of those technologies had very different visions for what they would bring to the world compared with what actually happened.
Can we “shoot ourselves in the foot” with this one? Yes, but you could do the same with a rock in a bamboo pole with a little gunpowder, too.
But that’s just one guy’s poor foot, or several volunteers tragically crashing to their deaths, etc. These guys are terraforming the planet to be hostile to ours and many (most?) other species as a side effect of their technology.
Even if this technology is twice as earthshattering as the Internet or gunpowder or you name it, it is hard to see the point if most or even many of us have to suffer and die, similarly condemning our heirs, just to force it to market before its time.
This is not some remote future possibility or granola eating hippie concern. There are permit requests on file all over the North America, among others, for AI data facilities that propose to use more water and power than their entire province/state they reside in does, and are securing sweetheart deals for those resources, passing the costs onto the community. The aquifers are draining, the cities are sinking, and their power bills are rising already, and they’re not even built all built yet.
These technologies have a lot of promise and in some non-LLM applications, have already produced incredible results, but to unleash it for mass consumption is irresponsible on a global scale. I can’t help but see current cloud LLMs as large scale basic answer and structured data generators also capable of the not useless ability to perform hundreds of “bad programmer” man-hours in minutes, but they run on a furnace fueled by metaphorical countless burning orphans, and our business and political leaders are taking the stance that is the non-negotiable cost of progress, and it isn’t.