

Part of this is a debate on what the definition of intelligence and/or consciousness is, which I am not qualified to discuss. (I say “discuss” instead of “answer” because there is not an agreed upon answer to either of those.)
That said, one of the main purposes of AGI would be able to learn novel subject matter, and to come up with solutions to novel problems. No machine learning tool we have created so far is capable of that, on a fundamental level. They require humans to frame their training data by defining what the success criteria is, or they spit out the statistically likely human-like response based on all of the human-generated content they’ve consumed.
In short, they cannot understand a concept that humans haven’t yet understood, and can only echo solutions that humans have already tried.
I think that’s a bad idea, both legally and ethically. Vehicles cause tens of thousands of deaths - not to mention injuries - per year in North America. You’re proposing that a company who can meet that standard is absolved of liability? Meet, not improve.
In that case, you’ve given these companies license to literally make money off of removing responsibility for those deaths. The driver’s not responsible, and neither is the company. That seems pretty terrible to me, and I’m sure to the loved ones of anyone who has been killed in a vehicle collision.