To extend this a little bit, I’m not convinced “is X conscious?” is really the question anyone is trying to answer. What I think we’re really trying to sus out is “does X require rights?” and where is the line for that.
As another commenter asked, something like “is turning this off equivalent to murder?” is effectively asking if the thing deserves a “right to life” like any human might. At what point does a “thinking machine” cross the line from “person-like” to “person”? I doubt anyone has a satisfactory answer to that question and, unfortunately, I strongly doubt we’ll have one until well after it’s actually needed.
I think grappling with that question is maybe a little more straightforward when we consider other animals we already consider highly intelligent (e.g. pigs, dolphins, or octopi) but that we don’t give the same kinds of rights to that we would a human. At what point would we consider a non-human animal to be equal to ourselves? How many person-like traits does something need before it is a person?
Anyways, all that aside, I think we should start asking the questions we’re really trying to answer and stop using other questions as proxies for that one.
To extend this a little bit, I’m not convinced “is X conscious?” is really the question anyone is trying to answer. What I think we’re really trying to sus out is “does X require rights?” and where is the line for that.
As another commenter asked, something like “is turning this off equivalent to murder?” is effectively asking if the thing deserves a “right to life” like any human might. At what point does a “thinking machine” cross the line from “person-like” to “person”? I doubt anyone has a satisfactory answer to that question and, unfortunately, I strongly doubt we’ll have one until well after it’s actually needed.
I think grappling with that question is maybe a little more straightforward when we consider other animals we already consider highly intelligent (e.g. pigs, dolphins, or octopi) but that we don’t give the same kinds of rights to that we would a human. At what point would we consider a non-human animal to be equal to ourselves? How many person-like traits does something need before it is a person?
Anyways, all that aside, I think we should start asking the questions we’re really trying to answer and stop using other questions as proxies for that one.