One thing that can neither be proven nor disproven is that there exists an external mind-independent world. Why does it appear (according to our best science) that we live in a purely physical world devoid of qualities? Erwin Schrödinger–Mind and Matter
I don’t believe we’ll ever have AI without consciousness. If we are simply opening a new portal to consciousness (like having a baby) it will take a mathematical formula that is self examining.
Voluminous computations will not produce a conscious entity. AI would need consciousness to become intelligent. It is quite likely there is only one consciousness and many apertures, many openings into it—Our brains are receivers. If somehow we created a new form of consciousness it would be immiscible and likely catastrophic to it or us.
Just as everything is made of one process (the collapsing wave function) of a single, fundamental phenomenon, adding another form could be catastrophic. Since there is no evidence for a multiple consciousness model vs a single source model, I would think science should err on the side of caution —but I doubt they will.
Who could we trust to manage the source code?
TheCommonAtheist 