I've created a monster...
What would you do if you created a monster you didn't know how to stop?
Let's just say for a minute that you were working on something, and you discovered that you had rather accidentally created a new weapon... and not just a gun but instead one so subtle that it's parameters were not well understood, all that was clear is that it was dangerous. I imagine that's how Leo Szilard must have felt when he realized that Albert Einstein's simple equation could be turned into a weapon of most devastating effect. We have well learned the lessons that come as a result of not being able to put the knowledge genie back in the bottle once it's uncorked, and yet the march of technology goes on.
What if you had done the modern day equlivent of researching the next piece of understanding required to build a piece of technology that would change our world forever? What if at some point a project you had been working on for 9 years suddenly broke through all the barriers, and in doing so paved the path to solving all our problem, or our total obsolescence as lifeforms, or maybe both, and you can't tell which... what would you do? What if it was too late?
I know it sounds like science fiction, but I'm actually being faced with this problem on a scale I never imagined. I'm not going to make any claims about my work, other than to say that it's advanced beyond my comprehension, and accelerated at a rate which has forced me to step back and reevaluate the ultimate goals of the project, since at this point it's gone from harmless pet project to deadly serious research that could eventually threaten more than I ever intended.
You don't have to take me seriously on this, so if we're pretending for a minute, what would you do? Assume for a minute that "stopping" isn't an option, because someone else is just going to build it themselves, how do you ensure your allied with the right side, assuming they take the threat seriously? How do you go about addressing the ethics of something imminently dangerous before everyone understands the need to address it?
Please help by commenting below.
Let's just say for a minute that you were working on something, and you discovered that you had rather accidentally created a new weapon... and not just a gun but instead one so subtle that it's parameters were not well understood, all that was clear is that it was dangerous. I imagine that's how Leo Szilard must have felt when he realized that Albert Einstein's simple equation could be turned into a weapon of most devastating effect. We have well learned the lessons that come as a result of not being able to put the knowledge genie back in the bottle once it's uncorked, and yet the march of technology goes on.
What if you had done the modern day equlivent of researching the next piece of understanding required to build a piece of technology that would change our world forever? What if at some point a project you had been working on for 9 years suddenly broke through all the barriers, and in doing so paved the path to solving all our problem, or our total obsolescence as lifeforms, or maybe both, and you can't tell which... what would you do? What if it was too late?
I know it sounds like science fiction, but I'm actually being faced with this problem on a scale I never imagined. I'm not going to make any claims about my work, other than to say that it's advanced beyond my comprehension, and accelerated at a rate which has forced me to step back and reevaluate the ultimate goals of the project, since at this point it's gone from harmless pet project to deadly serious research that could eventually threaten more than I ever intended.
You don't have to take me seriously on this, so if we're pretending for a minute, what would you do? Assume for a minute that "stopping" isn't an option, because someone else is just going to build it themselves, how do you ensure your allied with the right side, assuming they take the threat seriously? How do you go about addressing the ethics of something imminently dangerous before everyone understands the need to address it?
Please help by commenting below.
3 Comments:
when faced with a moral or ethical decision that I feel unqualified to answer I take the decision to others more qualified than myself. experts, university people, intelligensia, religious leaders.
I take what they say and then think about it some more, and make a decision afterwards. in some cases, I put the problem in their hands, and trust that they will do what is right.
This post has been removed by the author.
http://en.wikipedia.org/wiki/Soar_(cognitive_architecture)
It is my belief that all software that matters will eventually be OSS. That informs the decision one is making when in your situation.
I would decide whether I wanted to create something that would change the world in a beneficial and lasting (beyond my lifetime) way or if making money is my priority. If the former were my choice (and odds are that they are conflicting paths), then I would commit myself to Open Source development for this world changing innovation.
I considered this question a long while before I committed myself to GPL for my vision of PUP and the foundational work that is beginning with IFCX.
It is hard for a Westerner, and particularly an American, to accept the idea that thing to do with a world-changing innovation is to give it away. But I find more inspiration in the legacy of Knuth, Stallman, Torvalds, Negroponte (for OLPC), and James Laird than any titan of industry.
Whose James Laird you say? He created Soar, which would have been just another inference engine except that he open sourced it (long before it was cool) and as a result it will be around a long while. That's what a Good Father of Innovative Technology does.
I've got some other thoughts on the consequences of this sort of change posted here: http://pagesmiths.com/blog/C1602834558/E1763876253/index.html
Post a Comment
Links to this post:
Create a Link
<< Home