The Pinocchio Function, Part 2: The Program Understanding Program
The story I'm telling here is true. The dates are approximate and the events are summarized, but I'm not making this shit up. This is part two of the recap of the story so far. You can find part 1 here.
I was, at the time, a fledgling programmer: full of vim and vigor, and capable of producing ten times the code that I produce now-a-days. I worked everywhere they would pay me, and did a lot of teaching myself on the job. In fact my first programming position I knew zero about the language I was hired to write in, but that's a different story for another blog post.
It was more or less at the height of the dot-com bubble that I overheard a friend of a friend talking about their idea at a wedding reception in Los Vegas (Jim and Michael M, if you read this I mean you) for a "Program Understanding Program", or more to the point a program that could fix bugs in other programs. He even had a model for it all worked out, which revolved around a programmer defining a fitness function for when the code was working vs. when it was broken ( a unit test ). Thus the role of the programmer changed from defining the path to take to get to the end result by supplying the code to do so, to the role of defining the expected end result and allowing the computer to find the path there.
On the surface, this seemed quite intuitive. We are often in the habit of moving from a concept to a design to a finalized product, all he was doing was shortcutting the design process and going directly from defining the concept to the finalized product and letting the computer figure out the design.
Of course the devil lies in the details, because a Photoshop mockup of a web page just isn't sufficient to properly express all the complex behavior that actually has to go into the end result. However when you applied this at a more modular level, like individual pieces of functionality, the prospects looked rather good.
And so I set about understanding how I would go about building such a thing, with the idea in mind that if I was able to crack this nut, that might in turn lead to a more complex understanding of how life might be born within a pure digital context. And the more I thought about it, the more I came to understand the nature of such a complex thing, and why it would be rather tricky to make work, and in fact I really didn't see the entire path there yet. So this left me with a challenge, one I felt I could overcome by becoming a better computer scientist. I admitted to myself that this was in fact a long term project, and placed it firmly on the top of my list of thing to do before I die, realizing that it would be much later in my life before I was able to even write the first lines of code, and I set about learning the things that I would need to learn.
I studied programming methodologies like object oriented programming and aspect oriented programming. I learned as many programming languages as I could and even invented a few of my own that are still in active production today. I became an expert in telecommunication networks, the most recent of which is the internet. I ate up papers about multi-tape Turing machines and nondeterministic finite state machines with a passion. I joined the security exploit community and became a professional white-hat hacker and amateur theoretical mathematician. I put my self to sleep each night with books and websites on electronics, physics, math theory, computing theory, game theory, set theory, behavioral theory, and linguistic theory. I made games and later even wrote game engines. I stole my father's tools from his toolkit and taught myself how to wire circuits together from the examples given in those 124-in-1 kid's electronic kits. I learned how to program micro controllers from a friend and build entirely custom devices (yet another story for another day). I dove head first into the state of the art in artificial intelligence and neural networks, cellular automata and the game of life, fuzzy logic, machine learning and classifier systems, genetic algorithm optimizations and genetic programming, white boxes, gray codes and black ICE. In short, I embraced my geekiness.
What I found was a lot of science and very little dreams. Neural networks worked great as a classifier system, but nobody figured out how to make them work even nearly as well as a simple animal's brain, let alone approach the complexity of a human. OpenCyc had figured out how to make machines deduce AND infer information, but only when supplied with a highly specific form of context, often anthropomorphic and ultimately utilitarian in nature. They couldn't actually "think" for themselves, or apply the deduced/infered information to any application other than the one of extrapolating more information. And while this is a critical process that we as humans replicate with ease, it doesn't actually supply the computer with that all important Pinocchio Function which makes it go from static machine to evolutionary system.
Somewhere in there I got married and had a kid too, and this overshadowed my early ambitions of creating groundbreaking technology and solving the programmers of the world's problems as the need to devote more time and energy to my family was of much greater importance to me. This of course required doing things like taking well paying jobs writing mundane code, but the idea never left me. I would still think about it often in passing, and remember spending a few nights really thinking about how I would ever get to write what by now had become a pet project for me, my version of the Program Understanding Program, or just PUP.
I sat on the idea, but I never let it go. It was always that thing that I would do later when I understood more. And then I stopped programming for a while and focused on running my own business instead. The dream seemed further away, but still I never gave up on the idea.
At some point I must have realized that it had become -the- beacon for my inspiration. Every time I wrote a piece of code, there was a small part of me that was doing it because I was just practicing my skills for when I could really do what I wanted to do, which sure wasn't this code monkey bullcrap the spiky haired boss was asking me to write when I knew full well that code wasn't going anywhere. Every time I explored another faucet of computer science there was always that glimmer of hope that if I just put another piece of the puzzle together, I would figure it out and be able to unlock the boundless potential that I had glimpsed at as a young adult. Every day where I worked at a job and money wasn't enough to get me through the day and I missed my family, I would console myself by stealing away to read an article on the latest breakthrough in mathmatics, and lose myself in formulas that I only now begin to catch a glimpse of the meaning, and even that is primitive. Every job itself was another vehicle for me to continue my quest.
Was I ambitious? I'd like to think so.
Was I a dreamer? Yeah. No doubt.
Was I fooling myself in thinking that such a quest could ever be embarked upon? Was this just supreme arrogance on my part to think that I, a self taught hacker, could unlock the secrets to such concepts when the people who clearly were much better and brighter than I had stumbled and failed?
I often times found myself questioning that very thing. Pragmatism is such a necessary evil when you have a life worth keeping up that dreams become things that must be maintained if they are to not fall into disuse. Dreams came and went, often dashed by realities too harsh to recall frequently, but this one persisted.
It persisted for 9 years, as I grew the mental model and learned all I could about it. It would show up on occasion in failed code experiments that got lost through the annuls of time and bit-decay. It would get talked about any time someone showed interest in such fields. It would puzzle me constantly in idle moments when I was seeking a mental challenge, and my understanding of what I was after was constantly growing, constantly evolving in my mind as I came to understand more and more about everything I could find that I deemed applicable. But alas, always something was missing, something just beyond my grasp yet tantalizingly present on the horizon.
The piece that was missing was that Pinocchio Function. I could write code, and develop complex systems with my eyes closed. But a system that could write itself, and develop an even more complex system than itself... I just couldn't see the path there.
Not yet.
Coming up in Part 3: Standing On The Shoulders of Giants
I was, at the time, a fledgling programmer: full of vim and vigor, and capable of producing ten times the code that I produce now-a-days. I worked everywhere they would pay me, and did a lot of teaching myself on the job. In fact my first programming position I knew zero about the language I was hired to write in, but that's a different story for another blog post.
It was more or less at the height of the dot-com bubble that I overheard a friend of a friend talking about their idea at a wedding reception in Los Vegas (Jim and Michael M, if you read this I mean you) for a "Program Understanding Program", or more to the point a program that could fix bugs in other programs. He even had a model for it all worked out, which revolved around a programmer defining a fitness function for when the code was working vs. when it was broken ( a unit test ). Thus the role of the programmer changed from defining the path to take to get to the end result by supplying the code to do so, to the role of defining the expected end result and allowing the computer to find the path there.
On the surface, this seemed quite intuitive. We are often in the habit of moving from a concept to a design to a finalized product, all he was doing was shortcutting the design process and going directly from defining the concept to the finalized product and letting the computer figure out the design.
Of course the devil lies in the details, because a Photoshop mockup of a web page just isn't sufficient to properly express all the complex behavior that actually has to go into the end result. However when you applied this at a more modular level, like individual pieces of functionality, the prospects looked rather good.
And so I set about understanding how I would go about building such a thing, with the idea in mind that if I was able to crack this nut, that might in turn lead to a more complex understanding of how life might be born within a pure digital context. And the more I thought about it, the more I came to understand the nature of such a complex thing, and why it would be rather tricky to make work, and in fact I really didn't see the entire path there yet. So this left me with a challenge, one I felt I could overcome by becoming a better computer scientist. I admitted to myself that this was in fact a long term project, and placed it firmly on the top of my list of thing to do before I die, realizing that it would be much later in my life before I was able to even write the first lines of code, and I set about learning the things that I would need to learn.
I studied programming methodologies like object oriented programming and aspect oriented programming. I learned as many programming languages as I could and even invented a few of my own that are still in active production today. I became an expert in telecommunication networks, the most recent of which is the internet. I ate up papers about multi-tape Turing machines and nondeterministic finite state machines with a passion. I joined the security exploit community and became a professional white-hat hacker and amateur theoretical mathematician. I put my self to sleep each night with books and websites on electronics, physics, math theory, computing theory, game theory, set theory, behavioral theory, and linguistic theory. I made games and later even wrote game engines. I stole my father's tools from his toolkit and taught myself how to wire circuits together from the examples given in those 124-in-1 kid's electronic kits. I learned how to program micro controllers from a friend and build entirely custom devices (yet another story for another day). I dove head first into the state of the art in artificial intelligence and neural networks, cellular automata and the game of life, fuzzy logic, machine learning and classifier systems, genetic algorithm optimizations and genetic programming, white boxes, gray codes and black ICE. In short, I embraced my geekiness.
What I found was a lot of science and very little dreams. Neural networks worked great as a classifier system, but nobody figured out how to make them work even nearly as well as a simple animal's brain, let alone approach the complexity of a human. OpenCyc had figured out how to make machines deduce AND infer information, but only when supplied with a highly specific form of context, often anthropomorphic and ultimately utilitarian in nature. They couldn't actually "think" for themselves, or apply the deduced/infered information to any application other than the one of extrapolating more information. And while this is a critical process that we as humans replicate with ease, it doesn't actually supply the computer with that all important Pinocchio Function which makes it go from static machine to evolutionary system.
Somewhere in there I got married and had a kid too, and this overshadowed my early ambitions of creating groundbreaking technology and solving the programmers of the world's problems as the need to devote more time and energy to my family was of much greater importance to me. This of course required doing things like taking well paying jobs writing mundane code, but the idea never left me. I would still think about it often in passing, and remember spending a few nights really thinking about how I would ever get to write what by now had become a pet project for me, my version of the Program Understanding Program, or just PUP.
I sat on the idea, but I never let it go. It was always that thing that I would do later when I understood more. And then I stopped programming for a while and focused on running my own business instead. The dream seemed further away, but still I never gave up on the idea.
At some point I must have realized that it had become -the- beacon for my inspiration. Every time I wrote a piece of code, there was a small part of me that was doing it because I was just practicing my skills for when I could really do what I wanted to do, which sure wasn't this code monkey bullcrap the spiky haired boss was asking me to write when I knew full well that code wasn't going anywhere. Every time I explored another faucet of computer science there was always that glimmer of hope that if I just put another piece of the puzzle together, I would figure it out and be able to unlock the boundless potential that I had glimpsed at as a young adult. Every day where I worked at a job and money wasn't enough to get me through the day and I missed my family, I would console myself by stealing away to read an article on the latest breakthrough in mathmatics, and lose myself in formulas that I only now begin to catch a glimpse of the meaning, and even that is primitive. Every job itself was another vehicle for me to continue my quest.
Was I ambitious? I'd like to think so.
Was I a dreamer? Yeah. No doubt.
Was I fooling myself in thinking that such a quest could ever be embarked upon? Was this just supreme arrogance on my part to think that I, a self taught hacker, could unlock the secrets to such concepts when the people who clearly were much better and brighter than I had stumbled and failed?
I often times found myself questioning that very thing. Pragmatism is such a necessary evil when you have a life worth keeping up that dreams become things that must be maintained if they are to not fall into disuse. Dreams came and went, often dashed by realities too harsh to recall frequently, but this one persisted.
It persisted for 9 years, as I grew the mental model and learned all I could about it. It would show up on occasion in failed code experiments that got lost through the annuls of time and bit-decay. It would get talked about any time someone showed interest in such fields. It would puzzle me constantly in idle moments when I was seeking a mental challenge, and my understanding of what I was after was constantly growing, constantly evolving in my mind as I came to understand more and more about everything I could find that I deemed applicable. But alas, always something was missing, something just beyond my grasp yet tantalizingly present on the horizon.
The piece that was missing was that Pinocchio Function. I could write code, and develop complex systems with my eyes closed. But a system that could write itself, and develop an even more complex system than itself... I just couldn't see the path there.
Not yet.
Coming up in Part 3: Standing On The Shoulders of Giants
0 Comments:
Post a Comment
Links to this post:
Create a Link
<< Home