IEI's Patented Creativity Machine® Paradigm

Summary - An artificial neural network that has been trained upon some body of knowledge, and then perturbed in a specially prescribed way, tends to activate into concepts and/or strategies (e.g., new ideas) generalized from that conceptual space. These continuously perturbed networks are called 'imagination engines' or 'imagitrons'. If another computational agent, such as a traditional rule-based algorithm or, even better, another trained neural network is allowed to filter for the very best of these emerging ideas, in terms of novelty, utility, or value, we arrive at an extremely valuable neural architecture, the patented Creativity Machine. Optional feedback interactions (e.g., reinforcement learning) between this latter computational agent and the imagination engine assure swift convergence toward useful ideas or strategies. This new AI paradigm is vastly more powerful than genetic algorithms (GA), efficiently generating new concepts on mere desktop computers rather than on the computational clusters required of GAs. This generative neural network paradigm can and has been extended to whole assemblies of perturbed neural nets generating complex ideas as a multitude of neural modules watch, selectively reinforcing those notions offering novelty, utility, or value of any kind.

Details - Ordinary neural networks excel at learning from raw data, cumulatively learning to associate one pattern with another, as when raw sensory inputs from our five senses activate mental images or feelings. Note, however, that such a direct link between the external world, and our own internal mental life is only a small part of brain activity. To produce a more faithful emulation of human cognition, some mechanism must be established that provides an internal genesis of thoughts and ideas that draws upon cumulative experience, rather than what our senses are telling us at a particular moment.

Creativity Machines represent a new kind of neural network paradigm that can generate rather than just associate patterns. They are based upon what we believe to be a significant scientific discovery: that a neural network exposed to any conceptual space and then internally irritated, in a specially prescribed way, tends to generate coherent and plausible ideas derived from its absorbed wisdom. Another neural network called an "Alert Associative Center" (AAC), trained to filter out the most salient of these notions, patrols the former net's outputs, accumulating useful concepts (i.e., new drug or automobile designs) or using these output patterns in real time to devise strategies (i.e., robotics and control systems). Furthermore, feedback from the AAC may increase the intensity of perturbations being injected into the imagination to produce more novel concepts or decrease such noise to permit reinforcement learning of more meritorious notions.

Perhaps the most appropriate benchmark for Creativity Machines is the highly popularized genetic algorithm (GA) wherein mathematically simulated 'genes' randomly combine and mutate to produce new potential offspring that in turn represent new concepts, designs, or strategies. The problem with such systems is that they work on geologic time scales relative to the inherent processing speed of digital computers. Furthermore, GAs fail to deliver solutions as the dimensionality of the problem increases and the algorithm's generative process leads to combinatorial explosion and nonsensical offspring.

Note that problem dimensionality is not an issue for Creativity Machines. In a nutshell, this advantage stems from the fact that this neural architecture emulates the thalamocortical loop of the brain (i.e., the seat of intelligence and consciousness) rather than blind, and excruciatingly slow processes of genetic mutation and natural selection. Furthermore, the actual time to build a Creativity Machine is negligible to genetic programs since the CM is a self-organizing system. GA's, in contrast are not, and require that human experts hardwire myriad constraint relationships into their human-originated computer codes. In other words, genetic algorithms must be written by human programmers, whereas Creativity Machines build themselves!

Compound Creativity MachinesAdvanced Creativity Machines - Of course, since the mid 80s, Creativity Machines have become much more sophisticated, and by the mid 90s the canonical architecture discussed above had significantly grown to include a plurality of perturbed neural nets monitored and controlled by yet another plurality of critic nets (Thaler, 1996, 1998, 1999, 2013). In these new schemes, the combination of memories of notions and/or strategies offering utility or value to AACs could be selectively strengthened within the generative component at the expense of those deemed less valuable. Continuing this process into the early 2000s, the more promising of these compound notions could be reinforced based upon their predicted consequences (i.e., the chain of events expected to follow). Currently, these advanced, generative neural architectures are serving as laboratories for the study of human cognition and consciousness (Thaler, 2012, 2014, 2016)), in many cases revealing the neurodynamic connection between various psychopathologies and creativity (Thaler, 2016, 2017).

Future Creativity Machines - Over the last few years, IEI has begun the development of Creativity Machine having as many as one trillion neurons and millions of individual neural nets, all in an effort to create brain-like neural systems that can create, invent, and discover on unprecedented levels. For a taste of what can be done artistically with such extensive Creativity Machines, see Leech, 2016. For examples of what these advanced Creativity Machines can invent, see The Artificial Inventor Project.