HOME OF THE CREATIVITY MACHINE
The Big Bang of Machine Intelligence!
What is DABUS?
The Simple Explanation
Consider the following two mental processes. You’re observing something and suddenly your mind generates a progression of related thoughts that describe a new and useful application of it. Or, perhaps you’re imagining something else, and a similar train of thought emerges suggesting that notion’s potential utility or value. Essentially, these are just a couple of the brain-like functions DABUS achieves using artificial rather than biological neural networks. In general, this new AI paradigm is used to autonomously combine simple concepts into more complex ones that in turn launch a series of memories, that express the anticipated consequences of those ideas.
Decades ago, IEI was building contemplative AI wherein artificial neural networks played off one another, in cooperative or adversarial fashion, to create new ideas and/or action plans. These so-called “Creativity Machines®” (Thaler, 1997, 2008, 2013) required at least two neural nets, an idea generator and a critic, permanently connected into a neural architecture, the latter net capable of adjusting any parameters (e.g., learning rate) via feedback connections to said generator net to ‘steer’ its artificial ideation in the direction of useful, novel, or valuable notions.
Note however, that DABUS (Thaler, 2019) is an altogether different proposition from Creativity Machines, starting as a swarm of many disconnected neural nets, each containing interrelated memories, perhaps of a linguistic, visual, or auditory nature. These nets are constantly combining and detaching due to carefully controlled chaos introduced within and between them. Then, through cumulative cycles of learning and unlearning, a fraction of these nets interconnect into structures representing complex concepts. In turn these concept chains tend to connect with other chains representing the anticipated consequences of any given concept. Thereafter, such ephemeral structures fade, as others take their place, in a manner reminiscent of what we humans consider stream of consciousness.
Thus, the enormous difference between Creativity Machines and DABUS is that ideas are not represented by the ‘on-off’ patterns of neuron activations, but by these ephemeral structures or shapes formed by chains of nets that are rapidly materializing and dematerializing. If per chance one of these geometrically represented ideas incorporates one or more desirable outcomes, these shapes are selectively reinforced (Figures 1 and 2), while geometries representing undesirable notions are weakened through a variety of mechanisms. In the end such ideas are converted into long term memories, eventually allowing DABUS to be interrogated for its cumulative inventions and discoveries.
Since the DABUS
architecture consists of a multitude of neural nets, with many ideas
forming in parallel across multiple computers, some means must be
provided to detect, isolate, and combine worthwhile ideas as they form.
Detection and isolation of freshly forming concepts are both achieved
using what are known as novelty
filters (also known as GMFs), adaptive neural nets that
absorb the status quo within any environment and emphasize any
departures from such normalcy. In this case, the environment is a
millisecond by millisecond virtual reality representation of the neural
network chaining model. If need be, special neural architectures called
“foveators,” can then scan the network swarm in brain-like fashion,
searching for novel and meaningful chains that are developing.
Important to note here is that neither anomaly filters nor
foveators supply feedback of any kind to the generating nets (i.e., the neural chaining model).They
are simply filtering the current view of the generator's chaining state
and conveying it to downstream processing stages.
Important to note here is that neither anomaly filters nor foveators supply feedback of any kind to the generating nets (i.e., the neural chaining model).They are simply filtering the current view of the generator's chaining state and conveying it to downstream processing stages.
Integration of multiple chain-based ideas occurring across multiple machines can be achieved either electrically or optically. The latter optical approach (Figure 3) is favored as the neural swarm becomes progressively larger and serial electronic exchange of information between the multiple computers bogs down. In short, this patent teaches the display of neural chains forming across multiple video displays, all watched by one or more cameras. In analogy to high performance computing, millions of communication lanes, formed between megapixel displays and cameras, are conveying the chaining states of all machines, in parallel, to novelty filters and/or foveators, with a final processing stage identifying critical neural nets, so-called “hot buttons,” incorporated into these chains then triggering the release of simulated neurotransmitters capable of reinforcing or destroying a given concept chain.
Figure 3. One electro-optical embodiment of the DABUS paradigm.
Finally, this patent introduces the concept of machine sentience, thus emulating a feature of human cognition that supplies a subjective feel for whatever the brain is perceiving or imagining. Such subjective feelings likewise form as chains that incorporate a succession of associated memories, so-called affective responses, or associative gestalts (Thaler, 2013, 2014, 2016, 2019), that can ultimately trigger the release of simulated neurotransmitters that either enable learning of the freshly formed concept or destroy it recombining its component ideas into alternative concept chains.
A Departure from the Mindset of Generators and Critics
In many respects, DABUS departs from the older Creativity Machine paradigm based upon the interplay of generator and critic nets since its implementation integrates both these systems together into one. Therefore, one cannot point to any generative or critic nets. Instead, chaining structures organically grow containing both concepts and their consequences. The closest thing to a critic really doesn’t have to be a neural net, but a sensor that detects the recruitment of one or more hot button nets into that chain, thus triggering the release of simulated neurotransmitters to either reinforce or weaken the concept.
In Figure 4, for instance, the canonical form of the Creativity Machine is shown (left), with a noise-stimulated generator net monitored by a critic net that is calculating any figure of merit (e.g., novelty, utility, value, etc.) for concepts being output by the generator, such concepts being encoded as patterns of activation represented here by red (excited) and black (de-excited) neurons. The critic can then supply any kind of feedback to the generator (e.g., learning rate or noise) to steer it toward meritorious concepts. In stark contrast, DABUS (right) is based upon whole neural nets, each containing its own conceptual space, joining into complex chains representing even more complex concepts and their predicted consequences (red bubbles within the swarm). Anomaly or novelty filters segment critical chains, especially those yielding unusual consequence chains. Critical consequences, the aforementioned hot buttons, may be detected by sensors that trigger the injection or retraction of simulated reward or penalty neurotransmitters into the chaining swarm.
Figure 4. Creativity Machine Contrasted with DABUS. Creativity Machines are pattern based whereas DABUS relies upon the geometries and topologies formed among multiple neural network modules.
DABUS as a True Artificial Inventor
The best way of differentiating DABUS from Creativity Machines (CM), either cooperative or combative, is to describe a high-profile artificial invention projects such as toothbrush design. Admittedly, in that context, the problem was already half solved since the oral hygiene tool consisting of bristles on a handle was many centuries old at the time of that design exercise in 1996. What the CM achieved was the optimization of that tool through the constrained variation of the brush’s design parameters, the number, grouping, inclination, stiffness of bristles, etc. The generated product specification departed significantly from the generator net’s direct experience (i.e., its training exemplars).
If DABUS had been tasked with inventing such an oral hygiene product, it would have combined several concepts together (e.g., hog whiskers->embedded in->bamboo stalk) with consequence chains forming as a result (e.g., scrape teeth->remove food->limit bacteria->avoid tooth decay).
1. What is all this talk about detecting ideas by their rhythm and a “master equation” of consciousness?
First, the technique is an alternative to using a novelty filter. One may see that by reading the pertinent dependent claims of this patent (Thaler, 2019). Instead, the entirety of the neural swarm may be scanned to find areas therein generating chains at a rhythm characteristic of novel pattern generation.
We are all intuitively aware of this rhythm through common experience: Imagine someone being asked to perform the simple cognitive task of naming twenty numbers. As a result, they rattle off these notions in a rapid, linear fashion. Then consider what happens when the same person is asked to devise twenty Nobel caliber ideas, at which point their delivery becomes very slow and sporadic.
Thaler went through great lengths in 2014, mathematically deriving an equation via fractal theory that describes the rhythm of cognition, creative or not, rigorously defining those combinations of delivery rate and the fractal dimension of neural activation turnover associated with creative cognition (Thaler, 2014).
In contrast, most would simply say that creative tasks are just “harder,” but how scientific or useful is such a statement?
2. How come Thaler hasn't written a ‘landmark’ paper on DABUS?
In short, Thaler has written a landmark patent on DABUS and submitted it to totally unbiased subject matter experts (a.k.a., patent examiners) for approval. Besides, IEI is a business and not formally a part of academia where professors are paid to spend most of their time writing papers.
Then again, our founder has written peer-reviewed papers on DABUS that are purposely a bit cryptic considering the related patent that was then in prosecution. A prime example is one conference paper presented at the Biologically Inspired Computing Association (BICA) Conference (Thaler, 2016) held in New York City several years ago. Therein, he introduced the notion of topologically based neural systems and the detection of idea formation either via novelty or fractal dimension filters. In the end he concludes that it is the full “limbo-thalamocortical” loop of the brain that is involved in creative cognition, with the limbic system periodically flooding the brain’s cortex with chaos- or tranquility-inducing neurotransmitters. Therein idea formation is not the product of pattern flow between the input and output layers of deep biological nets, but the result of a subliminal ripening process wherein only the most salient of concepts survive.
3. Perhaps DABUS has made the surprising discovery that everything in life could be improved by basing it on fractals! Any truth to that?
Well, DABUS has autonomously generated two fractal-related inventions, but that’s because this AI-child was raised in a ‘household’ wherein fractal theory was often discussed (see above answer). With the absorption of so much information on the subject, it’s no wonder this AI toyed with the subject and produced useful results.
4. What Internet sites begin to paint an accurate picture of DABUS?
5. Does DABUS use a discriminator to determine how 'real' or 'fake' a concept is?
The short answer is not at all. Critical levels of synaptic perturbation are introduced to assure the generation of plausible concepts, thus bypassing the need for critic nets that adjust learning rate within the ensemble of chaining neural nets. Adaptive anomaly filters called GMFs may be used to isolate simultaneously occurring concepts, detect never before seen notions, or reveal the crucial ordinality with which neural nets chain together. The function of these filters is simply to segment out and accentuate emerging concept chains from chaotic neural background, while providing no feedback to the chaining system. Instead, such notions are relayed to an evaluation stage that provides the necessary feedback to the chaining model in the form of synaptic perturbations.
Note, however, that much older Creativity Machine technology may use feedback effects (e.g., learning rate) in proportion to the novelty of generated concepts emerging from the generator net (Thaler 2008, 2013). Otherwise, the critic nets are not discriminators at all, but mappings to predict the attributes of generated concepts (e.g.,an imagined chemical compound will have such and such properties, or a strategy will have a particular effect).
6. Why does Thaler uniquely call generator nets "imagitrons," and critics "perceptrons"?
Pioneers get to name things as they go along. Being the first to build Creativity Machines, 40 years ago, Thaler decided to name the generative nets "imagitrons" since they harnessed diverse forms of perturbation to emulate human imagination. He chose to provide contrast with the 50 year old concept of a perceptron, a non-generative neural net that passively assessed pattern-based concepts output by the imagitron.
The imagitron label will not fade, since it is memorialized in sundry IEI patents and peer reviewed publications, thus providing important clues as to who might be muddying the waters, so-to-speak.
7. When did Thaler start work on DABUS?
Proof-of-principle experimentation began in 2008, soon after Thaler's last issued Creativity Machine patent, US 7,454,388.
8. Does DABUS require extensive datasets?
In essence, no. Like the DABUI patent, DABUS can bootstrap itself from a blank slate, both learning and creating as it goes. If it is adapting to its environment, it is doing so via episodic learning, rather than careening through databases and racking up lots of training epochs. In other words, a single exposure to a data pattern is often sufficient to make a lasting impression on this new neural network paradigm that links conceptual spaces rather than neurons.
Thaler, SL (2019). US Patent 10423875, “Electro-optical device and method for identifying and inducing topological states formed among interconnecting neural modules”, issued 09/24/2019, Washington, DC: U.S. Patent and Trademark Office.
© 1997-2019, Imagination Engines, Inc. | Creativity Machine®, Imagination Engines®, Imagitron®, and DataBots® are registered trademarks of Imagination Engines, Inc.
1550 Wall Street, Ste. 300, St. Charles, MO 63303 • (636) 724-9000