• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!



This version was saved 11 years, 2 months ago View current version     Page history
Saved by Jim Davies
on July 17, 2012 at 11:50:57 am
Visual Imagination Modeling
When I say I walked my dog this morning, you can picture what that looks like, even though I have told you nothing about where it was, what kind of dog I have, etc. People bring lots of knowledge to bear on their visual imaginations. How is this knowledge used to create imaginings?
This project's long-term goal is to create a computer-program model of human imagination.
In the short term, I am using the results of psychological experiments and data from image information to create a demo. This demo should be able to take simple inputs (e.g. a cat and a house) and create a visual image that has those elements in them, in a way that has some psychological plausibility. Visuo, Detectors, and the Image Oracle are sub-projects.
We have built several programs that report on spatial characteristics and relationships in peekaboom data (e.g., above, occluding). The detectors output a fuzzy belief value between 0 and 1, representing how true the agent believes the relationship to be in the input image. There are several directions we are going with this work:
     Publishing the Detectors:
        We want to publish AI papers based on the detectors as cognitive models. (Connor Smith)
        We also want to publish psychological studies that compare them to human data. (Kensi Dickinson, Michael Forceno)   
      Image Retrieval:
         We are building an image search engine that retrieves from the peekaboom dataset images that have the input description (e.g., bird over tree). (Connor)
      Integration with Visuo:
          Visuo will use the fuzzy belief values output by the detectors as input in its training phase to encode the meaning of prepositions. (Connor, Sterling, Jonathan)
Visuo is a cognitive model of estimation of quantitative magnitudes (e.g., height, size). It takes in data in the training phase. In the visualization phase it uses analogy to estimate magnitudes of things it has never experienced before. Some papers have been published on this work already.
    Integration with Image Oracle:
          The image oracle outputs distance and angle data that Visuo will take as input. (Sterling, Cesar, Jonathan)
    Integration with Detectors:
           See detectors description.
     Natural Language Interface:
           Trying to replicate the work of Wordseye, so that complex paragraphs can be turned into Visuo input (David Dodds)
Image Oracle: 
The Image Oracle has two functions:
     1) takes as input some words, and returns the probability of the top other labels likely to be in the same images.
     2) takes in some words a and some other word b  and returns how likely b is to be in the same image with a.
     Integration with Visuo:
           See description above.
     Helping Computer Vision:
            The oracle will help object recognition systems make better guesses based on the context of other thing in the image. (Cesar)
             Input some labels, and it will output a structured scene description of what is in the image and where they are in relation to each other. (Cesar)
             Grab the pixels from the various images and stitch them together. There have been papers published on this topic in computer graphics.
            We need someone to work on this. 
The Lake Cognitive Architecture
Cognitive models are computer programs made to imitate how people think. Many are special-purpose, for one cognitive task. My dissertation was a cognitive model. A cognitive architecture, in contrast, is a piece of software that you use to create models. So it's got a memory system, attention, etc., that 1) help you create specific models with it, and 2) constrain the model you create so it's forced to be realistic. There are only about 15 in the world.
Colleague Robert West runs the PythonACT-R system: http://sites.google.com/site/pythonactr/


Comments (0)

You don't have permission to comment on this page.