The problem with trying to recreate a dynamic cognitive agent lies not just in simply reproducing how the brain stores away the sensory information gathered from the respective sensory organs and relayed to the processing regions of the brain, it also includes the problem (for purposes of emerging these abilities de novo say via an artificial substrate) of needing to emerge the hierarchy of storage that we see in the brain.
The neocortex is not an amorphous block of memory cells devoted to capturing sensory data for each sensation, it is a layered sandwich with particular layers tuned to capturing different parts of the sensory input stream. In the visual cortex for example, specific neurons are in place for detecting walls and edges, this fascinating truth regarding how the actual memory elements are organized into a hierarchy of sensation begs the question as to what emerges the hierarchy in the first place.
Champions of the idea that our brains freely associate relations between things over time from our birth skip this important question, why do we create this stratification...and how would we re-create it in an artificial agent.
I realized that the answer to the riddle takes us in a sense outside of the brain, not far outside though...it forces an appeal to the environmental characteristics (local) to the brains development as a fetus that necessarily constrains how it will grow new neuronal layers...and while it is doing this, the incoming sensation that modulates the growth of those layers precisely in lock step with the external sensory experiences.
Thus the hypothesis is that the brain creates the layers in response to a) the proliferation of cells in the brain as it forms during gestation in the womb and b) the modulation of the pruning of those layers as novel experiences and sensations are gathered by the developing fetus. Thus, the reason there are neurons that are dedicated seemingly to isolating walls from floors is due to the learned realization when we are toddlers that walls block forward motion and floors are necessary to enable it. We learn these things by trial and error and the neurons in our brain are simply gathering this information at that particular point in our brains development and encoding it, as we continue to develop the application of those early patterns learned about the configuration of things in the world are applied to discrete understanding of other concepts (such as walking, dancing and moving through space) so the hiearachy of our brain is not built in...it is learned.
Going deeper in our experience in the womb one should find the foundations for making sense of the external world in the experiences in womb. Hearing and Touch are activated quite early in the fetal development process and the encoding of those experiences mean that they are highly persistent experiences that play a low level role in shaping how the fetus will develop outside of the womb. Again, these experiences lay down the first networks of experience upon which more complex relations are then built as the child develops.
If true this hypothesis points to another critical aspect that must be trained into dynamic cognitive agents that absent it would emerge a seemingly disembodied consciousness rather than the integrated consciousness that would mirror our own, getting it wrong could put us on the wrong side of what we'd prefer to be given a moral stand point...it also runs the risk of generating a rogue consciousness that is not amenable towards being benevolent to us...and that is something we do not want to happen.