The researchers have discovered that the animals' pheromone-processing machinery in the brain forms, in essence, a specific "pheromonal image" of another animal. Such an "image" of another animal's sex, identity, social standing and female reproductive status governs a range of mating, fighting, maternal-infant bonding and other behaviors.
The scientists said that the specificity they discovered in the neurons that process pheromonal signals is akin to the "face neurons" in the visual areas of primate brains that are specifically triggered by facial features of other animals.
The researchers reported their findings in an article to be published in the journal Science, and in a talk delivered at the annual meeting of the American Association for the Advancement of Science. The researchers, postdoctoral fellow Minmin Luo and Howard Hughes Medical Institute investigator Lawrence Katz are both at Duke University Medical Center.
"This was truly opening one of the last "black boxes" in the brain," said Katz. "We had no idea what to expect; what these cells were doing. What I found so exciting is that this was a sensory-related brain region into which no one has ever stuck an electrode."
A wide range of mammals, from mice to elephants, possess such a "sixth sense" for detecting pheromones. In such mammals, pheromones are initially sensed via a specialized sense organ, called the vomeronasal organ, in the nasal cavity. The vomeronasal organ actively pumps samples of pheromones into a sensory cavity, where they are detected by chemical receptors similar to those used in taste and smell. These sensory neurons send connections to a neural structure called the accessory olfactory bulb, where Katz's team did their recordings. Whether the same structures persist in humans remains controversial.
"However, this pheromonal system is not a subset of the olfactory system," emphasized Katz. "In a way, it is no more like smell than taste is like smell. The two senses just happen to reside in the same physical region in mammals, but the processing pathways don't directly talk to each other in the brain." In the case of mice, communicating via pheromones is critical to survival, said Katz.
"Since mice live largely in the dark and don't have very good vision, they don't use vision to make critical discriminations among animals," he said. "So, we think our experiments have revealed the front end of a distinct neural processing system that enables them to make these kinds of distinctions."
Previous studies of pheromone-processing in the mammalian brain consisted largely of recording the activity of sensory neurons in slices of the vomeronasal organ -- the peripheral sensory organ -- exposed to such substances as urine, known to contain pheromones. However, said Katz, such studies could not explore how the responses of these receptors are integrated in the animal's brain during the natural functioning of the pheromone system.
"It's like attempting to record from the flight muscles of a bird while it's sitting on a perch," he said. "The vomeronasal organ simply doesn't work unless the animal is conscious and using it. Unless the vomeronasal organ is actively pumping, delivering pheromones to receptors, neurons in the brain that process pheromones cannot be activated. And furthermore, we don't know what these chemical stimuli actually are, their concentrations or their behavioral context.
"Only by recording from an animal that is actively investigating another animal can we hope to study this system realistically." Also, said Katz, pheromones are not volatile, and under natural circumstances can only be detected when one animal contacts another, nuzzling it with its snout.
To measure the neural signals from such behavior, Luo and Katz used an electrode recording system originally developed by co-author Michale Fee of Bell Laboratories to record activity of neurons in the brains of birds as they sang. Basically, the system consists of three tiny micromotors attached to hair-thin electrodes that are insinuated into the pheromone-processing region of the animal's brain. By remote control, the scientists can retract or extend the electrodes by infinitesimal increments, seeking out individual neurons to record their activity during behavior. The electrode system is so small and light that the animal can move about freely, interacting with other animals, while still sending signals over a thin wire to a computerized recorder.
In their experiments, Luo and Katz positioned such electrodes in the "accessory olfactory bulb" (AOB) -- the brain structure that processes signals from receptors in the vomeronasal organ -- of male mice. They then placed other mice into the home cages of the instrumented mice and recorded the activity of the AOB neurons as the test animals investigated the visitors.
Luo and Katz exposed the test mice to different strains of male and female mice.
"The use of different strains was important, because they represent individuals that are genetically different from each other," explained Katz. "So, in exposing the mice to other strains, we were attempting to simulate the animals' exposure to other individuals with different genetic backgrounds."
An initial surprise from the experiments, said Katz, was that some AOB neurons fired most actively as the test animals explored the other animals' heads.
"Urine has been used in much of this work as the main source of chemical stimuli, since it is known that animals investigate the anogenital area extensively," he said. "But we found that actually the head area must also be a rich source of pheromones. While it was known that chemical glands were located there, it was not possible to isolate the tiny amounts of secretion that they produced." Analysis of the firing activity of the AOB neurons revealed that they were highly specific, said Katz.
"We found neurons that are highly selective for another strain, which means in essence that are selective for other individuals," he said. "We also found neurons that responded selectively to a combination of the strain and gender of other mice.
"But what we did not find, even though we expected to, were neurons that responded to all members of one gender or another. We found no neurons that said 'this is a male' or 'this is a female.'"
Thus, said Katz, the pheromonal system may include specialized neurons that combine to create an overall "chemical image" of another animal.
"It may be the rodent equivalent of face recognition in higher primates and humans," said Katz. "In primates there are neurons called "face cells" that are selective for features that we attribute particular importance to, such as the eyes and the mouth. And like those neurons, the selective pheromonal neurons we found seem to respond to specific combinations of features."
Luo and Katz also found that the pheromonal neurons reacted about ten times slower than olfactory neurons, which might reflect their function.
"The main olfactory system needs to react quickly, for example to the whiff of a predator," said Katz. "But one could argue that the pheromonal system is not designed just to get a whiff and make a quick yes or no decision. Rather, it may place a premium on deciphering information about individuals, rather than simply telling informing that another animal is present."
Beyond the new insights into the pheromonal system, the scientists' findings offer general lessons about the chemical senses, said Katz.
"There has been considerable debate about how the brain uses information from the vast array of chemical receptors in the olfactory, taste and pheromonal systems," said Katz. "One view is that the receptors convey rather general information and that the brain uses elaborate computational schemes to extract specific information embedded in the receptor's messages. In that view, the brain circuitry needs to take into account such aspects as the timing and coding of the receptor input. The other view is that receptors can be the front end of 'labeled lines' whose activation conveys the presence of a specific stimulus in the environment.
"The responses we're seeing argue for the second view, because we don't see evidence for broadly responsive neurons that require timing or other correlational information. They just seem to signal the presence of a certain quality in the other animal.
"Also, our conclusion is that this neuronal specificity is likely to reflect specificity present in the vomeronasal receptors as well. So, the most likely conclusion is that these specific responses reflect to a large degree a peripheral specificity, not a sculpting of the response by local circuits within the AOB," said Katz.
Now that Luo and Katz have begun to map the "front-end" pheromonal circuitry, they are tracing the processing of that information deeper into the brain, seeking to understand how pheromones trigger behaviors. They are also exploring the genetic programs that are activated within the brain in the process of pheromone-triggered learning.
"Particularly fascinating about this system is the phenomenon of pheromonal imprinting, especially in females," said Katz. "They essentially memorize the pheromonal image of a male that they mate with, forming a long-term imprint. And what's striking is that only a single exposure can form a memory that lasts a very long time. Studying such systems, we believe, could give us important insights into the molecular basis of memory formation in a mammalian brain."