The research team, led by Lawrence C. Katz, a Howard Hughes Medical Institute investigator at Duke University Medical Center, recorded the firing of neurons in the accessory olfactory bulb, part of a poorly understood sensory pathway that is thought to be important in sex discrimination and social behavior in most mammals.
The results, to be published in a forthcoming issue of the journal Science, show that chemical signals called pheromones trigger highly specific patterns of neural excitation in the brain. These "pheromone images" provide vital information about the sexual receptiveness of females and the dominance hierarchy in males, among other things, said Katz.
"Mice, which live in the darkness in the wild, can readily identify each other on the basis of a pheromonal image rather than a visual image," said Katz.
Both wild and domestic animals, such as dogs and cats, collect pheromone signals through the "flehmen" response, in which the upper lip curls back during exploration of the oral and anogenital areas of other animals during social encounters. These pheromone signals are collected by the vomeronasal organ (VNO), a hollow tube in the nasal cavity. Sensory neurons lining the VNO, in turn, stimulate neurons in the accessory olfactory bulb, a part of the central nervous system. Finally, signals are sent to the amygdala, a part of the brain responsible for basic drives, such as fear, aggression, mating behavior and maternal instincts.
The information contained in pheromone signals is key to survival and reproduction, said Katz. Male mice establish dominance hierarchies, so they need to know if another male is dominant or non-dominant. In addition, males respond to females who are in estrus because they smell differently. "In essence," said Katz, "these pheromonal cues help mice decide 'should I mate or fight.'"
Important clues to the VNO's importance in sex recognition have emerged from genetic studies. For example, HHMI investigator Catherine Dulac and her colleagues at Harvard University reported in January 2002 that mice lacking a key molecule in the pheromone-signaling pathway were unable to distinguish males from females and behaved as if all mice were female.
To capture the pheromonal image created by this accessory olfactory system, Katz and his colleagues, which included Minmin Luo of Duke and Michale Fee of Lucent Technologies in Murray Hill, N.J., developed miniature electrodes and micromotors to record the firing of individual neurons in mice that were awake and behaving normally. The electrodes were implanted in the accessory olfactory bulb, which along with the main olfactory system, processes pheromone signals. The micromotors, which are about the size and shape of a pencil eraser, were light and unobtrusive, so they did not interfere with the normal activities of the mice, said Katz. Once the recording device was attached to the mouse, the researchers introduced another mouse into the cage and allowed the two to interact. In each case, test animals repeatedly explored the faces and anogenital areas of the stimulus animals with their snouts.
The scientists then recorded male mouse responses to females, males of the same and different genetic backgrounds, and castrated males. To be certain they were recording responses to pheromones, the scientists also recorded responses as the test mice investigated fake mice, which never evoked any neuronal response.
"No one has ever recorded from this area because it only works while the animals are awake and exploring their environment," said Katz. "What we've done is look at how that sensory information is sent into a central location and what kind of information is represented in the brain."
When they began their studies, the scientists hypothesized that individual neurons might be responsible for detecting "maleness" or "femaleness," but instead they found a much more sophisticated sensory system that could distinguish individuals with great fidelity.
"The most exciting thing we found was that individual neurons were responsive to individual animals. Each type of animal encountered set off a unique pattern of neural excitation or inhibition," said Katz. "We did not see any neurons that responded to all male mice or to all female mice. They responded to the male mice of a specific genetic identity, but not to male mice of other genetic backgrounds. This suggests there must be pheromones that male mice of one genetic identity have, but that male mice of another genetic identity do not. In essence, each individual animal has a different pheromonal signature."
"What we also learned," he added, "is that there must be pheromonal signals, whose identity we do not yet know, that carry information about sexual identity."
There is evidence that humans also respond to pheromone signals, said Katz. "Don't forget that for years the main ingredient in perfume was a secretion from the anal gland of the civet cat, which is probably full of pheromones. In addition, there is evidence in humans that pheromone-like molecules activate different parts of the brain than standard odorants. And a lot of people think that kissing and all of the other oral investigations that humans engage in is a vestige or even an ongoing part of this pheromone system."