Researchers from the University of Liverpool have identified the genetic basis of how mice can recognise close relatives, even if they have never encountered them before.
In the study, published in Current Biology, researchers have demonstrated that a species-specific genetic marker called the major urinary protein (MUP), which is detected through the animal's scent, is used by female house mice to select closely related females as nest partners to help look after their offspring.
The researchers also showed that another scent-based genetic marker, the vertebrate-wide major histocompatibility complex (MHC), is not involved in kin recognition, contrary to previous assumptions that this is how most animals recognise their relatives.
It is well established that animals, including people, bias cooperation towards close relatives because it increases the odds of the genes that they share with relatives being passed to the next generation.
Female house mice can breed cooperatively and usually select relatives as nest partners regardless of prior familiarity, but the genetic markers involved in this recognition have proven extremely difficult to identify.
Professor Jane Hurst, from the University's Institute of Integrative Biology and lead author of the study, said: "This work extends far beyond any previous attempt to identify the genetic basis of kin recognition in vertebrates and strongly challenges the current assumption that there is a common kin-recognition mechanism 'inbuilt' into the immune physiology of all vertebrates."
Previous work by the team provided the first hint that MUP but not MHC might provide a genetic kinship marker to avoid inbreeding with close kin, but could not prove the mechanisms involved.
The next step for researchers is to investigate if other species have evolved similar genetic markers to recognise their relatives and, if so, whether these signals evolve only in species that cooperate with relatives to increase their breeding success.
Professor Hurst said: "We also need to consider the consequences in species that have not evolved these markers - are they more vulnerable to inbreeding accidentally with relatives that they cannot recognise?
"More widely, a better understanding of the importance of social groupings in populations could also have implications for captive breeding programmes and help those managing animals promote better cooperation and social tolerance among animals."