A colonoscopy is the main test used to detect bowel cancer, but like most tests, it is not always 100% accurate and cancers can be missed.
New research led by the University of Leeds looked at the number of patients whose colonoscopy found no evidence of bowel cancer, but who were subsequently diagnosed with the disease.
They found that overall the rate of these post-colonoscopy bowel cancers decreased in England. However, they found that the rates of these potentially missed cancers were lower for colonoscopies performed by the NHS than those performed by independent providers on behalf of the NHS.
Co-author Professor Eva Morris, from the University of Leeds, said: "Overall we found the proportion of people whose bowel cancer is being missed by a colonoscopy procedure has declined, meaning that our diagnostic services are getting more and more accurate.
"However, this study reveals wide variation in the accuracy of colonoscopy tests depending on the provider, meaning that some cancers are still being missed. This urgently needs to be addressed to ensure we are detecting cancer cases as early as possible."
Amongst people diagnosed with bowel cancer who had a colonoscopy within the NHS Bowel Cancer Screening Programme, 3.6% could potentially have been diagnosed at an earlier time point. In contrast, if a person's colonoscopy was undertaken by an NHS commissioned independent provider 9.3% could potentially have been diagnosed sooner.
The researchers say that if the lower rate had been achieved over the entire nine-year study period, more than 3,900 cases of colorectal cancer could have been prevented or diagnosed earlier.
Bowel cancer, also called colorectal cancer, is a major public health problem in the UK, with over 40,000 new cases diagnosed and 16,000 deaths each year.
In their new study published in The BMJ, scientists assessed more than 120,000 people diagnosed with bowel cancer in England between 2005 and 2013. They looked at the number of people who had previously received a colonoscopy, from six months to three years earlier, whose cancer could potentially have been spotted earlier.
Overall, they found that the rates of post-colonoscopy bowel cancer have significantly reduced and are now amongst the lowest rates seen worldwide, reflecting the high quality of colonoscopy for bowel cancer in England.
Amongst those diagnosed with bowel cancer, the proportion of people who had received a colonoscopy within the previous three years who could potentially have been diagnosed sooner declined from 9.0% in 2005 to 6.5% in 2013.
However, what needs addressing is the variation between providers with some having much lower, and some much higher rates.
The British Society of Gastroenterology say post-colonoscopy bowel cancers should be used as a benchmark for the quality of a colonoscopy service.
Lead author Dr Nick Burr, from the University of Leeds, said: "We need a targeted improvement programme to reduce this variation in the quality of colonoscopy procedures between providers.
"Ensuring we have uniform high standards for colonoscopies will help us reduce the number of post-colonoscopy cancers further, and improve mortality from this preventable disease."
Rates of post-colonoscopy bowel cancer diagnosis were higher among women, people aged 80 or over, and those with inflammatory bowel disease.
The lowest rates were achieved by accredited bowel cancer screening providers and some of the higher rates by private providers working on behalf of the NHS. By reducing this variation, and lowering rates further, many cancers could be diagnosed at an earlier stage or even prevented, say the researchers.
Bowel cancer survivor John Barnes said: "Many people undergoing colonoscopy tests aren't aware that cancers or their precursors can be left undetected. The results of this study are both heartening and concerning.
"Clearly when strict colonoscopy standards are applied, as they are in the NHS Bowel Cancer Screening Programme, quality can be improved and lives can potentially be saved.
"This research is an excellent example of how intelligence generated through healthcare data can have a real-world impact and benefit patients."
Co-author Roland Valori, a Consultant Gastroenterologist in the NHS, said: "This research is good news - it shows that the significant effort by the NHS to improve the quality of colonoscopies is working, and the rates of missed cancers are coming down.
"However, we are seeing unacceptable variation in post colonoscopy bowel cancers between providers in the English NHS and this variation in quality needs to be addressed urgently."
This is an observational study so it can't establish cause, and the researchers point to some limitations in the data, which may have affected the accuracy of the calculations.
Nevertheless, they say this is a large, nationally representative study, and the variation between providers remained after further analyses, suggesting that the findings are robust.
The research was funded by the Bobby Moore Fund, Cancer Research UK, Yorkshire Cancer Research, and Crohn's and Colitis UK.