New paper: Machine learning and statistical inference in microbial population genomics

We have published a new review article in Genome Biology contrasting machine learning and statistics in microbial genomics. This is joint work with Sam Sheppard, Nick Arning and David Eyre.

The availability of large genome datasets has changed the microbiology research landscape. Analyzing such data requires computationally demanding analyses, and new approaches have come from different data analysis philosophies. Machine learning and statistical inference have overlapping knowledge discovery aims and approaches.

In this review, we highlight how machine learning focuses on optimizing prediction, whereas statistical inference focuses on understanding the processes relating variables. We outline the different aims, assumptions, and resulting methodologies, with examples from microbial genomics. These approaches are essentially complementary, and we argue that exploiting both machine learning and statistics - selecting the right tool for the job - has the greatest potential for advancing pathogen research in the big data era.

Machine Learning versus Statistical Inference in Microbial Genomics

My talk given today at the 2025 Microbiology Society Conference in Liverpool:

Abstract

The advent of vast genomic datasets has transformed microbiology, presenting opportunities and challenges for data analysis. The distinct philosophies of machine learning (ML) and statistical inference gives them complementarity strengths and weaknesses in tackling big data problems in pathogen research. While statistical inference prioritizes understanding underlying relationships, ML focuses on optimizing predictive performance. In this talk I will contrast the approaches and offer a view on their relative utility for three problems: source attribution, bacterial genome-wide association studies, and predicting antimicrobial resistance phenotypes from whole genome sequences.

New paper: Machine learning to predict the source of campylobacteriosis using whole genome data

This study, published in October in PLOS Genetics, brings together machine learning, large bacterial isolate collections and whole genome sequencing to address the general problem of how to trace the source of human infections.

Specifically, we investigated campylobacteriosis, a common infection of animal origin causing ~1.5 million cases of gastroenteritis and 10,000 hospitalizations every year in the United States alone. We show that our combined machine learning/genomics analyses:

• Improve the accuracy with which infections can be traced back to farm reservoirs.
• Identify evolutionary shifts in bacterial affinity for livestock host species.
• Detect changes in human infection capability within related strains.

These results will improve understanding not only of Campylobacter, but more generally as these technologies can readily be applied to other important bacterial pathogen species.

This paper builds on previous work published by the group, including our well cited Tracing the source of campylobacteriosis (Wilson et al 2008, PLOS Genetics 4:e1000203). The use of these methods for tracing infection has influenced public health policy and contributed to reducing disease burden.

This work demonstrates the potential for modern genomics and artificial intelligence approaches to address common and serious problems that affect our everyday lives. The awareness of the importance of infection to society has rarely been higher than in 2021, and while the current pandemic imposes an acute global problem, other infections continue to present long-term threats to health and productivity.

This work was led by Nicolas Arning, in collaboration with David Clifton and Sam Sheppard.

New paper: Rapid host switching in Campylobacter

Our new open access paper Rapid host switching in generalist Campylobacter strains erodes the signal for tracing human infections was published last week in the ISME Journal.

 

With Bethany Dearlove, Sam Sheppard and colleagues, we investigated common strains of campylobacter, the most frequent cause of bacterial gastroenteritis worldwide. Campylobacter infection is associated with food poisoning, particularly contaminated chicken. But in previous work, we found that certain strains (the ST-21, ST-45 and ST-828 complexes) are often found contaminating a range of meat and poultry, making it difficult to trace the source of human infection.

That previous work was based on partial genome sequencing known as MLST. In MLST, less than 1% of the information in the genome is captured. Now that whole genome sequencing is available, the expectation was that we should be able to distinguish easily between between ST-21, 45 and 828 strains contaminating poultry versus beef versus lamb, and so on.

What we found was surprising. Instead of these strains harbouring previously unobserved sub-structure that allowed them to be associated with different animal sources, we found rapidly mixing populations undergoing extremely fast transmission between animal species, with campylobacter strains ricocheting among animal species on a timescale of just a few years. This is faster than they can accumulate enough mutations to differentiate populations colonizing different animal species.

Our results present an unforeseen roadblock to tracing transmission with whole genome sequencing, and suggests these strains are adapted to a generalist lifestyle, shedding new light on the ecology of this pathogen. These findings push back against the tide of opinion that whole genome sequencing is necessarily a panacea for detecting transmission, and demonstrate that going forwards, a detailed understanding of the biology of zoonotic bacteria (those transmitting between multiple species) and intensive sampling of potential sources are essential for effectively tracing the source of human infection.

World Health Day: Food-borne disease theme

For World Health Day 2015, the group's research into food-borne campylobacter infection was featured on the Nuffield Department of Medicine's home page. The piece features recent work Bethany Dearlove and I have conducted into zoonotic (animal-human) transmission with Sam Sheppard. The paper is currently under review, and a preprint can be downloaded from the website.