More than two decades after scientists first sequenced the human genome’s 3 billion DNA letters, most of that code’s meaning remains unclear. Only 1 to 2% of human DNA codes for proteins; the rest is a mix of leftover junk DNA and regulatory elements that control gene expression. These non-coding regions of the genome could hold the key to understanding a variety of inherited traits, but first, scientists have to understand how variants in this DNA contribute to the multitude of traits that make each of us unique.
To address this, researchers at the University of California, Berkeley have built a genomic language AI model, called GPN-Star, that outperforms competing models at identifying genetic variants most likely to influence inherited traits, including disease risk, while using far less computing power to train. The study was published in Nature.
“Our model excels in making predictions about the pathogenicity of genetic variants, and identifying functional versus non-functional elements in the genome,” said Yun Song, the study’s senior author. “We hope our work will help drive biological discovery,” Song said. “We believe our predictions will help to prioritize the experiments that could have the greatest impact on human health.”
Genomic language models work a little like chatbots trained on DNA instead of text, learning to recognize patterns that reveal a genome’s functional elements. Most such models, including the massive Evo 2 model, train on unaligned genomes across all domains of life, an approach demanding enough that Evo 2 took 2,000 processors and months to build. GPN-Star instead trains on whole-genome alignments, comparing hundreds of species’ genomes to a single reference to flag conserved or changed code, work that can take just days on a handful of processors. “We tried to help the model learn by curating data that’s more likely to harbor functional elements,” Song said.
The team trained GPN-Star on alignments anchored to humans and five other species, including human-anchored versions built at primate, mammal, and vertebrate timescales. “We found that models trained at different evolutionary time scales were actually optimized for interpreting different kinds of genetic variants,” said co-first author Chengzhong Ye. Models trained on longer timescales were better at judging rare, slow-evolving protein variants, while the primate-specific model better predicted variants tied to complex traits like schizophrenia risk. “For complex traits, we were surprised and pleased to see that training a model that’s specific to primate genomes—which are more relevant to recent human evolution—really helped us make better predictions,” Song said.
Because GPN-Star needs so little computing power to train, the researchers hope other teams can easily adapt and build on their work. “We’re making great progress,” added co-first author Gonzalo Benegas. “But the more people that can work with these models, the better they will get.”
To address this, researchers at the University of California, Berkeley have built a genomic language AI model, called GPN-Star, that outperforms competing models at identifying genetic variants most likely to influence inherited traits, including disease risk, while using far less computing power to train. The study was published in Nature.
“Our model excels in making predictions about the pathogenicity of genetic variants, and identifying functional versus non-functional elements in the genome,” said Yun Song, the study’s senior author. “We hope our work will help drive biological discovery,” Song said. “We believe our predictions will help to prioritize the experiments that could have the greatest impact on human health.”
Genomic language models work a little like chatbots trained on DNA instead of text, learning to recognize patterns that reveal a genome’s functional elements. Most such models, including the massive Evo 2 model, train on unaligned genomes across all domains of life, an approach demanding enough that Evo 2 took 2,000 processors and months to build. GPN-Star instead trains on whole-genome alignments, comparing hundreds of species’ genomes to a single reference to flag conserved or changed code, work that can take just days on a handful of processors. “We tried to help the model learn by curating data that’s more likely to harbor functional elements,” Song said.
The team trained GPN-Star on alignments anchored to humans and five other species, including human-anchored versions built at primate, mammal, and vertebrate timescales. “We found that models trained at different evolutionary time scales were actually optimized for interpreting different kinds of genetic variants,” said co-first author Chengzhong Ye. Models trained on longer timescales were better at judging rare, slow-evolving protein variants, while the primate-specific model better predicted variants tied to complex traits like schizophrenia risk. “For complex traits, we were surprised and pleased to see that training a model that’s specific to primate genomes—which are more relevant to recent human evolution—really helped us make better predictions,” Song said.
Because GPN-Star needs so little computing power to train, the researchers hope other teams can easily adapt and build on their work. “We’re making great progress,” added co-first author Gonzalo Benegas. “But the more people that can work with these models, the better they will get.”
