Unconfigured Ad

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts
  • uri.laserson
    Junior Member
    • Mar 2011
    • 1

    SQL vs NoSQL databases for next-gen data?

    I am interested in people's best practices in using a database to store sequencing data.

    I perform lots of immune sequencing, and each one of my reads gets highly annotated. Early on, when I would want to compute something, I used to iterated through an entire flatfile, building some data structure in python (typically dictionaries of dictionaries). Then I realized that I am doing exactly what databases were designed for, including searching, grouping, etc.

    So I elected to try MongoDB, which is a noSQL database. It was easy to use, as you can just dump JSON objects into it, which was a very natural conversion from python. So far, the performance has been very fast, and I am quite pleased.

    But I am now preparing to generate a large amount of additional data, and was wondering whether it would make sense to try out a traditional SQL database, like MySQL or Postgres. Any thoughts as to how they all compare for sequencing data? How do people typically organize their data in a relational database?

    Thanks!
    Uri
  • Luyi Tian
    Member
    • Mar 2012
    • 15

    #2
    Maybe you should try an ORM(Object Relational Mapper) that mixes the features of Object and relational database. I strongly recommend sqlalchemy, which is written in python and has clear documents and examples.

    Comment

    Latest Articles

    Collapse

    • SEQadmin2
      Nine Things a Sample Prep Scientist Thinks About Before Sequencing
      by SEQadmin2


      I’m not a sequencing expert. I’m a purification scientist who uses NGS to evaluate workflows my group develops. With this perspective, we think about the sample first and the NGS workflow second. The sequencer is an exceptionally honest reporter, but it can only report on what you give it, so whether you get clean, interpretable data from an NGS workflow is largely determined before you begin.

      Here are nine questions we think about, in roughly the order they matter, before...
      06-18-2026, 07:11 AM
    • SEQadmin2
      From Collection to Sequencing: Why Sample Preparation and Preservation Define Sequencing Data
      by SEQadmin2


      Data variability is still an issue in sequencing technologies despite the advances in reproducibility and accuracy of these platforms. But the problem does not originate in the sequencing itself, but in the previous steps, before the sample reaches the sequencer.


      The first step is collection, followed by preservation and sample preparation for analysis. Most scientists overlook those steps, but not being careful might just be skewing the experiment’s results.
      ...
      06-02-2026, 10:05 AM

    ad_right_rmr

    Collapse

    News

    Collapse

    Topics Statistics Last Post
    Started by SEQadmin2, 06-26-2026, 11:10 AM
    0 responses
    15 views
    0 reactions
    Last Post SEQadmin2  
    Started by SEQadmin2, 06-17-2026, 06:09 AM
    0 responses
    49 views
    0 reactions
    Last Post SEQadmin2  
    Started by SEQadmin2, 06-09-2026, 11:58 AM
    0 responses
    107 views
    0 reactions
    Last Post SEQadmin2  
    Started by SEQadmin2, 06-05-2026, 10:09 AM
    0 responses
    125 views
    0 reactions
    Last Post SEQadmin2  
    Working...