Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Demultiplexing FASTQ with custom indices

    Hi all,

    I'm fairly new to the realm of bioinformatics with large data sets, so apologies if I've missed something crucial here...

    I've recently received some Illumina HiSeq2500 data in FASTQ format which haven't been demultiplexed. We've used custom i5 and i7 sequences in unique combinations for 96 samples. I was given the data in 8 FASTQ files, 2 per lane (4 lanes) with paired-ends. I've concatenated all of the forward and all of the reverse reads into 2 files for simplicity. I've been using the demuxbyname.sh method through BBMap - but I keep running into a couple of problems:
    1. When I run demuxbyname.sh with a single string I only receive ~2500 reads in the output files. I've noticed that a lot of the index sequences in the FASTQ files contain N's - especially as the first base call (for i5 and i7).
    2. This generally takes ~3hrs, but when I then attempt to run the script with an index.txt file containing multiple index combinations, the compute time increases exponentially.

    Any help on either of these points is greatly appreciated!

  • #2
    Before we get into specifics can you ask your sequence provider to do this demultiplexing with Illumina's program called bcl2fastq (you can't do this since it requires access to the full data folder for the flowcell). That should be trivial for them to do (and they should have done it in first place unless you chose not to give them the sample_ID_index combinations).

    Can you tell us how you are running "demuxbyname.sh" (full command line)? You should run it like this: https://www.biostars.org/p/139395/#139409 You could start multiple runs (even 96 with just one index combo) to speed things up.

    There is also another package called deML that can be used for this.

    Comment


    • #3
      fast and accurate sequence manipulation

      Comment


      • #4
        Thanks for your feedback on this, it's much appreciated!

        I've contacted BGI and they've said that they'll help me with the demultiplexing. I thought it was strange that they simply provided FASTQ files for each lane, especially as they contacted me early on and asked me to provide the index sequences...

        I've run the command a few ways, this is ideally what I'm going for:

        ../sw/bbmap/demuxbyname.sh in=all_lanes_1.fq in2=all_lanes_2.fq out=demux_out/%_1.fq out2=demux_out/%_2.fq prefixmode=f substringmode=f names=index_names_s1.txt

        However, I have run it using single sequence strings, and also just running 1 lane of data at a time. Thanks again for your help.

        Comment


        • #5
          Your indexes most likely look like Index1+Index2 (e.g. GGACTCCT+GCGATCTA) then that is how you need to include them in the file one per line. Is that how you are doing this?

          Comment


          • #6
            Yep my indexes are index1_index2 in the read header, and my .txt file reflects these. I get output files with the index complex names, but these are typically not populated with reads...

            Comment

            Latest Articles

            Collapse

            • seqadmin
              Strategies for Sequencing Challenging Samples
              by seqadmin


              Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
              03-22-2024, 06:39 AM
            • seqadmin
              Techniques and Challenges in Conservation Genomics
              by seqadmin



              The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

              Avian Conservation
              Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
              03-08-2024, 10:41 AM

            ad_right_rmr

            Collapse

            News

            Collapse

            Topics Statistics Last Post
            Started by seqadmin, Yesterday, 06:37 PM
            0 responses
            10 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, Yesterday, 06:07 PM
            0 responses
            9 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 03-22-2024, 10:03 AM
            0 responses
            49 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 03-21-2024, 07:32 AM
            0 responses
            67 views
            0 likes
            Last Post seqadmin  
            Working...
            X