Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Finding optimal reads generated on SOLiD:

    Hi,

    We're having some issues with the data coming off the SOLiD. Our people are unable to tell me which primary.* files to use and the 'latest' primary.* file doesn't always have the 'latest' write-to time stamp (unix time attribute) off the machine- nor is it the best quality. Similarly, the primary.* directories may differ across the run from sample to sample and so one sample might use primary.20101016etc on one sample and another sample of the same exp/lib/run might use primary.20101014 and both are present in each directory. These are all attributable to doing reanalysis mid-seq during a run.

    To make matters worse, the way the files are being copied is creating an entire system/network of directories that I have to go through to find the proper primary.* file for mapping. Using the wrong one results in very poor mapping. Could someone shed some light on a manual to look through or a way to ID the best-run and best primary* file to use off a run? Finally, is there a way to name these directories prior to export off the SOLiD? Thanks.

    J

  • #2
    If you disable the auto-export and export them manually using scp you can rename the files when you copy them. This doesn't solve the problem of finding which primary.* to use, though in our case it's always been the one with the largest number (most recent timestamp).

    Comment


    • #3
      Thanks, mrawlins. Maybe someone can also shed some light on the primary* issue...

      Comment


      • #4
        Sorry not quite getting you.
        But if you mean how to set a rule to find the proper F3.csfasta files to use.
        then yes I know wat you mean.

        I usually do a
        find . -iname *.stats

        only the correct reads dir with a .stats file contains the F3 that I need.

        I can't understand why they can't put the timestamp in the filename instead of the directory as well.
        such a pain
        http://kevin-gattaca.blogspot.com/

        Comment


        • #5
          Originally posted by KevinLam View Post
          I can't understand why they can't put the timestamp in the filename instead of the directory as well.
          such a pain
          Perhaps because the processing software would then have to deal with different names for F3 and R3 (or F3/F5) files. In other words, currently the files look something like:

          primary.date1/reads/run_F3_sample.csfasta
          primary.date2/reads/run_R3_sample.csfasta

          Or however you have your experiment set up. The point is that the file names (not the directories) look similar thus making it easy to switch files in and out of the pipeline. Whereas with file stamps in the file names it would look like:

          primary.date1/reads/run_F3_sample_date1.csfasta
          primary.date2/reads/run_R3_sample_date2.csfasta

          Not much of a change but perhaps something that Bioscope does not want to handle. Although, granted, making the individual files a lot more trackable.

          Anyway the above is just being "the devil's advocate". Getting back to the original question:

          Our people are unable to tell me which primary.* files to use and the 'latest' primary.* file doesn't always have the 'latest' write-to time stamp (unix time attribute) off the machine- nor is it the best quality.
          The only time you should get multiple primary.* directories for the same primer is when you doing, as you say, a reanalysis mid-run. In this case the primary.* directory with the latest date in its name should be the reanalysis one ... which is presumably the analysis you want. If it turns out to not be the one you want (e.g., because of lower quality scores) then use a previous one.

          In our (Purdue Genomics) case the only time we do a reanalysis mid-run is when something is screwed up -- i.e., we are desperately trying to save a run from a total meltdown. Fortunately the SOLiD technology is really good at being able redo parts of a run and extract data from a potentially failed run. However there have been cases where we redid a partial part of run, did not like that, re-did that part again, and ended up with even worse results. At that point we had a jumble of primary.* directories from which to choose the best of many poor results. You may be in a similar situation -- i.e., none of the results will be that great. But one of "your people" should be able to say, via looking at the heat maps and the cycle scans, which primary.* has the best chance of containing good data.

          I know that the above doesn't really answer your question "... [a] way to ID the best-run and best primary* file to use ..." but this is in part because any time a reanalysis is done then implies that there is likely to be no "best". You have a non-typical case and thus are charting a non-typical path.

          Comment


          • #6
            Originally posted by westerman View Post
            Perhaps because the processing software would then have to deal with different names for F3 and R3 (or F3/F5) files. In other words, currently the files look something like:

            primary.date1/reads/run_F3_sample.csfasta
            primary.date2/reads/run_R3_sample.csfasta

            Or however you have your experiment set up. The point is that the file names (not the directories) look similar thus making it easy to switch files in and out of the pipeline. Whereas with file stamps in the file names it would look like:

            primary.date1/reads/run_F3_sample_date1.csfasta
            primary.date2/reads/run_R3_sample_date2.csfasta

            Not much of a change but perhaps something that Bioscope does not want to handle. Although, granted, making the individual files a lot more trackable.

            Anyway the above is just being "the devil's advocate". Getting back to the original question:



            The only time you should get multiple primary.* directories for the same primer is when you doing, as you say, a reanalysis mid-run. In this case the primary.* directory with the latest date in its name should be the reanalysis one ... which is presumably the analysis you want. If it turns out to not be the one you want (e.g., because of lower quality scores) then use a previous one.

            In our (Purdue Genomics) case the only time we do a reanalysis mid-run is when something is screwed up -- i.e., we are desperately trying to save a run from a total meltdown. Fortunately the SOLiD technology is really good at being able redo parts of a run and extract data from a potentially failed run. However there have been cases where we redid a partial part of run, did not like that, re-did that part again, and ended up with even worse results. At that point we had a jumble of primary.* directories from which to choose the best of many poor results. You may be in a similar situation -- i.e., none of the results will be that great. But one of "your people" should be able to say, via looking at the heat maps and the cycle scans, which primary.* has the best chance of containing good data.

            I know that the above doesn't really answer your question "... [a] way to ID the best-run and best primary* file to use ..." but this is in part because any time a reanalysis is done then implies that there is likely to be no "best". You have a non-typical case and thus are charting a non-typical path.
            Thanks, Westernman. That's my very issue. I can't tell which primary* directory is the right one and there's no real way for me to tell unless there is no reanalysis done and only one directory to choose from. I've had cases where there were reanalyses and the latest was invalid as you said, which resulted in terrible mapping results and a waste of my time and our cluster's time. I'll talk to my SOLiD tech to try and figure out a way to analyze the heat maps and cycle scans and pass that along to our technician. That has to be the solution. Thanks a lot!

            Comment

            Latest Articles

            Collapse

            • seqadmin
              Latest Developments in Precision Medicine
              by seqadmin



              Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

              Somatic Genomics
              “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
              05-24-2024, 01:16 PM
            • seqadmin
              Recent Advances in Sequencing Analysis Tools
              by seqadmin


              The sequencing world is rapidly changing due to declining costs, enhanced accuracies, and the advent of newer, cutting-edge instruments. Equally important to these developments are improvements in sequencing analysis, a process that converts vast amounts of raw data into a comprehensible and meaningful form. This complex task requires expertise and the right analysis tools. In this article, we highlight the progress and innovation in sequencing analysis by reviewing several of the...
              05-06-2024, 07:48 AM

            ad_right_rmr

            Collapse

            News

            Collapse

            Topics Statistics Last Post
            Started by seqadmin, 05-24-2024, 07:15 AM
            0 responses
            195 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 05-23-2024, 10:28 AM
            0 responses
            217 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 05-23-2024, 07:35 AM
            0 responses
            219 views
            0 likes
            Last Post seqadmin  
            Started by seqadmin, 05-22-2024, 02:06 PM
            0 responses
            12 views
            0 likes
            Last Post seqadmin  
            Working...
            X