Unconfigured Ad

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts
  • yao_licr
    Junior Member
    • Jul 2014
    • 7

    substitution using sed or awk

    Hi all,
    I just start to use sed and awk. I would like to change the following fastq file

    @SRR391606.9.1 1 length=50
    AGCACCGCGAGGGCGGAGCTGCGTTCTCCTCTGCACAGCTTTCGGTGGTA
    +SRR391606.9.1 1 length=50
    @9@?<<AA:3>7A/;@?B57@6;++4=4<,=6=+++17/)1?%%%%%%%%
    @SRR391606.10.1 2 length=50
    GTTGCGTTCTCCTCAGCACAGACCCGGAGAGCACCGCGAGGGCGGAGCTG
    +SRR391606.10.1 2 length=50
    ?BB?BA?AABAB>->ABB=99<AAB++35)137<>:37=<735(2-9492

    into following fastq file:

    @SRR391606.9 1 length=50
    AGCACCGCGAGGGCGGAGCTGCGTTCTCCTCTGCACAGCTTTCGGTGGTA
    +SRR391606.9 1 length=50
    @9@?<<AA:3>7A/;@?B57@6;++4=4<,=6=+++17/)1?%%%%%%%%
    @SRR391606.10 2 length=50
    GTTGCGTTCTCCTCAGCACAGACCCGGAGAGCACCGCGAGGGCGGAGCTG
    +SRR391606.10 2 length=50
    ?BB?BA?AABAB>->ABB=99<AAB++35)137<>:37=<735(2-9492

    Basically, it is a task to remove the ".1" on every other line. As I have a huge fastq file around 30 Gb, I only put the 9th and 10th reads out of it.

    Does anyone know how to do it using sed or awk? Please also explain the script in detail as I am a beginner.

    Thanks a lot!

    Yao
  • dschika
    Member
    • Mar 2010
    • 56

    #2
    This command should do the job:

    Code:
    awk '{if ($1 ~ /SRR/) {split($0, T, ".1 "); print T[1], T[2]}  else print $0}' YOURINPUT.fastq
    Since you mentioned sed and awk I assume you know that $1 = first field, $2 = second field (fields separated by whitespaces if not defined otherwise by FS) and $0 = whole line.

    The command checks if in the first field SRR is present ("if ($1 ~ /SRR/)"). If yes, it splits the content of the first line by ".1 " and stores the result in T. In this case ".1 " can be found only once in each line with SRR. Therefore, printing of T[0] and T[1] results in the line without ".1 ".

    If the line does not contain SRR (i.e., we are at a line with sequence or quality values) print the whole line.

    Comment

    • yao_licr
      Junior Member
      • Jul 2014
      • 7

      #3
      Thanks for the detailed explanation. It works well.

      Comment

      • dschika
        Member
        • Mar 2010
        • 56

        #4
        No problem, I just realized, that I was thinking way to complicated:

        sed 's/\.1 / /g' YOURINPUT.fastq > out.fastq

        Just replace ".1 " with " " and escape the "." with a backslash.

        Comment

        • yao_licr
          Junior Member
          • Jul 2014
          • 7

          #5
          Sorry, ".1" is the pattern to be replace by " ", so why do you put an empty space after 1? I mean the empty space in "/\.1 /" .

          Thanks!

          Comment

          • yao_licr
            Junior Member
            • Jul 2014
            • 7

            #6
            I got it now, as I have .1.1 in the first read; if there is no empty space after "\.1", both of them will be replaced. In your script, ".1 " is the pattern but not ".1" .

            Thanks!

            Comment

            • dschika
              Member
              • Mar 2010
              • 56

              #7
              Correct, you're welcome!

              Comment

              Latest Articles

              Collapse

              • SEQadmin2
                Nine Things a Sample Prep Scientist Thinks About Before Sequencing
                by SEQadmin2


                I’m not a sequencing expert. I’m a purification scientist who uses NGS to evaluate workflows my group develops. With this perspective, we think about the sample first and the NGS workflow second. The sequencer is an exceptionally honest reporter, but it can only report on what you give it, so whether you get clean, interpretable data from an NGS workflow is largely determined before you begin.

                Here are nine questions we think about, in roughly the order they matter, before...
                06-18-2026, 07:11 AM
              • SEQadmin2
                From Collection to Sequencing: Why Sample Preparation and Preservation Define Sequencing Data
                by SEQadmin2


                Data variability is still an issue in sequencing technologies despite the advances in reproducibility and accuracy of these platforms. But the problem does not originate in the sequencing itself, but in the previous steps, before the sample reaches the sequencer.


                The first step is collection, followed by preservation and sample preparation for analysis. Most scientists overlook those steps, but not being careful might just be skewing the experiment’s results.
                ...
                06-02-2026, 10:05 AM

              ad_right_rmr

              Collapse

              News

              Collapse

              Topics Statistics Last Post
              Started by SEQadmin2, 06-26-2026, 11:10 AM
              0 responses
              13 views
              0 reactions
              Last Post SEQadmin2  
              Started by SEQadmin2, 06-17-2026, 06:09 AM
              0 responses
              48 views
              0 reactions
              Last Post SEQadmin2  
              Started by SEQadmin2, 06-09-2026, 11:58 AM
              0 responses
              107 views
              0 reactions
              Last Post SEQadmin2  
              Started by SEQadmin2, 06-05-2026, 10:09 AM
              0 responses
              125 views
              0 reactions
              Last Post SEQadmin2  
              Working...