Adaper sequence to Read fastq

[CarnifexRex]

Hello,

Fairly new to bioinformatics and first time posting, hopefully I sound halfway intelligent

My FASTQs contain adapter sequences in the read name. This adapter sequence includes a UMI. However my postprocessing software requires the UMI to be part of the read. Is there some way of inserting the adapter sequence from the read name to the beginning of the read? I'm guessing I'll also need to insert some bogus quality scores as well.

To visualize, I need to turn this:
@M01179:478:000000000-C33YY:1:1101:17276:1520 2:N:0:CTTGTATGTATG
TTGTCGTTCCTTTCTTTTTGTCTCTTTCCTGTACCTCTAG
+
11111333@B1B11AA3A3B1A3B3BFEE333130110A0

into this:

@M01179:478:000000000-C33YY:1:1101:17276:1520 2:N:0:CTTGTATGTATG
CTTGTATGTATGTTGTCGTTCCTTTCTTTTTGTCTCTTTCCTGTACCTCTAG
+
AAAAAAAAAAAA11111333@B1B11AA3A3B1A3B3BFEE333130110A0

This would need to be a scriptable solution, and work on all reads from a High Out-put NextSeq run

Thanks in advance

[GenoMax]

Are you sure about that? Generally in case of Illumina sequencing index sequences (which are not part of actual reads) are transferred to the fastq header as a part of demultiplexing process. That is what you are probably seeing e.g. "CTTGTATGTATG".

[CarnifexRex]

That is correct, sequence CTTGTATGTATG is my index sequence. The first 6 bp (CTTGTA) is my sample barcode, the second 6 bp (TGTATG) is a molecular barcode. This second sequence is random, so changes across every read.

I need to add the entire 12 bp sequence to the beginning of my read in order to utilize the UMI portion of the adapter in my alignment and variant calling software.

[GenoMax]

This may be easier to do by obtaining the index read as a separate fastq file. You will also get real Q-scores for bases in index read when you do that. Stand-alone bcl2fastq allows one to get data in this format. I assume you may be able to do this using BaseSpace as well, if that is what you are using.

You can then use a program called "Seqkit" (and specifically option "seqkit concat" to concatenate your index read in front of the actual read).

[CarnifexRex]

Thats fantastic. This confirms by believe that there's a tool for everything.

This does exactly what I need, with one exception. It calls the entire FastQ into memory, which is a problem when the FastQ files are 20GB each.

I thought faidx indexing might help, but 1) bcl2fastq outputs gzip not bgzip files, and 2) my input files are FastQ and not FastA. I also thought of trying to stream the job by just providing a list of read names to concatenate using the seqkit common or grep command, but I couldn't get that to work.

Now I'm thinking of some combination of sort, split and bash loop but I'm still a novice at linux scripting.

Does anyone have some suggestions on how to process large files using seqkit?

Thanks