Efficiently Accessing FASTQ Files from S3 (ENA / GEO / SRA) — Best Practices?

[leofer]

Hi all,
I'm setting up an RNA-seq pipeline and currently working with GEO series [GSE210020], which points to SRA accessions like SRR20701*. I'm interested in optimizing the download and storage workflow, ideally accessing the paired-end FASTQ files directly from S3 (e.g., s3://sra-pub-run-odp/... or mirrored ENA buckets).

I’ve already scripted downloads via wget from ENA FTP and tested conversion via fasterq-dump, but:

• fasterq-dump is quite slow for large files.
• FTP works, but I'd prefer using S3 for speed and robustness.
• I'm uploading resulting FASTQs to our own S3 bucket for reuse, but would like to skip re-downloads if possible.

💡 Main questions:

1. Is there a reliable way to access FASTQ files directly from public S3 buckets (e.g., from ENA or NCBI)?
2. Are there any tools/scripts you recommend that natively use S3 as a source or destination?
3. Are there common pitfalls with region mismatches or index paths (e.g., odd directory layout under sra-pub-run-odp)?

Appreciate any pointers, workflows, or alternative tools I might be missing!

(Eventually this will be part of a scalable RNA-seq-to-DESeq2 pipeline.)

Thanks!