Stampy's test run

[lukas1848]

I am using stampy to map Illumina reads to a reference genome.

I started with a test run, using stampy's -T option. Everything went well, but I am having a hard time interpreting the output.

Is there any proper documentation to understand the results of the test run in stampy? Unfortunately, the readme does not offer much insight.

[gaffa]

Maybe you could post the output here?

[lukas1848]

ok. Here's a quote from the readme & the output:

Stampy includes test code that generates reads under an empirical read
error model and introduce SNPs van indel variants. From the output,
stampy estimates the sensitivity for mapping reads back to their
correct location, and it tabulates results by mapping posterior score
to see if these are well calibrated.

To run these tests, you need to create a genome file and corresponding
hash file, and provide one or two .fastq files with short reads and
quality scores. Stampy will use the length and qualities of these
reads, but generate sequences from random locations in the genome
provided.

You run a test by using the -S command to simulate reads, followed by a
-P command to parse the .sam output; or run the two together using -T:

stampy: Testing...
stampy: # Nucleotides (all/1/2):	0	0	0
stampy: # Variants:             	0	0	0
stampy: # Fraction:             	0.0000	0.0000	0.0000
# Score cutoff: 99999  Posterior cutoff: -99999
# Total maps: 7323580  Errors: 270107  (Non-identical starts: 293590)  Sensitivity: 96.3118174445 pct.
# Posterior >=  0  Total: 7323580 = 100.00%  Errors: 270107 = 3.69%  Sensitivity: 96.31%
# Posterior >=  1  Total: 7219839 = 98.58%  Errors: 217997 = 3.02%  Sensitivity: 95.61%
# Posterior >=  2  Total: 7195589 = 98.25%  Errors: 200550 = 2.79%  Sensitivity: 95.51%
# Posterior >=  3  Total: 7132041 = 97.38%  Errors: 160254 = 2.25%  Sensitivity: 95.20%
# Posterior >=  4  Total: 6833935 = 93.31%  Errors:  23733 = 0.35%  Sensitivity: 92.99%
# Posterior >=  5  Total: 6828977 = 93.25%  Errors:  21685 = 0.32%  Sensitivity: 92.95%
# Posterior >=  6  Total: 6828373 = 93.24%  Errors:  21331 = 0.31%  Sensitivity: 92.95%
# Posterior >=  7  Total: 6828035 = 93.23%  Errors:  21156 = 0.31%  Sensitivity: 92.94%
# Posterior >=  8  Total: 6827029 = 93.22%  Errors:  20837 = 0.31%  Sensitivity: 92.94%
# Posterior >=  9  Total: 6826486 = 93.21%  Errors:  20620 = 0.30%  Sensitivity: 92.93%
# Posterior >= 10  Total: 6826078 = 93.21%  Errors:  20423 = 0.30%  Sensitivity: 92.93%
# Posterior >= 11  Total: 6822401 = 93.16%  Errors:  20101 = 0.29%  Sensitivity: 92.88%
# Posterior >= 12  Total: 6821895 = 93.15%  Errors:  19888 = 0.29%  Sensitivity: 92.88%
# Posterior >= 13  Total: 6821195 = 93.14%  Errors:  19821 = 0.29%  Sensitivity: 92.87%
# Posterior >= 14  Total: 6820927 = 93.14%  Errors:  19757 = 0.29%  Sensitivity: 92.87%
# Posterior >= 15  Total: 6817217 = 93.09%  Errors:  19653 = 0.29%  Sensitivity: 92.82%
# Posterior >= 16  Total: 6816779 = 93.08%  Errors:  19572 = 0.29%  Sensitivity: 92.81%
# Posterior >= 17  Total: 6816293 = 93.07%  Errors:  19487 = 0.29%  Sensitivity: 92.81%
# Posterior >= 18  Total: 6815747 = 93.07%  Errors:  19383 = 0.28%  Sensitivity: 92.80%
# Posterior >= 19  Total: 6814800 = 93.05%  Errors:  19300 = 0.28%  Sensitivity: 92.79%
# Posterior >= 20  Total: 6813910 = 93.04%  Errors:  19245 = 0.28%  Sensitivity: 92.78%
# Posterior >= 21  Total: 6808801 = 92.97%  Errors:  19160 = 0.28%  Sensitivity: 92.71%
# Posterior >= 22  Total: 6805982 = 92.93%  Errors:  19022 = 0.28%  Sensitivity: 92.67%
# Posterior >= 23  Total: 6802178 = 92.88%  Errors:  18952 = 0.28%  Sensitivity: 92.62%
# Posterior >= 24  Total: 6801236 = 92.87%  Errors:  18866 = 0.28%  Sensitivity: 92.61%
# Posterior >= 25  Total: 6800745 = 92.86%  Errors:  18829 = 0.28%  Sensitivity: 92.60%
# Posterior >= 26  Total: 6796497 = 92.80%  Errors:  18766 = 0.28%  Sensitivity: 92.55%
# Posterior >= 27  Total: 6795235 = 92.79%  Errors:  18703 = 0.28%  Sensitivity: 92.53%
# Posterior >= 28  Total: 6794198 = 92.77%  Errors:  18660 = 0.27%  Sensitivity: 92.52%
# Posterior >= 29  Total: 6793250 = 92.76%  Errors:  18581 = 0.27%  Sensitivity: 92.50%
# Posterior >= 30  Total: 6784740 = 92.64%  Errors:  18460 = 0.27%  Sensitivity: 92.39%
# Posterior >= 31  Total: 6779967 = 92.58%  Errors:  18360 = 0.27%  Sensitivity: 92.33%
# Posterior >= 32  Total: 6775794 = 92.52%  Errors:  18253 = 0.27%  Sensitivity: 92.27%
# Posterior >= 33  Total: 6775513 = 92.52%  Errors:  18198 = 0.27%  Sensitivity: 92.27%
# Posterior >= 34  Total: 6774533 = 92.50%  Errors:  18138 = 0.27%  Sensitivity: 92.26%
# Posterior >= 35  Total: 6774253 = 92.50%  Errors:  18080 = 0.27%  Sensitivity: 92.25%
# Posterior >= 36  Total: 6773754 = 92.49%  Errors:  18027 = 0.27%  Sensitivity: 92.25%
# Posterior >= 37  Total: 6769285 = 92.43%  Errors:  17987 = 0.27%  Sensitivity: 92.19%
# Posterior >= 38  Total: 6768652 = 92.42%  Errors:  17935 = 0.26%  Sensitivity: 92.18%
# Posterior >= 39  Total: 6767439 = 92.41%  Errors:  17880 = 0.26%  Sensitivity: 92.16%
# Posterior >= 40  Total: 6766865 = 92.40%  Errors:  17801 = 0.26%  Sensitivity: 92.16%
# Posterior >= 41  Total: 6764996 = 92.37%  Errors:  17724 = 0.26%  Sensitivity: 92.13%
# Posterior >= 42  Total: 6763037 = 92.35%  Errors:  17464 = 0.26%  Sensitivity: 92.11%
# Posterior >= 43  Total: 6758374 = 92.28%  Errors:  17420 = 0.26%  Sensitivity: 92.04%
# Posterior >= 44  Total: 6757607 = 92.27%  Errors:  17378 = 0.26%  Sensitivity: 92.03%
# Posterior >= 45  Total: 6756677 = 92.26%  Errors:  17343 = 0.26%  Sensitivity: 92.02%
# Posterior >= 46  Total: 6755002 = 92.24%  Errors:  17305 = 0.26%  Sensitivity: 92.00%
# Posterior >= 47  Total: 6754352 = 92.23%  Errors:  17248 = 0.26%  Sensitivity: 91.99%
# Posterior >= 48  Total: 6749252 = 92.16%  Errors:  17171 = 0.25%  Sensitivity: 91.92%
# Posterior >= 49  Total: 6748495 = 92.15%  Errors:  17137 = 0.25%  Sensitivity: 91.91%
# Posterior >= 50  Total: 6747211 = 92.13%  Errors:  17085 = 0.25%  Sensitivity: 91.90%
# Posterior >= 51  Total: 6746683 = 92.12%  Errors:  17047 = 0.25%  Sensitivity: 91.89%
# Posterior >= 52  Total: 6745987 = 92.11%  Errors:  17007 = 0.25%  Sensitivity: 91.88%
# Posterior >= 53  Total: 6740141 = 92.03%  Errors:  16938 = 0.25%  Sensitivity: 91.80%
# Posterior >= 54  Total: 6738462 = 92.01%  Errors:  16851 = 0.25%  Sensitivity: 91.78%
# Posterior >= 55  Total: 6736678 = 91.99%  Errors:  16742 = 0.25%  Sensitivity: 91.76%
# Posterior >= 56  Total: 6733542 = 91.94%  Errors:  16663 = 0.25%  Sensitivity: 91.72%
# Posterior >= 57  Total: 6731124 = 91.91%  Errors:  16599 = 0.25%  Sensitivity: 91.68%
# Posterior >= 58  Total: 6728323 = 91.87%  Errors:  16545 = 0.25%  Sensitivity: 91.65%
# Posterior >= 59  Total: 6716309 = 91.71%  Errors:  16451 = 0.24%  Sensitivity: 91.48%
# Posterior >= 60  Total: 6709695 = 91.62%  Errors:  16352 = 0.24%  Sensitivity: 91.39%
# Posterior >= 61  Total: 6705435 = 91.56%  Errors:  16284 = 0.24%  Sensitivity: 91.34%
# Posterior >= 62  Total: 6704302 = 91.54%  Errors:  16224 = 0.24%  Sensitivity: 91.32%
# Posterior >= 63  Total: 6701511 = 91.51%  Errors:  16128 = 0.24%  Sensitivity: 91.29%
# Posterior >= 64  Total: 6693605 = 91.40%  Errors:  16038 = 0.24%  Sensitivity: 91.18%
# Posterior >= 65  Total: 6685131 = 91.28%  Errors:  15865 = 0.24%  Sensitivity: 91.07%
# Posterior >= 66  Total: 6619912 = 90.39%  Errors:  15032 = 0.23%  Sensitivity: 90.19%
# Posterior >= 67  Total: 6619212 = 90.38%  Errors:  14981 = 0.23%  Sensitivity: 90.18%
# Posterior >= 68  Total: 6618528 = 90.37%  Errors:  14919 = 0.23%  Sensitivity: 90.17%
# Posterior >= 69  Total: 6617656 = 90.36%  Errors:  14866 = 0.22%  Sensitivity: 90.16%
# Posterior >= 70  Total: 6610272 = 90.26%  Errors:  14761 = 0.22%  Sensitivity: 90.06%
# Posterior >= 71  Total: 6608222 = 90.23%  Errors:  14668 = 0.22%  Sensitivity: 90.03%
# Posterior >= 72  Total: 6607269 = 90.22%  Errors:  14600 = 0.22%  Sensitivity: 90.02%
# Posterior >= 73  Total: 6606014 = 90.20%  Errors:  14494 = 0.22%  Sensitivity: 90.00%
# Posterior >= 74  Total: 6604521 = 90.18%  Errors:  14338 = 0.22%  Sensitivity: 89.99%
# Posterior >= 75  Total: 6596798 = 90.08%  Errors:  14068 = 0.21%  Sensitivity: 89.88%
# Posterior >= 76  Total: 6594287 = 90.04%  Errors:  13966 = 0.21%  Sensitivity: 89.85%
# Posterior >= 77  Total: 6592886 = 90.02%  Errors:  13862 = 0.21%  Sensitivity: 89.83%
# Posterior >= 78  Total: 6590794 = 89.99%  Errors:  13737 = 0.21%  Sensitivity: 89.81%
# Posterior >= 79  Total: 6588434 = 89.96%  Errors:  13625 = 0.21%  Sensitivity: 89.78%
# Posterior >= 80  Total: 6586624 = 89.94%  Errors:  13496 = 0.20%  Sensitivity: 89.75%
# Posterior >= 81  Total: 6579625 = 89.84%  Errors:  13390 = 0.20%  Sensitivity: 89.66%
# Posterior >= 82  Total: 6577538 = 89.81%  Errors:  13339 = 0.20%  Sensitivity: 89.63%
# Posterior >= 83  Total: 6575800 = 89.79%  Errors:  13285 = 0.20%  Sensitivity: 89.61%
# Posterior >= 84  Total: 6573734 = 89.76%  Errors:  13241 = 0.20%  Sensitivity: 89.58%
# Posterior >= 85  Total: 6570961 = 89.72%  Errors:  13180 = 0.20%  Sensitivity: 89.54%
# Posterior >= 86  Total: 6561938 = 89.60%  Errors:  13111 = 0.20%  Sensitivity: 89.42%
# Posterior >= 87  Total: 6557819 = 89.54%  Errors:  13052 = 0.20%  Sensitivity: 89.37%
# Posterior >= 88  Total: 6552064 = 89.47%  Errors:  12984 = 0.20%  Sensitivity: 89.29%
# Posterior >= 89  Total: 6544926 = 89.37%  Errors:  12902 = 0.20%  Sensitivity: 89.19%
# Posterior >= 90  Total: 6532951 = 89.20%  Errors:  12827 = 0.20%  Sensitivity: 89.03%
# Posterior >= 91  Total: 6529083 = 89.15%  Errors:  12767 = 0.20%  Sensitivity: 88.98%
# Posterior >= 92  Total: 6527110 = 89.12%  Errors:  12636 = 0.19%  Sensitivity: 88.95%
# Posterior >= 93  Total: 6526026 = 89.11%  Errors:  12576 = 0.19%  Sensitivity: 88.94%
# Posterior >= 94  Total: 6524390 = 89.09%  Errors:  12483 = 0.19%  Sensitivity: 88.92%
# Posterior >= 95  Total: 6517356 = 88.99%  Errors:  12396 = 0.19%  Sensitivity: 88.82%
# Posterior >= 96  Total: 6514622 = 88.95%  Errors:  12300 = 0.19%  Sensitivity: 88.79%
# Posterior >= 97  Total: 6409828 = 87.52%  Errors:   8440 = 0.13%  Sensitivity: 87.41%
# Posterior >= 98  Total: 6402982 = 87.43%  Errors:   8394 = 0.13%  Sensitivity: 87.32%
# Total no-indel maps: 7323580  Mapcorrect: 7053473  indelinferred: 72447  FP rate: 0.989229311348 pct.

Length	Pos	FPR	mapSens	sizeSens	posSens
0	-1	0.9892	0.00	0.00	0.00
-1	-1	0.0024	0.00	0.00	0.00
-2	-1	0.0006	0.00	0.00	0.00
-3	-1	0.0003	0.00	0.00	0.00
-4	-1	0.0002	0.00	0.00	0.00
-5	-1	0.0001	0.00	0.00	0.00
-6	-1	0.0002	0.00	0.00	0.00
-7	-1	0.0001	0.00	0.00	0.00
-8	-1	0.0001	0.00	0.00	0.00
-9	-1	0.0001	0.00	0.00	0.00
-10	-1	0.0000	0.00	0.00	0.00
-11	-1	0.0000	0.00	0.00	0.00
-13	-1	0.0000	0.00	0.00	0.00
-16	-1	0.0000	0.00	0.00	0.00
-19	-1	0.0000	0.00	0.00	0.00
-21	-1	0.0000	0.00	0.00	0.00
-22	-1	0.0000	0.00	0.00	0.00
-24	-1	0.0000	0.00	0.00	0.00
1	-1	0.0126	0.00	0.00	0.00
2	-1	0.0131	0.00	0.00	0.00
3	-1	0.0127	0.00	0.00	0.00
4	-1	0.0128	0.00	0.00	0.00
5	-1	0.0126	0.00	0.00	0.00
6	-1	0.0132	0.00	0.00	0.00
7	-1	0.0143	0.00	0.00	0.00
8	-1	0.0140	0.00	0.00	0.00
9	-1	0.0146	0.00	0.00	0.00
10	-1	0.0149	0.00	0.00	0.00
11	-1	0.0156	0.00	0.00	0.00
12	-1	0.0162	0.00	0.00	0.00
13	-1	0.0134	0.00	0.00	0.00
14	-1	0.0145	0.00	0.00	0.00
15	-1	0.0160	0.00	0.00	0.00
16	-1	0.0154	0.00	0.00	0.00
17	-1	0.0154	0.00	0.00	0.00
18	-1	0.0147	0.00	0.00	0.00
19	-1	0.0145	0.00	0.00	0.00
20	-1	0.0132	0.00	0.00	0.00
21	-1	0.0120	0.00	0.00	0.00
22	-1	0.0120	0.00	0.00	0.00
23	-1	0.0105	0.00	0.00	0.00
24	-1	0.0102	0.00	0.00	0.00
25	-1	0.0104	0.00	0.00	0.00
26	-1	0.0099	0.00	0.00	0.00
27	-1	0.0097	0.00	0.00	0.00
28	-1	0.0098	0.00	0.00	0.00
29	-1	0.0104	0.00	0.00	0.00
30	-1	0.0099	0.00	0.00	0.00
31	-1	0.0103	0.00	0.00	0.00
32	-1	0.0097	0.00	0.00	0.00
33	-1	0.0098	0.00	0.00	0.00
34	-1	0.0101	0.00	0.00	0.00
35	-1	0.0102	0.00	0.00	0.00
36	-1	0.0103	0.00	0.00	0.00
37	-1	0.0103	0.00	0.00	0.00
38	-1	0.0101	0.00	0.00	0.00
39	-1	0.0103	0.00	0.00	0.00
40	-1	0.0100	0.00	0.00	0.00
41	-1	0.0100	0.00	0.00	0.00
42	-1	0.0098	0.00	0.00	0.00
43	-1	0.0096	0.00	0.00	0.00
44	-1	0.0100	0.00	0.00	0.00
45	-1	0.0093	0.00	0.00	0.00
46	-1	0.0102	0.00	0.00	0.00
47	-1	0.0100	0.00	0.00	0.00
48	-1	0.0107	0.00	0.00	0.00
49	-1	0.0098	0.00	0.00	0.00
50	-1	0.4060	0.00	0.00	0.00
0	0	0.9394	0.00	0.00	0.00
0	1	0.0260	0.00	0.00	0.00
0	2	0.0104	0.00	0.00	0.00
0	3	0.0045	0.00	0.00	0.00
0	4	0.0023	0.00	0.00	0.00
0	5	0.0013	0.00	0.00	0.00
0	6	0.0007	0.00	0.00	0.00
0	7	0.0004	0.00	0.00	0.00
0	8	0.0002	0.00	0.00	0.00
0	9	0.0002	0.00	0.00	0.00
0	10	0.0002	0.00	0.00	0.00
0	11	0.0001	0.00	0.00	0.00
0	12	0.0002	0.00	0.00	0.00
0	13	0.0001	0.00	0.00	0.00
0	14	0.0001	0.00	0.00	0.00
0	15	0.0000	0.00	0.00	0.00
0	16	0.0002	0.00	0.00	0.00
0	17	0.0002	0.00	0.00	0.00
0	18	0.0001	0.00	0.00	0.00
0	19	0.0001	0.00	0.00	0.00
0	20	0.0001	0.00	0.00	0.00
0	21	0.0001	0.00	0.00	0.00
0	22	0.0001	0.00	0.00	0.00
0	23	0.0001	0.00	0.00	0.00
0	24	0.0002	0.00	0.00	0.00
0	25	0.0001	0.00	0.00	0.00
0	26	0.0000	0.00	0.00	0.00
0	27	0.0001	0.00	0.00	0.00
0	28	0.0001	0.00	0.00	0.00
0	29	0.0016	0.00	0.00	0.00

# Estimated and empirical mapping quality statistics
Estimated	Empirical	CI95_low	CI95_high	Frequency	Mismaps	Outside_CI
0	3	3	3	103741	52110	+
1	1	1	1	24250	17447	
2	2	2	2	63548	40296	
3	3	3	3	298106	136521	
4	4	4	4	4958	2048	
5	2	2	3	604	354	*
6	3	2	4	338	175	*
7	5	5	5	1006	319	*
8	4	3	5	543	217	*
9	3	3	4	408	197	*
10	11	10	11	3677	322	
11	4	3	4	506	213	*
12	10	9	11	700	67	*
13	6	5	7	268	64	*
14	15	15	16	3710	104	+
15	7	6	8	438	81	*
16	8	7	9	486	85	*
17	7	6	8	546	104	*
18	11	10	12	947	83	*
19	12	11	13	890	55	*
20	18	17	19	5109	85	*
21	13	12	14	2819	138	*
22	17	16	18	3804	70	*
23	10	10	11	942	86	*
24	11	10	13	491	37	*
25	18	17	19	4248	63	*
26	13	12	14	1262	63	*
27	14	13	15	1037	43	*
28	11	10	12	948	79	*
29	18	18	19	8510	121	*
30	17	16	18	4773	100	*
31	16	15	17	4173	107	*
32	7	6	8	281	55	*
33	12	11	13	980	60	*
34	7	6	8	280	58	*
35	10	9	11	499	53	*
36	20	19	22	4469	40	*
37	11	10	12	633	52	*
38	13	12	15	1213	55	*
39	9	8	10	574	79	*
40	14	13	15	1869	77	*
41	9	8	9	1959	260	*
42	20	19	22	4663	44	*
43	13	11	14	767	42	*
44	14	13	16	930	35	*
45	16	15	18	1675	38	*
46	11	9	12	650	57	*
47	18	17	19	5100	77	*
48	13	12	15	757	34	*
49	14	13	15	1284	52	*
50	11	10	13	528	38	*
51	12	11	14	696	40	*
52	19	18	20	5846	69	*
53	13	12	14	1679	87	*
54	12	11	13	1784	109	*
55	16	15	17	3136	79	*
56	16	15	17	2418	64	*
57	17	16	18	2801	54	*
58	21	20	22	12014	94	*
59	18	17	19	6614	99	*
60	18	17	19	4260	68	*
61	13	12	14	1133	60	*
62	15	14	16	2791	96	*
63	19	19	20	7906	90	*
64	17	16	18	8474	173	*
65	19	19	19	65219	833	*
66	11	10	13	700	51	*
67	10	9	12	684	62	*
68	12	11	13	872	53	*
69	18	18	19	7384	105	*
70	13	13	14	2050	93	*
71	11	10	13	953	68	*
72	11	10	12	1255	106	*
73	10	9	11	1493	156	*
74	15	14	15	7723	270	*
75	14	13	15	2511	102	*
76	11	10	12	1401	104	*
77	12	11	13	2092	125	*
78	13	12	14	2360	112	*
79	11	11	12	1810	129	*
80	18	17	19	6999	106	*
81	16	15	17	2087	51	*
82	15	14	16	1738	54	*
83	17	15	18	2066	44	*
84	17	16	18	2773	61	*
85	21	20	22	9023	69	*
86	18	17	20	4119	59	*
87	19	18	20	5755	68	*
88	19	18	20	7138	82	*
89	22	21	23	11975	75	*
90	18	17	19	3868	60	*
91	12	11	13	1973	131	*
92	13	12	14	1084	60	*
93	12	12	13	1636	93	*
94	19	18	20	7034	87	*
95	15	14	15	2734	96	*
96	14	14	14	104794	3860	*
97	22	21	23	6846	46	*
98	18	17	19	8112	123	*
99	29	29	29	6394870	8271	*
stampy: Done

[lukas1848]

Sorry to bump this thread, but I am still wondering if I am supposed to adjust my settings for running stampy based on this test run.
Since stampy uses simulated reads with fake sequencing errors and SNPs, but real phred scores, one should be able to get a good idea of what settings should best match my reads. The thing is, I don't know where this info is hidden in the test run...

[Rocketknight]

It's unlikely that you need to adjust the default Stampy settings without a good reason to do so. It's possible if you're doing some kind of unusual alignment (such as reads from one species mapped to the genome of a different but related species) that you'll need to tweak Stampy's settings, but the defaults should be fine for most cases.

[lukas1848]

Thanks for your reply.

I am in fact mapping reads from a divergent population of the same species to my genome. Therefore I think I need to change the substitutionrate option. The default is 0.001 but I think this does not suit my data. That's why I was hoping to get some info out of the test run.

I wonder if there's a proper way of assessing the substitution rate from my reads without having much background about the genetic diversity in my species.

[Rocketknight]

Unfortunately, for the test run you'll need to set the substitution rate for the generated data, so the test run won't actually tell you much about your real-world data - what you get will depend totally on what value you key in for the test substitution rate.

One way to tell how divergent your sequence data is is to run Stampy with the recommended settings and take note of how many reads aren't mapped by BWA. If you're seeing >80% of the reads mapped by BWA then the default settings are probably fine.