[shocker8786]

I retried the code and was still not successful with one of my datasets. I am sure that the code I am using is the same for both datasets (I have been copying and pasting the code for both datasets). Below is the code and the results for the commands you asked me to enter for the dataset that did not work properly.

> cuff.res=read.delim(file="gene_exp.diff", sep="\t")
> cuff.fc=cuff.res$log2.fold_change.
> gnames=cuff.res$gene
> sel=gnames!="-"
> gnames=as.character(gnames[sel])
> cuff.fc=cuff.fc[sel]
> names(cuff.fc)=gnames
> gnames.eg=pathview::id2eg(gnames, category ="symbol", org="Ss")
Loading required package: org.Ss.eg.db

> sel2=gnames.eg[,2]>""
> cuff.fc=cuff.fc[sel2]
> names(cuff.fc)=gnames.eg[sel2,2]
> range(cuff.fc)
[1] -Inf Inf
> cuff.fc[cuff.fc>10]=10
> cuff.fc[cuff.fc< -10]=-10
> exp.fc=cuff.fc
> out.suffix="cuff"
> require(gage)
> kg.ssc=kegg.gsets("ssc")
> kegg.gs=kg.ssc$kg.sets[kg.ssc$sigmet.idx]
> fc.kegg.p <- gage(exp.fc, gsets = kegg.gs, ref = NULL, samp = NULL)
> sel <- fc.kegg.p$greater[, "q.val"] < 0.1 &
+ !is.na(fc.kegg.p$greater[, "q.val"])
> path.ids <- rownames(fc.kegg.p$greater)[sel]
> sel.l <- fc.kegg.p$less[, "q.val"] < 0.1 &
+ !is.na(fc.kegg.p$less[, "q.val"])
> path.ids.l <- rownames(fc.kegg.p$less)[sel.l]
> path.ids2 <- substr(c(path.ids, path.ids.l), 1, 8)
> require(pathview)
> pv.out.list <- sapply(path.ids2[1:3], function(pid) pathview(
+ gene.data = exp.fc, pathway.id = pid,
+ species = "ssc", out.suffix=out.suffix))
Start tag expected, '<' not found
Parsing ./sscNA.xml file failed, please check the file!
Start tag expected, '<' not found
Parsing ./sscNA.xml file failed, please check the file!
Start tag expected, '<' not found
Parsing ./sscNA.xml file failed, please check the file!
> str(cuff.fc)
Named num [1:7434] 0.6134 0 0.3767 0.0177 -2.1022 ...
- attr(*, "names")= chr [1:7434] "100620481" "100512748" "100513321" "100154202" ...
> head(cuff.fc)
100620481 100512748 100513321 100154202 733660 100153485
0.6133590 0.0000000 0.3766890 0.0177489 -2.1022400 -0.2612780
> lapply(kegg.gs[1:3], head, 3)
$`ssc00970 Aminoacyl-tRNA biosynthesis`
[1] "100144457" "100153423" "100155115"

$`ssc02010 ABC transporters`
[1] "100048963" "100127449" "100144586"

$`ssc03008 Ribosome biogenesis in eukaryotes`
[1] "100151786" "100151831" "100152658"

> head(fc.kegg.p$greater)
p.geomean stat.mean p.val
ssc04064 NF-kappa B signaling pathway 0.02469658 1.990928 0.02469658
ssc04620 Toll-like receptor signaling pathway 0.03526210 1.828538 0.03526210
ssc04066 HIF-1 signaling pathway 0.04658115 1.693048 0.04658115
ssc04640 Hematopoietic cell lineage 0.04929811 1.670237 0.04929811
ssc04670 Leukocyte transendothelial migration 0.04995310 1.657766 0.04995310
ssc04610 Complement and coagulation cascades 0.07305357 1.467898 0.07305357
q.val set.size exp1
ssc04064 NF-kappa B signaling pathway 0.8234987 53 0.02469658
ssc04620 Toll-like receptor signaling pathway 0.8234987 58 0.03526210
ssc04066 HIF-1 signaling pathway 0.8234987 59 0.04658115
ssc04640 Hematopoietic cell lineage 0.8234987 49 0.04929811
ssc04670 Leukocyte transendothelial migration 0.8234987 63 0.04995310
ssc04610 Complement and coagulation cascades 0.8234987 44 0.07305357

Below are the results for the dataset that did work properly:

Working in directory /srv/mds01/shared/Epigenetics/Rnaseq/kyle_results/TJT/tophat_expression/cuffdiff
Writing image file ssc04151.cuff.png
Working in directory /srv/mds01/shared/Epigenetics/Rnaseq/kyle_results/TJT/tophat_expression/cuffdiff
Writing image file ssc04010.cuff.png
Working in directory /srv/mds01/shared/Epigenetics/Rnaseq/kyle_results/TJT/tophat_expression/cuffdiff
Writing image file ssc04080.cuff.png
> str(cuff.fc)
Named num [1:344025] 1.279 0 0.155 0.806 -0.958 ...
- attr(*, "names")= chr [1:344025] "100620481" "100512748" "100513321" "100154202" ...
> head(cuff.fc)
100620481 100512748 100513321 100154202 100156632 733660
1.278660 0.000000 0.154690 0.806045 -0.958121 -0.103950
> lapply(kegg.gs[1:3], head, 3)
$`ssc00970 Aminoacyl-tRNA biosynthesis`
[1] "100144457" "100153423" "100155115"

$`ssc02010 ABC transporters`
[1] "100048963" "100127449" "100144586"

$`ssc03008 Ribosome biogenesis in eukaryotes`
[1] "100151786" "100151831" "100152658"

> head(fc.kegg.p$greater)
p.geomean stat.mean p.val
ssc00770 Pantothenate and CoA biosynthesis 0.9995680 -3.988628 0.9995680
ssc00630 Glyoxylate and dicarboxylate metabolism 0.9995962 -3.942065 0.9995962
ssc00052 Galactose metabolism 0.9999012 -4.281375 0.9999012
ssc04740 Olfactory transduction 0.9999254 -4.576407 0.9999254
ssc00100 Steroid biosynthesis 0.9999490 -4.852319 0.9999490
ssc00970 Aminoacyl-tRNA biosynthesis 0.9999580 -4.829070 0.9999580
q.val set.size exp1
ssc00770 Pantothenate and CoA biosynthesis 1 10 0.9995680
ssc00630 Glyoxylate and dicarboxylate metabolism 1 11 0.9995962
ssc00052 Galactose metabolism 1 15 0.9999012
ssc04740 Olfactory transduction 1 12 0.9999254
ssc00100 Steroid biosynthesis 1 11 0.9999490
ssc00970 Aminoacyl-tRNA biosynthesis 1 12 0.9999580

[sindrle]

Couldnt get gnCnt to work, I read in HTseq files instead like this:

fls <- list.files(path="~/RNAseq/02_MG-myocytes/MG_Myocytes_HTSeq/A1A2pT2D",pattern="*.txt", full.names =T)

#tab separated values with a header
datalist = lapply(fls, function(x)read.table(x, header=F, colClasses=c("NULL", "numeric"), nrow = nrow(read.table(x, header = F)) - 5))

#same header/columns for all files
datafr = do.call("cbind", datalist)

#column names
colnames(datafr) <- c("D102-A1","D102-A2","D104-A1","D104-A2","D117-A1","D117-A2","D121-A1","D121-A2","D153-A1","D153-A2","D155-A1","D155-A2","D161-A1","D161-A2","D162-A1","D162-A2","D167-A1","D167-A2","D173-A1","D173-A2","D176-A1","D176-A2","D177-A1","D177-A2","D179-A1","D179-A2")

#row names
x <- read.table(file = "~/RNAseq/02_MG-myocytes/MG_Myocytes_HTSeq/A1A2pT2D/D102-A1.txt", header=F, colClasses=c( "character", "NULL"), nrow = nrow(read.table(file = "~/RNAseq/02_MG-myocytes/MG_Myocytes_HTSeq/A1A2pT2D/D102-A1.txt", header = F)) - 5)

rownames(datafr) <- x$V1

hnrnp.cnts <- datafr

[bigmw]

Your kegg.gs has no problem, and it is the same for both datasets:
> lapply(kegg.gs[1:3], head, 3)
$`ssc00970 Aminoacyl-tRNA biosynthesis`
[1] "100144457" "100153423" "100155115"
…

The problem resides in yoru expression data. In your problematic dataset (let’s call it dataset 1), you have only 7434 entries (not exactly genes):
> str(cuff.fc)
Named num [1:7434] 0.6134 0 0.3767 0.0177 -2.1022 ...
…

In your good dataset 2, you have 344025 entries:
> str(cuff.fc)
Named num [1:344025] 1.279 0 0.155 0.806 -0.958 ...
…

Pig has 57700 genes in total based on NCBI data. Assume your dataset 2 has a full or decent coverage on pig genes, each pig gene maps to about 6 entries in this dataset. Assume the same level of redundancy, there are only ~ 1200 unique genes in your dataset 1. In other words, this is likely a sublist of selected or significant genes. Pathway or gene set analysis like GAGE requires a comprehensive and balance coverage of genes for your research speices, e.g. the full list of genes measured in a microarray or RNA-seq study, usually in the order of several thousand or tens of thousands of genes.
Therefore, very likely you gage analysis of dataset 1 doesn’t yield any significant pathways due to the biased preslected/short gene list with only ~ 1200 genes. Hence you get NA pathway IDs when you select significant pathways. Your dataset 2 does give you some sensible results because you seem to have a good and balanced coverage of pig genes there.

However, you should really make sure each gene ID has only 1 expression value when doing gene set or pathway analysis because genes are treated as independent variables. In the GAGE/Pathview native workflow or joint workflow with DESeq/DESeq2/edgeR/limma, we explicitly mapped reads to Entrez Genes and summarized expression counts per genes. But your data were mapped and processed by Cufflinks likely to some transcript sequences like RefSeq mRNA’s (with Entrez Gene annotation). You should summarize the expression data for multiple transcripts of the same gene into 1 unified expression level, and then conduct pathway analysis. We describe how to do so in a secondary gage tutorial, “Gene set and data preparation”, please check Section 5-gene or transcript ID conversion:

http://bioconductor.org/packages/release/bioc/vignettes/gage/inst/doc/dataPrep.pdf

I retried the code and was still not successful with one of my datasets. I am sure that the code I am using is the same for both datasets (I have been copying and pasting the code for both datasets). Below is the code and the results for the commands you asked me to enter for the dataset that did not work properly.

> cuff.res=read.delim(file="gene_exp.diff", sep="\t")
> cuff.fc=cuff.res$log2.fold_change.
> gnames=cuff.res$gene
> sel=gnames!="-"
> gnames=as.character(gnames[sel])
> cuff.fc=cuff.fc[sel]
> names(cuff.fc)=gnames
> gnames.eg=pathview::id2eg(gnames, category ="symbol", org="Ss")
Loading required package: org.Ss.eg.db

> sel2=gnames.eg[,2]>""
> cuff.fc=cuff.fc[sel2]
> names(cuff.fc)=gnames.eg[sel2,2]
> range(cuff.fc)
[1] -Inf Inf
> cuff.fc[cuff.fc>10]=10
> cuff.fc[cuff.fc< -10]=-10
> exp.fc=cuff.fc
> out.suffix="cuff"
> require(gage)
> kg.ssc=kegg.gsets("ssc")
> kegg.gs=kg.ssc$kg.sets[kg.ssc$sigmet.idx]
> fc.kegg.p <- gage(exp.fc, gsets = kegg.gs, ref = NULL, samp = NULL)
> sel <- fc.kegg.p$greater[, "q.val"] < 0.1 &
+ !is.na(fc.kegg.p$greater[, "q.val"])
> path.ids <- rownames(fc.kegg.p$greater)[sel]
> sel.l <- fc.kegg.p$less[, "q.val"] < 0.1 &
+ !is.na(fc.kegg.p$less[, "q.val"])
> path.ids.l <- rownames(fc.kegg.p$less)[sel.l]
> path.ids2 <- substr(c(path.ids, path.ids.l), 1, 8)
> require(pathview)
> pv.out.list <- sapply(path.ids2[1:3], function(pid) pathview(
+ gene.data = exp.fc, pathway.id = pid,
+ species = "ssc", out.suffix=out.suffix))
Start tag expected, '<' not found
Parsing ./sscNA.xml file failed, please check the file!
Start tag expected, '<' not found
Parsing ./sscNA.xml file failed, please check the file!
Start tag expected, '<' not found
Parsing ./sscNA.xml file failed, please check the file!
> str(cuff.fc)
Named num [1:7434] 0.6134 0 0.3767 0.0177 -2.1022 ...
- attr(*, "names")= chr [1:7434] "100620481" "100512748" "100513321" "100154202" ...
> head(cuff.fc)
100620481 100512748 100513321 100154202 733660 100153485
0.6133590 0.0000000 0.3766890 0.0177489 -2.1022400 -0.2612780
> lapply(kegg.gs[1:3], head, 3)
$`ssc00970 Aminoacyl-tRNA biosynthesis`
[1] "100144457" "100153423" "100155115"

$`ssc02010 ABC transporters`
[1] "100048963" "100127449" "100144586"

$`ssc03008 Ribosome biogenesis in eukaryotes`
[1] "100151786" "100151831" "100152658"

> head(fc.kegg.p$greater)
p.geomean stat.mean p.val
ssc04064 NF-kappa B signaling pathway 0.02469658 1.990928 0.02469658
ssc04620 Toll-like receptor signaling pathway 0.03526210 1.828538 0.03526210
ssc04066 HIF-1 signaling pathway 0.04658115 1.693048 0.04658115
ssc04640 Hematopoietic cell lineage 0.04929811 1.670237 0.04929811
ssc04670 Leukocyte transendothelial migration 0.04995310 1.657766 0.04995310
ssc04610 Complement and coagulation cascades 0.07305357 1.467898 0.07305357
q.val set.size exp1
ssc04064 NF-kappa B signaling pathway 0.8234987 53 0.02469658
ssc04620 Toll-like receptor signaling pathway 0.8234987 58 0.03526210
ssc04066 HIF-1 signaling pathway 0.8234987 59 0.04658115
ssc04640 Hematopoietic cell lineage 0.8234987 49 0.04929811
ssc04670 Leukocyte transendothelial migration 0.8234987 63 0.04995310
ssc04610 Complement and coagulation cascades 0.8234987 44 0.07305357

Below are the results for the dataset that did work properly:

Working in directory /srv/mds01/shared/Epigenetics/Rnaseq/kyle_results/TJT/tophat_expression/cuffdiff
Writing image file ssc04151.cuff.png
Working in directory /srv/mds01/shared/Epigenetics/Rnaseq/kyle_results/TJT/tophat_expression/cuffdiff
Writing image file ssc04010.cuff.png
Working in directory /srv/mds01/shared/Epigenetics/Rnaseq/kyle_results/TJT/tophat_expression/cuffdiff
Writing image file ssc04080.cuff.png
> str(cuff.fc)
Named num [1:344025] 1.279 0 0.155 0.806 -0.958 ...
- attr(*, "names")= chr [1:344025] "100620481" "100512748" "100513321" "100154202" ...
> head(cuff.fc)
100620481 100512748 100513321 100154202 100156632 733660
1.278660 0.000000 0.154690 0.806045 -0.958121 -0.103950
> lapply(kegg.gs[1:3], head, 3)
$`ssc00970 Aminoacyl-tRNA biosynthesis`
[1] "100144457" "100153423" "100155115"

$`ssc02010 ABC transporters`
[1] "100048963" "100127449" "100144586"

$`ssc03008 Ribosome biogenesis in eukaryotes`
[1] "100151786" "100151831" "100152658"

> head(fc.kegg.p$greater)
p.geomean stat.mean p.val
ssc00770 Pantothenate and CoA biosynthesis 0.9995680 -3.988628 0.9995680
ssc00630 Glyoxylate and dicarboxylate metabolism 0.9995962 -3.942065 0.9995962
ssc00052 Galactose metabolism 0.9999012 -4.281375 0.9999012
ssc04740 Olfactory transduction 0.9999254 -4.576407 0.9999254
ssc00100 Steroid biosynthesis 0.9999490 -4.852319 0.9999490
ssc00970 Aminoacyl-tRNA biosynthesis 0.9999580 -4.829070 0.9999580
q.val set.size exp1
ssc00770 Pantothenate and CoA biosynthesis 1 10 0.9995680
ssc00630 Glyoxylate and dicarboxylate metabolism 1 11 0.9995962
ssc00052 Galactose metabolism 1 15 0.9999012
ssc04740 Olfactory transduction 1 12 0.9999254
ssc00100 Steroid biosynthesis 1 11 0.9999490
ssc00970 Aminoacyl-tRNA biosynthesis 1 12 0.9999580

[shocker8786]

Thank you very much for that explanation, this is starting to make more sense to me now. I will make the necessary changes and retry. Thanks again!

[sindrle]

Im a retard. Forgot to convert kegg.gs to gene symbols:

data(egSymb)

kegg.gs.sym<-lapply(kegg.gs, eg2sym)

[bigmw]

sindrle,
I couldn’t find your original problem, but if I remember correctly, your summarizeOverlaps step didn’t work. And you did something like:
…
flag <- scanBamFlag(isNotPrimaryRead=FALSE, isProperPair=TRUE)
param <- ScanBamParam(flag=flag)
gnCnt <- summarizeOverlaps(exByGn, bamfls, mode="Union", ignore.strand=TRUE, single.end=TRUE, param=param)
…

I guess you have single end data, so try:
flag <- scanBamFlag(isNotPrimaryRead=FALSE, isProperPair=NA)
The following flag line is for paired end data:
flag <- scanBamFlag(isNotPrimaryRead=FALSE, isProperPair=TRUE)

Check help info for details:
?scanBamFlag

[sindrle]

Ok, thanks!
I was thinking about the same, but I figured since I have already ran HTseq I just used the results from there.

Good to know until next time.

[shriram]

Hi,
I was looking at the expression data from pathview object and original data supplied to pathview, I found discrepancies in the two values:

Original count data (human data from reference manual):
> cnts.d["120", ]
ERR127302 ERR127303 ERR127304 ERR127305
0.277 0.577 0.441 0.021

output from Pathview object for human data from reference manual
>pv.out.list[1]
ERR127302 ERR127303 ERR127304 ERR127305
0.3952 0.8842 1.628 1.1983

Does pathview transforms the original data before plotting on the kegg pathways?

THanks,

[bigmw]

If you are look for the gene entry"120" underpv.out.list[[1]]$plot.data.gene, it may or may not be different from the original data, cnts.d, because multiple genes may be mapped to the same nodes in a KEGG pathway. What you found in $plot.data.gene is the node level summary instead of single gene data. Each node there is labeled or named after the most representative member gene.

Please check the pathview tutorial (page 7) and documentations for more details:

http://bioconductor.org/packages/release/bioc/html/pathview.html

http://bioconductor.org/packages/release/bioc/vignettes/pathview/inst/doc/pathview.pdf

[shriram]

Fold change ploting

Thank you for your reply, now I understand why there are differences at node level data and actual fold changes.
However when I want to compare the fold change that I see in the input data and on the KEGG pathway it doesn't correlate well.
e.g.
pv.out.list[[1]]
Gene A:
T1 T2 T3 T4 T5
0.66 4.1079 0.830 3.278 2.71

input fold change values:
Gene A
-1.2 0.7 -0.14 -0.48 -0.78

Color coding for this gene on pathway node:

#EF3030 #FF0000 #FF0000 #FF0000 #FF0000

which doesn't reflect the trend seen in the input data.

When I change node.sum="median" it little bit shows same trend as input fold changes.
pv.out.list[[1]]
-1.15 0.7626 0.000 1.387 1.534
#00FF00 #EF3030 #BEBEBE #FF0000 #FF0000

It is bit confusing for me.


Thanks

[bigmw]

Again, that’s because in this pathway, node labelled “Gene A” includes gene(s) other than “Gene A” but with similar function. It is a summary of fold changes for all genes mapped to this node, should you be surprised that it is different from fold change of “Gene A” alone?
You may use node.sum argument to control the the node summary is calculated, mean, median, max etc.

[shriram]

Thanks for the quick reply.
In above example GeneA is the only gene [shown green in original kegg] on that node for that species as other genes on the node are not present in the given species.
Thanks,
Shriram

[sindrle]

Im having problems with Pathview. I can only get native KEGG, the kegg.native=F does not work.

Also the native KEGG only has green color, not red (up regulated) and green (down regulated).

Why am I having these two problems?

Native KEGG
# pv.out.list <- sapply(path.ids2, function(pid) pathview(gene.data = d[,
# 1:2], pathway.id = pid, species = "hsa", kegg.dir = "~/RNAseq/13_Acute-Changes/13_GAGE_native_A1A2/A1A2pT2D/Pathview"))

Graphviz view
# pv.out.list <- sapply(path.ids2, function(pid) pathview(gene.data = d[,
# 1:2], pathway.id = pid, species = "hsa", kegg.native=F,
# sign.pos="bottomleft", kegg.dir = "~/RNAseq/13_Acute-Changes/13_GAGE_native_A1A2/A1A2pT2D/Pathview"))


# > sessionInfo()
# R version 3.0.3 (2013-09-25)
# Platform: x86_64-apple-darwin10.8.0 (64-bit)

# locale:
# [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

# attached base packages:
# [1] parallel stats graphics grDevices utils datasets methods
# [8] base

# other attached packages:
# [1] Rsamtools_1.14.3
# [2] Biostrings_2.30.1
# [3] TxDb.Hsapiens.UCSC.hg19.knownGene_2.10.1
# [4] GenomicFeatures_1.14.5
# [5] AnnotationDbi_1.24.0
# [6] Biobase_2.22.0
# [7] GenomicRanges_1.14.4
# [8] XVector_0.2.0
# [9] IRanges_1.20.7
# [10] BiocGenerics_0.8.0
# [11] BiocInstaller_1.12.0

# loaded via a namespace (and not attached):
# [1] biomaRt_2.18.0 bitops_1.0-6 BSgenome_1.30.0
# [4] DBI_0.2-7 RCurl_1.95-4.1 RSQLite_0.11.4
# [7] rtracklayer_1.22.5 stats4_3.0.2 tools_3.0.2
# [10] XML_3.95-0.2 zlibbioc_1.8.0

[bigmw]

May I know what node, gene, pathway and what species you are talking about?

Thanks for the quick reply.
In above example GeneA is the only gene [shown green in original kegg] on that node for that species as other genes on the node are not present in the given species.
Thanks,
Shriram

[bigmw]

You don’t even have pathview package loaded based on your sessionInfo().

Im having problems with Pathview. I can only get native KEGG, the kegg.native=F does not work.

Also the native KEGG only has green color, not red (up regulated) and green (down regulated).

Why am I having these two problems?

# > sessionInfo()
# R version 3.0.3 (2013-09-25)
# Platform: x86_64-apple-darwin10.8.0 (64-bit)

# locale:
# [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

# attached base packages:
# [1] parallel stats graphics grDevices utils datasets methods
# [8] base

# other attached packages:
# [1] Rsamtools_1.14.3
# [2] Biostrings_2.30.1
# [3] TxDb.Hsapiens.UCSC.hg19.knownGene_2.10.1
# [4] GenomicFeatures_1.14.5
# [5] AnnotationDbi_1.24.0
# [6] Biobase_2.22.0
# [7] GenomicRanges_1.14.4
# [8] XVector_0.2.0
# [9] IRanges_1.20.7
# [10] BiocGenerics_0.8.0
# [11] BiocInstaller_1.12.0

# loaded via a namespace (and not attached):
# [1] biomaRt_2.18.0 bitops_1.0-6 BSgenome_1.30.0
# [4] DBI_0.2-7 RCurl_1.95-4.1 RSQLite_0.11.4
# [7] rtracklayer_1.22.5 stats4_3.0.2 tools_3.0.2
# [10] XML_3.95-0.2 zlibbioc_1.8.0