Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Crossbow on EMR (bowtie)

    Hello,

    I am running Bowtie using Crossbow on EMR, and getting the following. Any clues would be appreciated.

    Thanks,


    -----------------------------------

    2013-09-05 21:03:44,825 WARN org.apache.hadoop.conf.Configuration (main): DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of map tasks: null
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Setting default number of map tasks based on cluster size to : 84
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of reduce tasks: 0
    2013-09-05 21:03:51,187 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
    2013-09-05 21:03:51,208 WARN com.hadoop.compression.lzo.LzoCodec (main): Could not find build properties file with revision hash
    2013-09-05 21:03:51,208 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev UNKNOWN]
    2013-09-05 21:03:51,227 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library is available
    2013-09-05 21:03:51,227 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library loaded
    2013-09-05 21:03:51,336 INFO org.apache.hadoop.mapred.FileInputFormat (main): Total input paths to process : 1
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): getLocalDirs(): [/mnt/var/lib/hadoop/mapred, /mnt1/var/lib/hadoop/mapred, /mnt2/var/lib/hadoop/mapred, /mnt3/var/lib/hadoop/mapred]
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): Running job: job_201309052100_0001
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:03:51,926 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:03:52,930 INFO org.apache.hadoop.streaming.StreamJob (main): map 0% reduce 0%
    2013-09-05 21:04:15,006 INFO org.apache.hadoop.streaming.StreamJob (main): map 4% reduce 0%
    2013-09-05 21:04:18,015 INFO org.apache.hadoop.streaming.StreamJob (main): map 7% reduce 0%
    2013-09-05 21:04:21,024 INFO org.apache.hadoop.streaming.StreamJob (main): map 9% reduce 0%
    2013-09-05 21:04:24,032 INFO org.apache.hadoop.streaming.StreamJob (main): map 43% reduce 0%
    2013-09-05 21:04:25,035 INFO org.apache.hadoop.streaming.StreamJob (main): map 55% reduce 0%
    2013-09-05 21:04:27,040 INFO org.apache.hadoop.streaming.StreamJob (main): map 84% reduce 0%
    2013-09-05 21:04:33,056 INFO org.apache.hadoop.streaming.StreamJob (main): map 95% reduce 0%
    2013-09-05 21:04:36,064 INFO org.apache.hadoop.streaming.StreamJob (main): map 96% reduce 0%
    2013-09-05 21:04:54,110 INFO org.apache.hadoop.streaming.StreamJob (main): map 100% reduce 100%
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:04:54,113 ERROR org.apache.hadoop.streaming.StreamJob (main): Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201309052100_0001_m_000000
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): killJob...

  • #2
    Hello Huzefa,

    Did you manage to solve the issue? I am getting the similar error.

    Thanks

    Comment

    Latest Articles

    Collapse

    • seqadmin
      Best Practices for Single-Cell Sequencing Analysis
      by seqadmin



      While isolating and preparing single cells for sequencing was historically the bottleneck, recent technological advancements have shifted the challenge to data analysis. This highlights the rapidly evolving nature of single-cell sequencing. The inherent complexity of single-cell analysis has intensified with the surge in data volume and the incorporation of diverse and more complex datasets. This article explores the challenges in analysis, examines common pitfalls, offers...
      06-06-2024, 07:15 AM
    • seqadmin
      Latest Developments in Precision Medicine
      by seqadmin



      Technological advances have led to drastic improvements in the field of precision medicine, enabling more personalized approaches to treatment. This article explores four leading groups that are overcoming many of the challenges of genomic profiling and precision medicine through their innovative platforms and technologies.

      Somatic Genomics
      “We have such a tremendous amount of genetic diversity that exists within each of us, and not just between us as individuals,”...
      05-24-2024, 01:16 PM

    ad_right_rmr

    Collapse

    News

    Collapse

    Topics Statistics Last Post
    Started by seqadmin, 06-17-2024, 06:54 AM
    0 responses
    10 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 06-14-2024, 07:24 AM
    0 responses
    21 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 06-13-2024, 08:58 AM
    0 responses
    17 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 06-12-2024, 02:20 PM
    0 responses
    20 views
    0 likes
    Last Post seqadmin  
    Working...
    X