Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Crossbow on EMR (bowtie)

    Hello,

    I am running Bowtie using Crossbow on EMR, and getting the following. Any clues would be appreciated.

    Thanks,


    -----------------------------------

    2013-09-05 21:03:44,825 WARN org.apache.hadoop.conf.Configuration (main): DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of map tasks: null
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Setting default number of map tasks based on cluster size to : 84
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of reduce tasks: 0
    2013-09-05 21:03:51,187 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
    2013-09-05 21:03:51,208 WARN com.hadoop.compression.lzo.LzoCodec (main): Could not find build properties file with revision hash
    2013-09-05 21:03:51,208 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev UNKNOWN]
    2013-09-05 21:03:51,227 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library is available
    2013-09-05 21:03:51,227 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library loaded
    2013-09-05 21:03:51,336 INFO org.apache.hadoop.mapred.FileInputFormat (main): Total input paths to process : 1
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): getLocalDirs(): [/mnt/var/lib/hadoop/mapred, /mnt1/var/lib/hadoop/mapred, /mnt2/var/lib/hadoop/mapred, /mnt3/var/lib/hadoop/mapred]
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): Running job: job_201309052100_0001
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:03:51,926 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:03:52,930 INFO org.apache.hadoop.streaming.StreamJob (main): map 0% reduce 0%
    2013-09-05 21:04:15,006 INFO org.apache.hadoop.streaming.StreamJob (main): map 4% reduce 0%
    2013-09-05 21:04:18,015 INFO org.apache.hadoop.streaming.StreamJob (main): map 7% reduce 0%
    2013-09-05 21:04:21,024 INFO org.apache.hadoop.streaming.StreamJob (main): map 9% reduce 0%
    2013-09-05 21:04:24,032 INFO org.apache.hadoop.streaming.StreamJob (main): map 43% reduce 0%
    2013-09-05 21:04:25,035 INFO org.apache.hadoop.streaming.StreamJob (main): map 55% reduce 0%
    2013-09-05 21:04:27,040 INFO org.apache.hadoop.streaming.StreamJob (main): map 84% reduce 0%
    2013-09-05 21:04:33,056 INFO org.apache.hadoop.streaming.StreamJob (main): map 95% reduce 0%
    2013-09-05 21:04:36,064 INFO org.apache.hadoop.streaming.StreamJob (main): map 96% reduce 0%
    2013-09-05 21:04:54,110 INFO org.apache.hadoop.streaming.StreamJob (main): map 100% reduce 100%
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:04:54,113 ERROR org.apache.hadoop.streaming.StreamJob (main): Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201309052100_0001_m_000000
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): killJob...

  • #2
    Hello Huzefa,

    Did you manage to solve the issue? I am getting the similar error.

    Thanks

    Comment

    Latest Articles

    Collapse

    • seqadmin
      Strategies for Sequencing Challenging Samples
      by seqadmin


      Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
      03-22-2024, 06:39 AM
    • seqadmin
      Techniques and Challenges in Conservation Genomics
      by seqadmin



      The field of conservation genomics centers on applying genomics technologies in support of conservation efforts and the preservation of biodiversity. This article features interviews with two researchers who showcase their innovative work and highlight the current state and future of conservation genomics.

      Avian Conservation
      Matthew DeSaix, a recent doctoral graduate from Kristen Ruegg’s lab at The University of Colorado, shared that most of his research...
      03-08-2024, 10:41 AM

    ad_right_rmr

    Collapse

    News

    Collapse

    Topics Statistics Last Post
    Started by seqadmin, Yesterday, 06:37 PM
    0 responses
    8 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, Yesterday, 06:07 PM
    0 responses
    8 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 03-22-2024, 10:03 AM
    0 responses
    49 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 03-21-2024, 07:32 AM
    0 responses
    66 views
    0 likes
    Last Post seqadmin  
    Working...
    X