Seqanswers Leaderboard Ad

Collapse

Announcement

Collapse
No announcement yet.
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Crossbow on EMR (bowtie)

    Hello,

    I am running Bowtie using Crossbow on EMR, and getting the following. Any clues would be appreciated.

    Thanks,


    -----------------------------------

    2013-09-05 21:03:44,825 WARN org.apache.hadoop.conf.Configuration (main): DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of map tasks: null
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Setting default number of map tasks based on cluster size to : 84
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of reduce tasks: 0
    2013-09-05 21:03:51,187 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
    2013-09-05 21:03:51,208 WARN com.hadoop.compression.lzo.LzoCodec (main): Could not find build properties file with revision hash
    2013-09-05 21:03:51,208 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev UNKNOWN]
    2013-09-05 21:03:51,227 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library is available
    2013-09-05 21:03:51,227 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library loaded
    2013-09-05 21:03:51,336 INFO org.apache.hadoop.mapred.FileInputFormat (main): Total input paths to process : 1
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): getLocalDirs(): [/mnt/var/lib/hadoop/mapred, /mnt1/var/lib/hadoop/mapred, /mnt2/var/lib/hadoop/mapred, /mnt3/var/lib/hadoop/mapred]
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): Running job: job_201309052100_0001
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:03:51,926 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:03:52,930 INFO org.apache.hadoop.streaming.StreamJob (main): map 0% reduce 0%
    2013-09-05 21:04:15,006 INFO org.apache.hadoop.streaming.StreamJob (main): map 4% reduce 0%
    2013-09-05 21:04:18,015 INFO org.apache.hadoop.streaming.StreamJob (main): map 7% reduce 0%
    2013-09-05 21:04:21,024 INFO org.apache.hadoop.streaming.StreamJob (main): map 9% reduce 0%
    2013-09-05 21:04:24,032 INFO org.apache.hadoop.streaming.StreamJob (main): map 43% reduce 0%
    2013-09-05 21:04:25,035 INFO org.apache.hadoop.streaming.StreamJob (main): map 55% reduce 0%
    2013-09-05 21:04:27,040 INFO org.apache.hadoop.streaming.StreamJob (main): map 84% reduce 0%
    2013-09-05 21:04:33,056 INFO org.apache.hadoop.streaming.StreamJob (main): map 95% reduce 0%
    2013-09-05 21:04:36,064 INFO org.apache.hadoop.streaming.StreamJob (main): map 96% reduce 0%
    2013-09-05 21:04:54,110 INFO org.apache.hadoop.streaming.StreamJob (main): map 100% reduce 100%
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:04:54,113 ERROR org.apache.hadoop.streaming.StreamJob (main): Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201309052100_0001_m_000000
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): killJob...

  • #2
    Hello Huzefa,

    Did you manage to solve the issue? I am getting the similar error.

    Thanks

    Comment

    Latest Articles

    Collapse

    • seqadmin
      Current Approaches to Protein Sequencing
      by seqadmin


      Proteins are often described as the workhorses of the cell, and identifying their sequences is key to understanding their role in biological processes and disease. Currently, the most common technique used to determine protein sequences is mass spectrometry. While still a valuable tool, mass spectrometry faces several limitations and requires a highly experienced scientist familiar with the equipment to operate it. Additionally, other proteomic methods, like affinity assays, are constrained...
      04-04-2024, 04:25 PM
    • seqadmin
      Strategies for Sequencing Challenging Samples
      by seqadmin


      Despite advancements in sequencing platforms and related sample preparation technologies, certain sample types continue to present significant challenges that can compromise sequencing results. Pedro Echave, Senior Manager of the Global Business Segment at Revvity, explained that the success of a sequencing experiment ultimately depends on the amount and integrity of the nucleic acid template (RNA or DNA) obtained from a sample. “The better the quality of the nucleic acid isolated...
      03-22-2024, 06:39 AM

    ad_right_rmr

    Collapse

    News

    Collapse

    Topics Statistics Last Post
    Started by seqadmin, 04-11-2024, 12:08 PM
    0 responses
    13 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 04-10-2024, 10:19 PM
    0 responses
    17 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 04-10-2024, 09:21 AM
    0 responses
    14 views
    0 likes
    Last Post seqadmin  
    Started by seqadmin, 04-04-2024, 09:00 AM
    0 responses
    43 views
    0 likes
    Last Post seqadmin  
    Working...
    X