Unconfigured Ad

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • huzefa_mehta
    Junior Member
    • Aug 2013
    • 6
    #1

    Crossbow on EMR (bowtie)

    Hello,

    I am running Bowtie using Crossbow on EMR, and getting the following. Any clues would be appreciated.

    Thanks,


    -----------------------------------

    2013-09-05 21:03:44,825 WARN org.apache.hadoop.conf.Configuration (main): DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of map tasks: null
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Setting default number of map tasks based on cluster size to : 84
    2013-09-05 21:03:45,772 INFO org.apache.hadoop.mapred.JobClient (main): Default number of reduce tasks: 0
    2013-09-05 21:03:51,187 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
    2013-09-05 21:03:51,208 WARN com.hadoop.compression.lzo.LzoCodec (main): Could not find build properties file with revision hash
    2013-09-05 21:03:51,208 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev UNKNOWN]
    2013-09-05 21:03:51,227 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library is available
    2013-09-05 21:03:51,227 INFO org.apache.hadoop.io.compress.snappy.LoadSnappy (main): Snappy native library loaded
    2013-09-05 21:03:51,336 INFO org.apache.hadoop.mapred.FileInputFormat (main): Total input paths to process : 1
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): getLocalDirs(): [/mnt/var/lib/hadoop/mapred, /mnt1/var/lib/hadoop/mapred, /mnt2/var/lib/hadoop/mapred, /mnt3/var/lib/hadoop/mapred]
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): Running job: job_201309052100_0001
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:03:51,925 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:03:51,926 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:03:52,930 INFO org.apache.hadoop.streaming.StreamJob (main): map 0% reduce 0%
    2013-09-05 21:04:15,006 INFO org.apache.hadoop.streaming.StreamJob (main): map 4% reduce 0%
    2013-09-05 21:04:18,015 INFO org.apache.hadoop.streaming.StreamJob (main): map 7% reduce 0%
    2013-09-05 21:04:21,024 INFO org.apache.hadoop.streaming.StreamJob (main): map 9% reduce 0%
    2013-09-05 21:04:24,032 INFO org.apache.hadoop.streaming.StreamJob (main): map 43% reduce 0%
    2013-09-05 21:04:25,035 INFO org.apache.hadoop.streaming.StreamJob (main): map 55% reduce 0%
    2013-09-05 21:04:27,040 INFO org.apache.hadoop.streaming.StreamJob (main): map 84% reduce 0%
    2013-09-05 21:04:33,056 INFO org.apache.hadoop.streaming.StreamJob (main): map 95% reduce 0%
    2013-09-05 21:04:36,064 INFO org.apache.hadoop.streaming.StreamJob (main): map 96% reduce 0%
    2013-09-05 21:04:54,110 INFO org.apache.hadoop.streaming.StreamJob (main): map 100% reduce 100%
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): To kill this job, run:
    2013-09-05 21:04:54,112 INFO org.apache.hadoop.streaming.StreamJob (main): /home/hadoop/bin/hadoop job -Dmapred.job.tracker=172.31.2.68:9001 -kill job_201309052100_0001
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): Tracking URL: http://ip-172-31-2-68:9100/jobdetail...309052100_0001
    2013-09-05 21:04:54,113 ERROR org.apache.hadoop.streaming.StreamJob (main): Job not successful. Error: # of failed Map Tasks exceeded allowed limit. FailedCount: 1. LastFailedTask: task_201309052100_0001_m_000000
    2013-09-05 21:04:54,113 INFO org.apache.hadoop.streaming.StreamJob (main): killJob...
    Tags: bowtie, crossbow, emr
    • Ashwinub
      Junior Member
      • Sep 2013
      • 7
      #2
      Hello Huzefa,

      Did you manage to solve the issue? I am getting the similar error.

      Thanks

        Comment

        Previous template Next

        Latest Articles

        Collapse
        • SEQadmin2
          Multiomic Liquid Biopsy: What DNA/RNA Panels Reveal That DNA Alone Can’t
          by SEQadmin2



          Tumors contain complex mixtures of cell populations that evolve in response to treatment and host-specific factors. Tissue biopsies provide only static snapshots of a tumor’s state. But what if we could follow that evolution over time using a minimally invasive technique?

          Liquid biopsies can detect tumor-derived material circulating in the bloodstream, providing a way to study treatment response, detect emerging resistance, and discover new biomarkers. Circulating          ...
          10-14-2026, 09:55 AM
        • SEQadmin2
          New Genomics Technologies Take Aim at Long-Standing Limits
          by SEQadmin2


          Researchers using sequencing and genomics tools often have to make trade-offs. They can choose between speed or scale, short reads or long-range information, or targeted panels or a view of the whole transcriptome. New technologies that have been released this year are built to address those tough choices.

          We asked six companies the same four questions to learn about their latest products. The new technologies bring a lot to the table, including rethinking sequencing          ...
          09-28-2026, 10:25 AM
        View All

        ad_right_rmr

        Collapse

        News

        Collapse
        Topics Statistics Last Post
        Mayo Clinic-Led Study Detects DNA Mutations Across Noncancerous Tissues by SEQadmin2
        Started by SEQadmin2, 10-15-2026, 10:52 AM
        0 responses
        21 views
        0 reactions
        		 Last Post SEQadmin2
        by SEQadmin2
        10-15-2026, 10:52 AM  
        Study Finds Chromatin Motion Holds Steady Until Cells Face Stress by SEQadmin2
        Started by SEQadmin2, 10-14-2026, 05:39 AM
        0 responses
        26 views
        0 reactions
        		 Last Post SEQadmin2
        by SEQadmin2
        10-14-2026, 05:39 AM  
        Larger Datasets Reveal Additional Switches That Control Gene Expression by SEQadmin2
        Started by SEQadmin2, 10-13-2026, 09:56 AM
        0 responses
        22 views
        0 reactions
        		 Last Post SEQadmin2
        by SEQadmin2
        10-13-2026, 09:56 AM  
        Computational Tool Captures How Chromatin Folds at Scale by SEQadmin2
        Started by SEQadmin2, 10-09-2026, 11:14 AM
        0 responses
        28 views
        0 reactions
        		 Last Post SEQadmin2
        by SEQadmin2
        10-09-2026, 11:14 AM  
        View All
        Working...