sqoop：【error】从hdfs导入到mysql 报ERROR tool.ExportTool: Error during export: Export job failed!

本文主要是介绍sqoop：【error】从hdfs导入到mysql 报ERROR tool.ExportTool: Error during export: Export job failed!，对大家解决编程问题具有一定的参考价值，需要的程序猿们随着小编来一起学习吧！

文章目录

• • 问题部分描述
  • 问题全部描述
  • 问题解决

问题部分描述

ERROR mapreduce.ExportJobBase: Export job failed!
20/11/24 16:47:36 ERROR tool.ExportTool: Error during export:
Export job failed!
        at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445)
        at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

问题全部描述

[huang@bigdata03 sqoop-1.4.7]$ bin/sqoop export --connect jdbc:mysql://bigdata03:3306/company -root --password 123456 --table users_sum_money --num-mappers 1 --export-dir /user/hive/warehousm_money --input-fields-terminated-by "\001"
Warning: /opt/module/sqoop-1.4.7/bin/../../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /opt/module/sqoop-1.4.7/bin/../../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /opt/module/sqoop-1.4.7/bin/../../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
20/11/24 16:46:34 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
20/11/24 16:46:34 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecurr using -P instead.
20/11/24 16:46:34 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
20/11/24 16:46:34 INFO tool.CodeGenTool: Beginning code generation
20/11/24 16:46:35 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `users_sum_t LIMIT 1
20/11/24 16:46:35 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `users_sum_t LIMIT 1
20/11/24 16:46:35 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/module/hadoop-2.7.2
注: /tmp/sqoop-huang/compile/9b5ca81a5b66ecf4d25bae1169847043/users_sum_money.java使用或覆盖了已I。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
20/11/24 16:46:37 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-huang/compile/9b5ca4d25bae1169847043/users_sum_money.jar
20/11/24 16:46:37 INFO mapreduce.ExportJobBase: Beginning export of users_sum_money
20/11/24 16:46:37 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapredr
20/11/24 16:46:40 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is . Instead, use mapreduce.reduce.speculative
20/11/24 16:46:40 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is depnstead, use mapreduce.map.speculative
20/11/24 16:46:40 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use job.maps
20/11/24 16:46:40 INFO client.RMProxy: Connecting to ResourceManager at bigdata02/192.168.17.10
20/11/24 16:46:45 INFO input.FileInputFormat: Total input paths to process : 1
20/11/24 16:46:45 INFO input.FileInputFormat: Total input paths to process : 1
20/11/24 16:46:45 INFO mapreduce.JobSubmitter: number of splits:1
20/11/24 16:46:45 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is depnstead, use mapreduce.map.speculative
20/11/24 16:46:45 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1606196215606_000
20/11/24 16:46:46 INFO impl.YarnClientImpl: Submitted application application_1606196215606_000
20/11/24 16:46:46 INFO mapreduce.Job: The url to track the job: http://bigdata02:8088/proxy/app606196215606_0007/
20/11/24 16:46:46 INFO mapreduce.Job: Running job: job_1606196215606_0007
20/11/24 16:47:25 INFO mapreduce.Job: Job job_1606196215606_0007 running in uber mode : false
20/11/24 16:47:25 INFO mapreduce.Job:  map 0% reduce 0%
20/11/24 16:47:34 INFO mapreduce.Job:  map 100% reduce 0%
20/11/24 16:47:35 INFO mapreduce.Job: Job job_1606196215606_0007 failed with state FAILED due tiled task_1606196215606_0007_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

20/11/24 16:47:36 INFO mapreduce.Job: Counters: 8
        Job Counters
                Failed map tasks=1
                Launched map tasks=1
                Data-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=6332
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=6332
                Total vcore-milliseconds taken by all map tasks=6332
                Total megabyte-milliseconds taken by all map tasks=6483968
20/11/24 16:47:36 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apacmapreduce.FileSystemCounter instead
20/11/24 16:47:36 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 55.3161 seconds (0 bytes
20/11/24 16:47:36 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is depre org.apache.hadoop.mapreduce.TaskCounter instead
20/11/24 16:47:36 INFO mapreduce.ExportJobBase: Exported 0 records.
20/11/24 16:47:36 ERROR mapreduce.ExportJobBase: Export job failed!
20/11/24 16:47:36 ERROR tool.ExportTool: Error during export:
Export job failed!
        at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445)
        at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

问题解决

原因是数据中有中文，而数据库表编码不支持

解决方案如下：

将表的数据导出，删除表后重新创建表，并指定编码DEFAULT CHARSET=utf8

这篇关于sqoop：【error】从hdfs导入到mysql 报ERROR tool.ExportTool: Error during export: Export job failed!的文章就介绍到这儿，希望我们推荐的文章对大家有所帮助，也希望大家多多支持为之网！