gmane.comp.java.hadoop.pig.devel http://blog.gmane.org/gmane.comp.java.hadoop.pig.devel hourly 1 1901-01-01T00:00+00:00 Gmane http://gmane.org/img/gmane-25t.png http://gmane.org Subscription: PIG patch available http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31552 <pre>Issue Subscription Filter: PIG patch available (15 issues) Subscriber: pigdaily Key Summary PIG-3346 New property that controls the number of combined splits https://issues.apache.org/jira/browse/PIG-3346 PIG-3333 Fix remaining Windows core unit test failures https://issues.apache.org/jira/browse/PIG-3333 PIG-3295 Casting from bytearray failing after Union (even when each field is from a single Loader) https://issues.apache.org/jira/browse/PIG-3295 PIG-3292 Logical plan invalid state: duplicate uid in schema during self-join to get cross product https://issues.apache.org/jira/browse/PIG-3292 PIG-3288 Kill jobs if the number of output files is over a configurable limit https://issues.apache.org/jira/browse/PIG-3288 PIG-3257 Add unique identifier UDF https://issues.apache.org/jira/browse/PIG-3257 PIG-3247 Piggybank functions to mimic OVER clause in SQL https://issues.apache.org/jira/browse/PIG-3247 PIG-3210 Pig fails to start when it cannot write log to log files https://issues.apache.org/jira/browse/PIG-3210 PIG-3199 Expose LogicalPlan via PigServer API https://issues.apache.org/jira/browse/PIG-3199 PIG-3166 Update eclipse .classpath according to ivy library.properties https://issues.apache.org/jira/browse/PIG-3166 PIG-3123 Simplify Logical Plans By Removing Unneccessary Identity Projections https://issues.apache.org/jira/browse/PIG-3123 PIG-3088 Add a builtin udf which removes prefixes https://issues.apache.org/jira/browse/PIG-3088 PIG-3015 Rewrite of AvroStorage https://issues.apache.org/jira/browse/PIG-3015 PIG-2248 Pig parser does not detect when a macro name masks a UDF name https://issues.apache.org/jira/browse/PIG-2248 PIG-1914 Support load/store JSON data in Pig https://issues.apache.org/jira/browse/PIG-1914 You may edit this subscription at: https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225&amp;filterId=12322384 </pre> jira-1oDqGaOF3Lkdnm+yROfE0A< at >public.gmane.org 2013-06-20T01:02:00 Re: Improving macros http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31551 <pre>Jon is right. I am trying to ensure that each line is mostly parsed only once in https://issues.apache.org/jira/browse/PIG-3204. Have few issues with other commands in pig script like fs, shell, cd, illustrate, error messages not showing line numbers properly, etc which I have not got to solving yet. Register and import command file localization should be done only once with my patch as the parsing happens only once (unless you have fs or sh commands). Will check it out to be doubly sure. Regards, Rohini On Wed, Jun 19, 2013 at 1:03 PM, Jonathan Coveney &lt;jcoveney-Re5JQEeQqe8AvxtiuMwx3w&lt; at &gt;public.gmane.org&gt;wrote: </pre> Rohini Palaniswamy 2013-06-19T21:36:55 Re: Improving macros http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31550 <pre>This sounds excellent! Would love to see this in trunk. As far as #3, this is probably because pig does essentially reparse everything with each new line. I know there is a ticket where Rohini dealt with this in some cases where HDFS was being hit multiple times because of load statements getting reparsed, but I'm not sure if remote imports are fixed by that patch as well. 2013/6/19 Jonathan Packer &lt;jpacker-TPSCRWv7LJmzR313v1King&lt; at &gt;public.gmane.org&gt; </pre> Jonathan Coveney 2013-06-19T20:03:05 Improving macros http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31549 <pre>Hi, I'm an engineer at Mortar Data. I was working on some features to improve macros that I'd like to contribute (we're hoping to build a library of reusable pig macros implementing common algorithms), but I wanted to check-in here first to see if anyone has concerns about the changes I'd be making. The changes I've implemented are: 1. Macro files can register jars and udfs (avoiding namespace conflicts is the user's responsibility) 2. Macro files can be be redundantly imported (the extra import statements will be ignored). The use case is pigscript A imports macro files A and B, but A also imports B. Pig will emit a warning, but not fail as it currently does. 3. Registers and imports from S3 aren't repeatedly downloaded as a pigscript is parsed. I'm not sure why it was doing this in the first place, but it looked like a query was being assembled line-by-line and every time it would re-download jars etc. I was working on our fork of 0.9.2 with modifications, so please let me know if any of these have already been fixed in the latest version. Thanks, Jonathan Packer </pre> Jonathan Packer 2013-06-19T17:59:26 questions about roadmap http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31548 <pre>Hi, I have some questions for you about roadmap..Please give me some advice. Could you explain your public or private roadmap of your open source for years to come? We expect to know your development plan by year as detail as possible. (If you have no plan,please tell me your opinion.2013~2016) I want to know start and end time each technology. Thank you for giving me your time. </pre> 박주현 2013-06-19T15:37:05 [Commented] (PIG-3288) Kill jobs if the number of output files is over a configurable limit http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31547 <pre> [ https://issues.apache.org/jira/browse/PIG-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=13687624#comment-13687624 ] Aniket Mokashi commented on PIG-3288: ------------------------------------- Implementation is generic that the counter need not count number of files. It can really count arbitrary metrics and kill the job if exceeded. Should we rename the counter from "pig.exec.created.files.max.limit" to something else? Also, in the storefunc, you are relying on the fact that for each new file storefunc is reinitialize in a new object. Is it a guaranteed behavior? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-19T05:25:21 [Commented] (PIG-3357) Pig doesn't take care of declared float type and converts it to double http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31546 <pre> [ https://issues.apache.org/jira/browse/PIG-3357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=13687481#comment-13687481 ] Cheolsoo Park commented on PIG-3357: ------------------------------------ This is because Pig always converts Python float to Java double. {code:title=JythonUtils.java} } else if (pyObject instanceof PyFloat) { // J(P)ython is loosely typed, supports only float type, // hence we convert everything to double to save precision javaObj = pyObject.__tojava__(Double.class); } {code} In fact, that is a good thing because Python floating point numbers are implemented using double in C: http://docs.python.org/2/library/stdtypes.html#typesnumeric One workaround is to specify bytearray as the outputSchema of Python udf and do a cast to float in Pig. That is, {code:title=test.py} &lt; at &gt;outputSchema("v: bytearray") def test(v): return v {code} {code:title=test.pig} table_out = foreach table_in generate (float)udf.test(v); {code} Perhaps we should document this in the Pig manual. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Cheolsoo Park (JIRA 2013-06-19T01:14:20 Subscription: PIG patch available http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31545 <pre>Issue Subscription Filter: PIG patch available (15 issues) Subscriber: pigdaily Key Summary PIG-3346 New property that controls the number of combined splits https://issues.apache.org/jira/browse/PIG-3346 PIG-3333 Fix remaining Windows core unit test failures https://issues.apache.org/jira/browse/PIG-3333 PIG-3295 Casting from bytearray failing after Union (even when each field is from a single Loader) https://issues.apache.org/jira/browse/PIG-3295 PIG-3292 Logical plan invalid state: duplicate uid in schema during self-join to get cross product https://issues.apache.org/jira/browse/PIG-3292 PIG-3288 Kill jobs if the number of output files is over a configurable limit https://issues.apache.org/jira/browse/PIG-3288 PIG-3257 Add unique identifier UDF https://issues.apache.org/jira/browse/PIG-3257 PIG-3247 Piggybank functions to mimic OVER clause in SQL https://issues.apache.org/jira/browse/PIG-3247 PIG-3210 Pig fails to start when it cannot write log to log files https://issues.apache.org/jira/browse/PIG-3210 PIG-3199 Expose LogicalPlan via PigServer API https://issues.apache.org/jira/browse/PIG-3199 PIG-3166 Update eclipse .classpath according to ivy library.properties https://issues.apache.org/jira/browse/PIG-3166 PIG-3123 Simplify Logical Plans By Removing Unneccessary Identity Projections https://issues.apache.org/jira/browse/PIG-3123 PIG-3088 Add a builtin udf which removes prefixes https://issues.apache.org/jira/browse/PIG-3088 PIG-3015 Rewrite of AvroStorage https://issues.apache.org/jira/browse/PIG-3015 PIG-2248 Pig parser does not detect when a macro name masks a UDF name https://issues.apache.org/jira/browse/PIG-2248 PIG-1914 Support load/store JSON data in Pig https://issues.apache.org/jira/browse/PIG-1914 You may edit this subscription at: https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225&amp;filterId=12322384 </pre> jira-1oDqGaOF3Lkdnm+yROfE0A< at >public.gmane.org 2013-06-19T01:01:02 Fwd: IMPORTANT: Major Confluence Upgrade Coming Soon. Please review test instance now. http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31544 <pre> Julien Begin forwarded message: </pre> Julien Le Dem 2013-06-18T19:35:24 [Updated] (PIG-3355) ColumnMapKeyPrune bug with distinct operator http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31543 <pre> [ https://issues.apache.org/jira/browse/PIG-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-3355: -------------------------------- Resolution: Fixed Status: Resolved (was: Patch Available) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-18T17:38:21 [Commented] (PIG-3355) ColumnMapKeyPrune bug with distinct operator http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31542 <pre> [ https://issues.apache.org/jira/browse/PIG-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=13686997#comment-13686997 ] Aniket Mokashi commented on PIG-3355: ------------------------------------- [~knoguchi], Feel free to rebase and submit a patch for 0.11. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-18T17:38:21 [Commented] (PIG-3355) ColumnMapKeyPrune bug with distinct operator http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31541 <pre> [ https://issues.apache.org/jira/browse/PIG-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=13686815#comment-13686815 ] Koji Noguchi commented on PIG-3355: ----------------------------------- bq. Committed to trunk. Thanks Jeremy! [~aniket486], status is still "Patch Available"? Also, can we patch 0.11 as well so that it'll be included if we release another 0.11.* ? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Koji Noguchi (JIRA 2013-06-18T15:11:20 [Updated] (PIG-3357) Pig doesn't take care of declared float type and converts it to double http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31540 <pre> [ https://issues.apache.org/jira/browse/PIG-3357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey updated PIG-3357: ------------------------ Summary: Pig doesn't take care of declared float type and converts it to double (was: Pig doesn't take case of declared float and converts it to double) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Sergey (JIRA 2013-06-18T09:45:19 [Created] (PIG-3357) Pig doesn't take case of declared float and converts it to double http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31539 <pre>Sergey created PIG-3357: --------------------------- Summary: Pig doesn't take case of declared float and converts it to double Key: PIG-3357 URL: https://issues.apache.org/jira/browse/PIG-3357 Project: Pig Issue Type: Bug Components: piggybank Affects Versions: 0.11 Environment: cdh 4.3.0 Reporter: Sergey Here is the script: {code} register /usr/lib/pig/lib/avro-1.7.4.jar; register /usr/lib/pig/lib/json-simple-1.1.jar; register /usr/lib/pig/piggybank.jar; register test.py using jython as udf; table_in = load 'in' as (v: float); table_out = foreach table_in generate udf.test(v); store table_out into 'out' using org.apache.pig.piggybank.storage.avro.AvroStorage('schema', '{"name": "test", "type": "float"}'); {code} Here is UDF: {code=python} &lt; at &gt;outputSchema("v: float") def test(v): return v {code} Here is an input: {code} 1 {code} Here is the stacktrace: java.lang.Exception: org.apache.avro.file.DataFileWriter$AppendWriteException: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404) Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:260) at org.apache.pig.piggybank.storage.avro.PigAvroRecordWriter.write(PigAvroRecordWriter.java:49) at org.apache.pig.piggybank.storage.avro.AvroStorage.putNext(AvroStorage.java:722) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:558) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:285) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.writeFloat(PigAvroDatumWriter.java:281) at org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.write(PigAvroDatumWriter.java:88) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58) at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:257) ... 21 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Sergey (JIRA 2013-06-18T09:43:21 [Updated] (PIG-3357) Pig doesn't take case of declared float and converts it to double http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31538 <pre> [ https://issues.apache.org/jira/browse/PIG-3357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey updated PIG-3357: ------------------------ Description: Here is the script: {code} register /usr/lib/pig/lib/avro-1.7.4.jar; register /usr/lib/pig/lib/json-simple-1.1.jar; register /usr/lib/pig/piggybank.jar; register test.py using jython as udf; table_in = load 'in' as (v: float); table_out = foreach table_in generate udf.test(v); store table_out into 'out' using org.apache.pig.piggybank.storage.avro.AvroStorage('schema', '{"name": "test", "type": "float"}'); {code} Here is UDF: {code} &lt; at &gt;outputSchema("v: float") def test(v): return v {code} Here is an input: {code} 1 {code} Here is the stacktrace: java.lang.Exception: org.apache.avro.file.DataFileWriter$AppendWriteException: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404) Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:260) at org.apache.pig.piggybank.storage.avro.PigAvroRecordWriter.write(PigAvroRecordWriter.java:49) at org.apache.pig.piggybank.storage.avro.AvroStorage.putNext(AvroStorage.java:722) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:558) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:285) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.writeFloat(PigAvroDatumWriter.java:281) at org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.write(PigAvroDatumWriter.java:88) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58) at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:257) ... 21 more was: Here is the script: {code} register /usr/lib/pig/lib/avro-1.7.4.jar; register /usr/lib/pig/lib/json-simple-1.1.jar; register /usr/lib/pig/piggybank.jar; register test.py using jython as udf; table_in = load 'in' as (v: float); table_out = foreach table_in generate udf.test(v); store table_out into 'out' using org.apache.pig.piggybank.storage.avro.AvroStorage('schema', '{"name": "test", "type": "float"}'); {code} Here is UDF: {code=python} &lt; at &gt;outputSchema("v: float") def test(v): return v {code} Here is an input: {code} 1 {code} Here is the stacktrace: java.lang.Exception: org.apache.avro.file.DataFileWriter$AppendWriteException: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:404) Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:260) at org.apache.pig.piggybank.storage.avro.PigAvroRecordWriter.write(PigAvroRecordWriter.java:49) at org.apache.pig.piggybank.storage.avro.AvroStorage.putNext(AvroStorage.java:722) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:558) at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:85) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:106) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:285) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: java.io.IOException: Cannot convert to float:class java.lang.Double at org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.writeFloat(PigAvroDatumWriter.java:281) at org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter.write(PigAvroDatumWriter.java:88) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58) at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:257) ... 21 more -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Sergey (JIRA 2013-06-18T09:43:21 [Updated] (PIG-2244) Macros cannot be passed relation names http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31537 <pre> [ https://issues.apache.org/jira/browse/PIG-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi updated PIG-2244: -------------------------------- Status: Open (was: Patch Available) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-18T05:47:22 [Commented] (PIG-2244) Macros cannot be passed relation names http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31536 <pre> [ https://issues.apache.org/jira/browse/PIG-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=13686412#comment-13686412 ] Aniket Mokashi commented on PIG-2244: ------------------------------------- Canceling patch.. Please re-submit when its ready. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-18T05:47:21 [Commented] (PIG-3355) ColumnMapKeyPrune bug with distinct operator http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31535 <pre> [ https://issues.apache.org/jira/browse/PIG-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=13686411#comment-13686411 ] Aniket Mokashi commented on PIG-3355: ------------------------------------- Committed to trunk. Thanks Jeremy! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-18T05:45:24 [Assigned] (PIG-3355) ColumnMapKeyPrune bug with distinct operator http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31534 <pre> [ https://issues.apache.org/jira/browse/PIG-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aniket Mokashi reassigned PIG-3355: ----------------------------------- Assignee: Jeremy Karn -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-18T04:50:23 [Commented] (PIG-3355) ColumnMapKeyPrune bug with distinct operator http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31533 <pre> [ https://issues.apache.org/jira/browse/PIG-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&amp;focusedCommentId=13686389#comment-13686389 ] Aniket Mokashi commented on PIG-3355: ------------------------------------- +1. Will commit if ant test-commit passes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira </pre> Aniket Mokashi (JIRA 2013-06-18T04:48:20 Subscription: PIG patch available http://permalink.gmane.org/gmane.comp.java.hadoop.pig.devel/31532 <pre>Issue Subscription Filter: PIG patch available (17 issues) Subscriber: pigdaily Key Summary PIG-3355 ColumnMapKeyPrune bug with distinct operator https://issues.apache.org/jira/browse/PIG-3355 PIG-3346 New property that controls the number of combined splits https://issues.apache.org/jira/browse/PIG-3346 PIG-3333 Fix remaining Windows core unit test failures https://issues.apache.org/jira/browse/PIG-3333 PIG-3295 Casting from bytearray failing after Union (even when each field is from a single Loader) https://issues.apache.org/jira/browse/PIG-3295 PIG-3292 Logical plan invalid state: duplicate uid in schema during self-join to get cross product https://issues.apache.org/jira/browse/PIG-3292 PIG-3288 Kill jobs if the number of output files is over a configurable limit https://issues.apache.org/jira/browse/PIG-3288 PIG-3257 Add unique identifier UDF https://issues.apache.org/jira/browse/PIG-3257 PIG-3247 Piggybank functions to mimic OVER clause in SQL https://issues.apache.org/jira/browse/PIG-3247 PIG-3210 Pig fails to start when it cannot write log to log files https://issues.apache.org/jira/browse/PIG-3210 PIG-3199 Expose LogicalPlan via PigServer API https://issues.apache.org/jira/browse/PIG-3199 PIG-3166 Update eclipse .classpath according to ivy library.properties https://issues.apache.org/jira/browse/PIG-3166 PIG-3123 Simplify Logical Plans By Removing Unneccessary Identity Projections https://issues.apache.org/jira/browse/PIG-3123 PIG-3088 Add a builtin udf which removes prefixes https://issues.apache.org/jira/browse/PIG-3088 PIG-3015 Rewrite of AvroStorage https://issues.apache.org/jira/browse/PIG-3015 PIG-2248 Pig parser does not detect when a macro name masks a UDF name https://issues.apache.org/jira/browse/PIG-2248 PIG-2244 Macros cannot be passed relation names https://issues.apache.org/jira/browse/PIG-2244 PIG-1914 Support load/store JSON data in Pig https://issues.apache.org/jira/browse/PIG-1914 You may edit this subscription at: https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225&amp;filterId=12322384 </pre> jira-1oDqGaOF3Lkdnm+yROfE0A< at >public.gmane.org 2013-06-18T01:02:01 Search Engine Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.java.hadoop.pig.devel