I'm using Hue for PIG scripts on amazon EMR. I am using the declare and default statements as mentioned in the documentation.
I have some %default and %declare statements and it looks like they are not preprocessed within Hue. Therefore, although the parameters are defined in my script, the editor keeps popping in a parameter configuration window. If I leave the parameter blank, the job fails with an error.
%declare OUTPUT_FOLDER 'testingOutput01'; ts = LOAD 's3://testbucket1/input/testdata-00000.gz' USING PigStorage('\t'); STORE ts INTO 's3://testbucket1/$OUTPUT_FOLDER' USING PigStorage('\t');
Upon execution, it shows the pop-up window asking for values for OUTPUT_FOLDER. If I leave it blank it fails with the following error:
2015-06-23 20:15:54,908 [main] ERROR org.apache.pig.Main - ERROR 2997: Encountered IOException. org.apache.pig.tools.parameters.ParseException: Encountered "<EOF>" at line 1, column 12. Was expecting one of: <IDENTIFIER> ... <OTHER> ... <LITERAL> ... <SHELLCMD> ...
Is that the expected behavior? Is this a known issue or am I missing something?
AMI version:3.7.0 Hadoop distribution:Amazon 2.4.0 Applications:Hive 0.13.1, Pig 0.12.0, Impala 1.2.4, Hue
The same behavior is seen with default instead of declare.
If you need any clarifications then please do comment on this question. I will update it as needed.