When I run a query using tez , the number of output files are very huge. I have some 4-5 GB of data each having 46 MB or 16 MB. I want to have only 2-3 files as output files.
My output files location will be google cloud storage. How do I merge the files?
set mapred.reduce.tasks = 1; set hive.merge.mapfiles = true; set hive.mergejob.maponly = true; set hive.merge.mapredfiles=true;
I did set these parameters. And I did write insert overwrite query to overwrite the data in same location. No use. Please help.