Page 1 of 1
Reading excel file using Unstructured data stage
Posted: Thu Jan 21, 2016 3:25 pm
by Pravenai
Hi All,
I used "Unstructured Data" stage to read from an excel file. It ran perfectly fine for the first time. I got another file to load and when I ran the same job, it hangs everytime after loading 30,000 records (this file has a little more than 32K records).
I tried to debug the problem but could not identify the issue. File does not seem to have any issues, as I could load this file successfully into my table.
My job just loads data from excel into a table.
Any help on what could be possible problem or how to debug it will be really helpful.
Thanks in advance.
Regards,
PR
Posted: Thu Jan 21, 2016 4:02 pm
by ray.wurlod
Are there any warnings in the job log?
Posted: Thu Jan 21, 2016 4:07 pm
by Pravenai
No warnings. I have to kill the job in the end. Everytime it only loads 30K records.
Posted: Thu Jan 21, 2016 4:08 pm
by chulett
You may also want to involve your DBA when it 'hangs' to see if there are any issues on the database side, blocking locks and the like. Does it still hang if you run it on a single node? Assuming there are multiple nodes (>1) at the moment.
Posted: Thu Jan 21, 2016 4:11 pm
by Pravenai
Just to ensure it is not DB related issue, I removed Oracle stage (target) and replaced it with a dataset. Still I face the same issue.
Posted: Thu Jan 21, 2016 6:12 pm
by chulett
Okay... how about the nodes question?
Posted: Fri Jan 22, 2016 9:30 am
by ArndW
I recall having had issues with large data volumes and the Unstructured data stage. The Java engine uses a lot of memory, you might try setting the environment variable:
to increase the heap to 512 from the default 256. If the job still hangs in the same place after this change you can be sure that isn't due to Java space; but I have a suspicion that it might be cause.
Posted: Fri Jan 22, 2016 5:04 pm
by Pravenai
Tried running my job on one node and also tried to run it after adding the environmental parameter but still it is stuck at the same place.
Do you think if there are any ways to debug this job and see where it is stuck?
Regards,
PR
Posted: Fri Jan 22, 2016 7:12 pm
by chulett
If you haven't already, involve your official support provider as well.
Posted: Sun Jan 24, 2016 4:44 pm
by SURA
chulett wrote:Okay... how about the nodes question?
I don't have 9.1, though my thought is also about the nodes. Rather than change the nodes, if that stage have the option to make is as Sequential change it, may help some time!
Try to read using server job will also give some idea.
Posted: Mon Jan 25, 2016 3:53 pm
by Pravenai
Thanks everyone. I will check with the support.
Regards,
PR