Reading excel file using Unstructured data stage

Pravenai · Post by **Pravenai** » Thu Jan 21, 2016 3:25 pm

Hi All,
I used "Unstructured Data" stage to read from an excel file. It ran perfectly fine for the first time. I got another file to load and when I ran the same job, it hangs everytime after loading 30,000 records (this file has a little more than 32K records).

I tried to debug the problem but could not identify the issue. File does not seem to have any issues, as I could load this file successfully into my table.

My job just loads data from excel into a table.

Any help on what could be possible problem or how to debug it will be really helpful.

Thanks in advance.

Regards,
PR

ray.wurlod · Post by **ray.wurlod** » Thu Jan 21, 2016 4:02 pm

Are there any warnings in the job log?

Pravenai · Post by **Pravenai** » Thu Jan 21, 2016 4:07 pm

No warnings. I have to kill the job in the end. Everytime it only loads 30K records.

chulett · Post by **chulett** » Thu Jan 21, 2016 4:08 pm

You may also want to involve your DBA when it 'hangs' to see if there are any issues on the database side, blocking locks and the like. Does it still hang if you run it on a single node? Assuming there are multiple nodes (>1) at the moment.

Pravenai · Post by **Pravenai** » Thu Jan 21, 2016 4:11 pm

Just to ensure it is not DB related issue, I removed Oracle stage (target) and replaced it with a dataset. Still I face the same issue.

chulett · Post by **chulett** » Thu Jan 21, 2016 6:12 pm

Okay... how about the nodes question?

ArndW · Post by **ArndW** » Fri Jan 22, 2016 9:30 am

I recall having had issues with large data volumes and the Unstructured data stage. The Java engine uses a lot of memory, you might try setting the environment variable:

Code: Select all

CC_UNST_JAVA_HEAP=512

to increase the heap to 512 from the default 256. If the job still hangs in the same place after this change you can be sure that isn't due to Java space; but I have a suspicion that it might be cause.

Pravenai · Post by **Pravenai** » Fri Jan 22, 2016 5:04 pm

Tried running my job on one node and also tried to run it after adding the environmental parameter but still it is stuck at the same place.
Do you think if there are any ways to debug this job and see where it is stuck?
Regards,
PR

chulett · Post by **chulett** » Fri Jan 22, 2016 7:12 pm

If you haven't already, involve your official support provider as well.

SURA · Post by **SURA** » Sun Jan 24, 2016 4:44 pm

chulett wrote:Okay... how about the nodes question?

I don't have 9.1, though my thought is also about the nodes. Rather than change the nodes, if that stage have the option to make is as Sequential change it, may help some time!

Try to read using server job will also give some idea.

Pravenai · Post by **Pravenai** » Mon Jan 25, 2016 3:53 pm

Thanks everyone. I will check with the support.

Regards,
PR