Page 1 of 1
Compilation hangs forever
Posted: Thu Jul 19, 2007 10:09 pm
by aramachandra
Hi
I have simple job which does the following
1) reads from the dataset
2) does remove duplicates to retain unique headers
3) does a lookup to pull in all the details for the headers
4) transforms each looked up column with
if IsNull(Link.Col) then '' else Link.Col
5) Writes to an ouptut dataset
WE are using px edition 7.5.1 on a sun solaris unix environment
The only thing unique is that IT HAS 305 COLUMNS AND almost 299 DERIVATION JUST LIKE THE one above
When i compile the job, the jobs hangs with Please Wait compile message box.
When i remove the transformer the jobs just compiles and runs just fine
Is there a limitation on the NUMBER of columns one can process in a transformer?
Has anybody else seen this kind of a situation in their unix environment
What if any is the recommendations
Thanks all in advance for your ideas
Posted: Thu Jul 19, 2007 10:31 pm
by ArndW
Transform PX stages can take a long time to generate and compile. On some systems the C++ compiler has a concurrent user limit, so you not only need to wait the time for the compile but you might also be waiting for someone else to finish using a C++ license.
I have waited 10-15 minutes for a compile to finish on a system that didn't seem overly loaded. How long have you waited?
You can also start the compile and use truss (or equivalent) to monitor what the background process is doing; that lets you see if it is waiting, hanging or perhaps even aborted.
Posted: Thu Jul 19, 2007 10:43 pm
by pbatchu
Thanks for the reply
But i have waited for most than 4 hours for it to complete with no success
It actually died on my with a error which says
Compilation failed - no other information available
Hence my question on the limitation of the transformer
Thanks again
Arvind
Posted: Thu Jul 19, 2007 10:52 pm
by ArndW
I am sure that there is a limitation on the number of columns, but it is certainly higher than what you currently have.
I am confused, you state that the compile hangs forever, yet you have a message "compile failed" - which is it? If the compile failed, there will be additional error messages.
Posted: Thu Jul 19, 2007 11:24 pm
by balajisr
Arnd,
I am sure that there is a limitation on the number of columns, but it is certainly higher than what you currently have.
Can you please tell the limit.
Posted: Thu Jul 19, 2007 11:31 pm
by ray.wurlod
No, because IBM don't document that. We know there must be a limit because we're software engineers, and there's no such thing as unlimited memory - you always design maximum sizes for things. Years later, you wonder why you thought so small - was it not a young Bill Gates who asserted that 640KB is all the memory you will ever need?
Posted: Thu Jul 19, 2007 11:41 pm
by ArndW
No, I don't know the limit. But I just tested a job with 1000 columns and derivations and it compiled and ran without a problem. The compilation took about 10 minutes and as a comparison my current PX job compiles in about 30 seconds ( ~80 stages, no transforms).
Posted: Fri Jul 20, 2007 12:08 am
by balajisr
Thanks Arnd and Ray for clarifying this.
Posted: Fri Jul 20, 2007 7:58 am
by aramachandra
Actually it is a bit of both
the compilation hangs in my opinion in that it runs for foreever
Due to concurrent licensing issues we have a process that goes and kills sessions that are not active for over 2 hours
So while i work i am looking at it on my other screen still saying it is compiling please wait
I leave for home and log on from home, which is after two hours and i see that script has killed my session and hence the compilation is terminated
So i think it hangs but it is killed behind the scenes by our inactive session kill script
But again that is well past 4 hours since i kicked off the compilation