Compilation hangs forever

[aramachandra]

Hi

I have simple job which does the following

1) reads from the dataset
2) does remove duplicates to retain unique headers
3) does a lookup to pull in all the details for the headers
4) transforms each looked up column with
if IsNull(Link.Col) then '' else Link.Col
5) Writes to an ouptut dataset

WE are using px edition 7.5.1 on a sun solaris unix environment

The only thing unique is that IT HAS 305 COLUMNS AND almost 299 DERIVATION JUST LIKE THE one above

When i compile the job, the jobs hangs with Please Wait compile message box.

When i remove the transformer the jobs just compiles and runs just fine

Is there a limitation on the NUMBER of columns one can process in a transformer?

Has anybody else seen this kind of a situation in their unix environment

What if any is the recommendations


Thanks all in advance for your ideas

[ArndW]

Transform PX stages can take a long time to generate and compile. On some systems the C++ compiler has a concurrent user limit, so you not only need to wait the time for the compile but you might also be waiting for someone else to finish using a C++ license.

I have waited 10-15 minutes for a compile to finish on a system that didn't seem overly loaded. How long have you waited?

You can also start the compile and use truss (or equivalent) to monitor what the background process is doing; that lets you see if it is waiting, hanging or perhaps even aborted.

[pbatchu]

Thanks for the reply

But i have waited for most than 4 hours for it to complete with no success

It actually died on my with a error which says

Compilation failed - no other information available

Hence my question on the limitation of the transformer


Thanks again

Arvind

[ArndW]

I am sure that there is a limitation on the number of columns, but it is certainly higher than what you currently have.

I am confused, you state that the compile hangs forever, yet you have a message "compile failed" - which is it? If the compile failed, there will be additional error messages.

[balajisr]

Arnd,

I am sure that there is a limitation on the number of columns, but it is certainly higher than what you currently have.

Can you please tell the limit.

[ray.wurlod]

No, because IBM don't document that. We know there must be a limit because we're software engineers, and there's no such thing as unlimited memory - you always design maximum sizes for things. Years later, you wonder why you thought so small - was it not a young Bill Gates who asserted that 640KB is all the memory you will ever need?

[ArndW]

No, I don't know the limit. But I just tested a job with 1000 columns and derivations and it compiled and ran without a problem. The compilation took about 10 minutes and as a comparison my current PX job compiles in about 30 seconds ( ~80 stages, no transforms).

[balajisr]

Thanks Arnd and Ray for clarifying this.

[aramachandra]

Actually it is a bit of both

the compilation hangs in my opinion in that it runs for foreever

Due to concurrent licensing issues we have a process that goes and kills sessions that are not active for over 2 hours

So while i work i am looking at it on my other screen still saying it is compiling please wait

I leave for home and log on from home, which is after two hours and i see that script has killed my session and hence the compilation is terminated


So i think it hangs but it is killed behind the scenes by our inactive session kill script

But again that is well past 4 hours since i kicked off the compilation