Hi
I have simple job which does the following
1) reads from the dataset
2) does remove duplicates to retain unique headers
3) does a lookup to pull in all the details for the headers
4) transforms each looked up column with
if IsNull(Link.Col) then '' else Link.Col
5) Writes to an ouptut dataset
WE are using px edition 7.5.1 on a sun solaris unix environment
The only thing unique is that IT HAS 305 COLUMNS AND almost 299 DERIVATION JUST LIKE THE one above
When i compile the job, the jobs hangs with Please Wait compile message box.
When i remove the transformer the jobs just compiles and runs just fine
Is there a limitation on the NUMBER of columns one can process in a transformer?
Has anybody else seen this kind of a situation in their unix environment
What if any is the recommendations
Thanks all in advance for your ideas
Compilation hangs forever
Moderators: chulett, rschirm, roy
Transform PX stages can take a long time to generate and compile. On some systems the C++ compiler has a concurrent user limit, so you not only need to wait the time for the compile but you might also be waiting for someone else to finish using a C++ license.
I have waited 10-15 minutes for a compile to finish on a system that didn't seem overly loaded. How long have you waited?
You can also start the compile and use truss (or equivalent) to monitor what the background process is doing; that lets you see if it is waiting, hanging or perhaps even aborted.
I have waited 10-15 minutes for a compile to finish on a system that didn't seem overly loaded. How long have you waited?
You can also start the compile and use truss (or equivalent) to monitor what the background process is doing; that lets you see if it is waiting, hanging or perhaps even aborted.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
I am sure that there is a limitation on the number of columns, but it is certainly higher than what you currently have.
I am confused, you state that the compile hangs forever, yet you have a message "compile failed" - which is it? If the compile failed, there will be additional error messages.
I am confused, you state that the compile hangs forever, yet you have a message "compile failed" - which is it? If the compile failed, there will be additional error messages.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
No, because IBM don't document that. We know there must be a limit because we're software engineers, and there's no such thing as unlimited memory - you always design maximum sizes for things. Years later, you wonder why you thought so small - was it not a young Bill Gates who asserted that 640KB is all the memory you will ever need?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
No, I don't know the limit. But I just tested a job with 1000 columns and derivations and it compiled and ran without a problem. The compilation took about 10 minutes and as a comparison my current PX job compiles in about 30 seconds ( ~80 stages, no transforms).
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 55
- Joined: Tue Sep 20, 2005 10:58 am
Actually it is a bit of both
the compilation hangs in my opinion in that it runs for foreever
Due to concurrent licensing issues we have a process that goes and kills sessions that are not active for over 2 hours
So while i work i am looking at it on my other screen still saying it is compiling please wait
I leave for home and log on from home, which is after two hours and i see that script has killed my session and hence the compilation is terminated
So i think it hangs but it is killed behind the scenes by our inactive session kill script
But again that is well past 4 hours since i kicked off the compilation
the compilation hangs in my opinion in that it runs for foreever
Due to concurrent licensing issues we have a process that goes and kills sessions that are not active for over 2 hours
So while i work i am looking at it on my other screen still saying it is compiling please wait
I leave for home and log on from home, which is after two hours and i see that script has killed my session and hence the compilation is terminated
So i think it hangs but it is killed behind the scenes by our inactive session kill script
But again that is well past 4 hours since i kicked off the compilation