Hi;
thanks Ray and Craig and sorry for short,
i posted this problem before with the job desc, and what i'm trying to do is according to Ernie advise (thanks Ernie), and i have exactly this error,
so i sked according to that error.
Craig: i have tryed the URL/File path but it does't work with me, and if so can you please explain how cat it improve the job perforance.
the job design is like this:
FolderStage --> transformerStage --> XML inputStage --> transformerStage --> 2 SequentialFile + other stuff including pivot stage, >90% of the job time is between
FolderStage --> transformerStage --> XML inputStage --> transformerStage --> 2SequentialFile,
and yes i'm going down to the /text() for all my columns and they are around 255 columns.
"the job is running on monthly basis"
this is the link for my previous post on this issue.
viewtopic.php?p=219830&highlight=&sid=b ... 25f#219830
is there any way to increase the inter stag rowbuffer?
Moderators: chulett, rschirm, roy
What exactly "doesn't work" about it?waklook wrote:i have tryed the URL/File path but it does't work with me
Remove the first transformer, it can't be providing any value, only a 'chokepoint' in the processing. Go directly from the Folder stage to the XML Input stage and see how that helps. Make sure you are using the URL / File path option.waklook then wrote:the job design is like this:
FolderStage --> transformerStage --> XML inputStage --> transformerStage --> 2 SequentialFile + other stuff including pivot stage
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Hi guys;
Craig:
this is what happen with me when i run the job for only 10,000 xml file:
using the old job using 'XML DOCUMENT' option and
using your advise 'URL/FILE PATH' the load time was very close (arount 52Min),so i tryed
using LinkPart&LinkColl with 8 shared Containers(containing only the xml input stage) plus using your advise 'URL/FILE PATH' the load time was around 14Min and it seems huge difference, i will go for the second one unless one of you guys have better idea.
i used 8 shared containers bcz we have 8CPUs.
Many thanks for all of you guys specialy for those who gave me the advises.
Craig:
this is what happen with me when i run the job for only 10,000 xml file:
using the old job using 'XML DOCUMENT' option and
using your advise 'URL/FILE PATH' the load time was very close (arount 52Min),so i tryed
using LinkPart&LinkColl with 8 shared Containers(containing only the xml input stage) plus using your advise 'URL/FILE PATH' the load time was around 14Min and it seems huge difference, i will go for the second one unless one of you guys have better idea.
i used 8 shared containers bcz we have 8CPUs.
Many thanks for all of you guys specialy for those who gave me the advises.