Hi,
I have a DS job that read a input txt file of 4 GB and it's running for 15 hours and do nothing.
Can someone tell me if there is a limit or maximum size of a input txt file that DataStage van read?
Regards,
Erik
Maximum size of input text file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 22
- Joined: Thu Jun 02, 2005 5:12 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There is no limit on the size of a Sequential File that DataStage can read.
From the other posters' experiences I would suggest that the delay is not in reading the text file, but elsewhere in your job design.
You can prove this easily enough. Construct a job as follows.
Put @FALSE as the constraint on the Transformer stage output link and/or append the output to /dev/null. This will show you how fast DataStage can read the large file.
Everything you add from there will increase the total elapsed time.
From the other posters' experiences I would suggest that the delay is not in reading the text file, but elsewhere in your job design.
You can prove this easily enough. Construct a job as follows.
Code: Select all
Sequential File -----> Transformer -----> Sequential File
Everything you add from there will increase the total elapsed time.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 108
- Joined: Wed Jul 28, 2004 7:15 am
Please describe the job design, stages, etc. What if it's not stuck reading the file, but writing to the target? Is there a before/after stage/job routine call that's the true problem? Viewing the data on the Sequential file proves that the definitions are valid.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle