Source records equal to trailer records
Moderators: chulett, rschirm, roy
Source records equal to trailer records
HI,
This is the sequential source
Date,Cusip,Security,Speed,SpeedType,Yield,Duration,Moody,SP,MajorType
9/20/2005,79548KP77,SBM7 1999-C1 B,0,CPY,4.63,3.004,Aaa,,
9/20/2005,036234190,.GSR 2005-AR6 1A1,,,4.59,1.9,,,
TRAILER,2,Wed Sep 21 05:00:00 EDT 2005,,,,,,,
The following condition
If No.Source records = Trailer count then pass into another Target (sequential) file. If condition not satisfies then abort the job, it does not insert the records in the target file.
Please tell me any one know, How do I will implement this logic?
Thanks
Sri Hari
This is the sequential source
Date,Cusip,Security,Speed,SpeedType,Yield,Duration,Moody,SP,MajorType
9/20/2005,79548KP77,SBM7 1999-C1 B,0,CPY,4.63,3.004,Aaa,,
9/20/2005,036234190,.GSR 2005-AR6 1A1,,,4.59,1.9,,,
TRAILER,2,Wed Sep 21 05:00:00 EDT 2005,,,,,,,
The following condition
If No.Source records = Trailer count then pass into another Target (sequential) file. If condition not satisfies then abort the job, it does not insert the records in the target file.
Please tell me any one know, How do I will implement this logic?
Thanks
Sri Hari
-
- Premium Member
- Posts: 68
- Joined: Sat Feb 05, 2005 4:59 am
- Location: India
One option could be process the source file in the job control
Count the number of lines<excluding the first and last> and compare this with the count field of the trailer record. If they dont- call jobAbort.
There are also inbuilt routines that count the number of lines in datastage 7.5 which u cud use.
Count the number of lines<excluding the first and last> and compare this with the count field of the trailer record. If they dont- call jobAbort.
There are also inbuilt routines that count the number of lines in datastage 7.5 which u cud use.
adarsh shrinagesh wrote:One option could be process the source file in the job control
Count the number of lines<excluding the first and last> and compare this with the count field of the trailer record. If they dont- call jobAbort.
There are also inbuilt routines that count the number of lines in datastage 7.5 which u cud use.
Hi Adarsh,
Iam not get the ur point,please explain clearly.
Thanks
-
- Premium Member
- Posts: 68
- Joined: Sat Feb 05, 2005 4:59 am
- Location: India
U could either create a routine<which wud then be called in the before stage subroutine> or do some preprocessing in the Job Control ... where u could read the sequential file and count the number of rows and compare it with the value of the Trailer record's 2nd field.dslearner wrote:adarsh shrinagesh wrote:One option could be process the source file in the job control
Count the number of lines<excluding the first and last> and compare this with the count field of the trailer record. If they dont- call jobAbort.
There are also inbuilt routines that count the number of lines in datastage 7.5 which u cud use.
Hi Adarsh,
Iam not get the ur point,please explain clearly.
Thanks
Either ways u cud call DSLogFatal to abort the job if the condition fails(ie no of records counted<> no of records mentioned in the trailer record>
-
- Participant
- Posts: 232
- Joined: Sat May 07, 2005 2:49 pm
- Location: USA
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You can pre-process in UNIX (perhaps through a before-job subroutine).
The number of detail lines in the file is given by expr `wc -l $file` - 2 while the number from the trailer line is given by tail -1 $file | cut -d',' -f2,2
The number of detail lines in the file is given by expr `wc -l $file` - 2 while the number from the trailer line is given by tail -1 $file | cut -d',' -f2,2
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 252
- Joined: Mon Sep 19, 2005 10:28 pm
- Location: Melbourne, Australia
- Contact:
We are doing the exact same thing at our site. Incoming files have headers and trailers; if they are missing or contain incorrect numbers then we assume that the extract or transfer mechanism by which they arrived has failed and we do not process them. Whether you abort or skip depends on how important the file is and its dependencies.
We dealt with it as Ray suggested: with a Unix script before the DS job processes the file.
I wrote a generic Perl script that I call "File Receiver". It will:
- handle delimited or fixed width files by supplying an optional delimiter
- verify the existence of a header if required by checking for a key word
- verify the existence of a trailer if required
- verify a detail record count from a given column/char range in the trailer
- strip DOS carriage returns and EOF markers
- read the nominated file and pass detail records to a nominated output file.
If dslearner or anyone else is interested I would be happy to provide it free of charge (and free of liability, support, and responsibility). It is vanilla Perl and should run on any Unix system. It's probably a bit chunky to post here though; send me a private message if you want it emailed, if many want it I will post it on the web and leave the URL in this thread.
We dealt with it as Ray suggested: with a Unix script before the DS job processes the file.
I wrote a generic Perl script that I call "File Receiver". It will:
- handle delimited or fixed width files by supplying an optional delimiter
- verify the existence of a header if required by checking for a key word
- verify the existence of a trailer if required
- verify a detail record count from a given column/char range in the trailer
- strip DOS carriage returns and EOF markers
- read the nominated file and pass detail records to a nominated output file.
If dslearner or anyone else is interested I would be happy to provide it free of charge (and free of liability, support, and responsibility). It is vanilla Perl and should run on any Unix system. It's probably a bit chunky to post here though; send me a private message if you want it emailed, if many want it I will post it on the web and leave the URL in this thread.
Ross Leishman
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
It's currently at ADN, but stay tuned...
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.