Lucky wrote:
Important thing is, the file will have only one Header and one Trailer records.
Ok now you have lost me again. Do you mean that you are processing one file at a time and each file only has one header record and one trailer record? If that is the case then do as Ray suggests, just check the file for a header and trailer using a shell script. And if you wanted to you could also top and tail the file in that script and just process the detail information.
Lucky wrote:
Yes I meant Sort stage. I agree with your suggestion about the performance issues if we use a sort stage, but I am very new to data stage and still getting used it and still exploring the options available in DS. I have worked on Informatica so I am using that knowledge to create jobs in data stage. Having said that each tool is different in its own way, as you know.
1. Agree with you that there is no need to use Sorter Stage as all the records are being collected into an intermediate file which is then being used as a Source.
2. I should process one file at a time and the file will have one Header and one Trailer record.
3. Platform being used is Windows. So I should go for batch scripts to check for the characters 'T', 'H' & 'D' by using find /c /i "T" command but I am not sure if I can rely on the script.
Hi Lucy,
If you were able to pre process, its fine. Else as the way Shane mentioned, have three streams from the transformer, each one for H, D and T.
And do a lookup in later stage for availability of both H and T.
Btw : Its 'Sort' in Datastage.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Take two passes through the file. The first does nothing other than verify that each header record has detail and trailer records that belong to it. This is all done with stage variables and relies on the file being sorted (use a Sort stage if you need it). It might also populate a hashed file from the trailer records (use a constraint - record type = "T").
The second pass, in a separate job, ought then to be straightforward. But, since we still have no indication of your desired output format(s), it's difficult to suggest anything.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
What action needs to be taken if the H and T are not present?
If you want the file should be taken for processing only if it has H and T, then you can write control script around, which will use something like grep command to check if H and T are there. If yes, call job with this filename.
Hi All, this closely resembles my case but I have Header->Detail(s)->Trailer structure repeating in my source file. Target file would also have to be written the same order. How do I go about creating this ?
Any help on this is appreciated !
I suggest starting a new post with a link referring to this particular post for referrence. This way we will know job type, os type and other information that will help others provide a solution.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
