Flat file header issue

[babbu9]

Hi
I have some data coming from VMS systems and to be loaded into a flat file. The data has header and detail information. I would like to do some computations with the data, like find out the total of all values in detail and compare with th total in header. I know that data in sequential flat files can handle only one format and not header and details. Can you please suggest a way to do the above function using a flat file or any other means..


Thanks

[arunverma]

Hi Babbu ,

If the source file format is fixed then you can use Unix script to filter out
and some computataion , we had also same problem . we are getting file which header contain header level date and time and bottom Status total number of record etc . then through Unix script first we chack file with proper format if ok then remove header and footer line and move to FLAT file .


Wth Regards

Arun Verma

[sanjay]

arun u are wrong u can't do that in datastage. person should know unix scripting

Sanjay

Hi Babbu ,

If the source file format is fixed then you can use Unix script to filter out
and some computataion , we had also same problem . we are getting file which header contain header level date and time and bottom Status total number of record etc . then through Unix script first we chack file with proper format if ok then remove header and footer line and move to FLAT file .


Wth Regards

Arun Verma

[arunverma]

I am right boss , I have given solution with using Unix script only ,
you create Unix script then call form DS Jobs .

Yes , user has to know Unix shell script .


Arun

[xcb]

If all you are trying to do is remove the header and trailing records without having to call any kind of shell scripts\commands. To achieve this design a job to have a single source sequential file stage with a single column defined which has the maximum length of any record found in your source file.

Have this output into a transformer where on the output side you define the metadata for your header, data and trailing records. Then to make sure that the right data goes to the right stream use a constraint to filter the records. There could be a couple of ways of doing this. One could be checking the record length of each record, in my experience the header and trailing records are almost always of different lengths to the detail records, so its easy to find. Another might be markers or record types actually in the record itself identifing the type as either header, detail or trailer. Either way you make your constraint accordingly.

Then finally you can output the data from the transformer to any format that suits, it might be sequential files, universe files or hash files it depends on what you are after. The result should be 3 or more output files where each file contains a separate record type.

Techniques similar to this have already been posted on this forum. Try searching to find other ways of achieving this.

[ray.wurlod]

You might also investigate the possibilities of the complex flat file (CFF) stage, which can do some of what you require.