Repeating Headers

harshada · Post by **harshada** » Tue Apr 05, 2011 11:23 am

Hi,

I know that DS is not a reporting tool, but we donot need to do any fancy formatting, just insert repeating headers after certain records in a file.

One way I can think of is call a after sub-routine and call a unix shell script to insert the required details after every say 50 records.

Is there a better way for doing this.

Thanks,
Harshada

jwiles · Post by **jwiles** » Tue Apr 05, 2011 2:24 pm

You could add a filter to the sequential file output stage (using the Filter option), callling a script or some other program to create your headers. This would be the simplest solution for your example of one header every 50 records. A similar solution would be to use an external target stage.

There are other options that have been discussed here in the forum as well.

Regards,

harshada · Post by **harshada** » Tue Apr 05, 2011 3:04 pm

Hi James,

I tried to use the external target stage for the first time. It says 'It allows you to write data to one or more source programs', but my requirement is to insert repeating headers, so not sure how it is fulfilling that.
Also I could not find how to insert multiple headers in a flat file. Forum discusses about xml files etc and single header, trailer creations in a flat file.

Could you please help me more on this.

Regards,
Harshada

ray.wurlod · Post by **ray.wurlod** » Tue Apr 05, 2011 3:47 pm

Perhaps you could post an example of what you mean?

jwiles · Post by **jwiles** » Tue Apr 05, 2011 11:06 pm

You are correct, the external target stage allows you to write data to a source program, or in other words a program/script/something external to DataStage. The data you're writing (sending) to the external program is, I certainly hope, the records you have processed in your job.

What you do with those records within that external program is up to you. It sounds to me like you want to insert header records, so have your external program do that...read and write X number of records, then insert a header record and repeat. How it does that is up to you and the exact details depend on the language it's written in (C/C++/Perl/awk/Basic/Rexx/java/whatever). Read from standard input and write to your file. The filter option for SeqFile is similar, but write to standard output instead.

You mentioned an after-job subroutine, or unix shell script. DataStage provides you a method of calling that within the context of the parallel job, as an "external target", rather than post-processing the file after the job has completed.

Regards,

arunkumarmm · Post by **arunkumarmm** » Wed Apr 06, 2011 11:40 am

If your requirement is to just add a header after every 50 records, you can create a dummy column say SeqNum and assign values like, first fifty as 2, 2nd fifty as 4. In a separate file create all the header records you want with SeqNum column with values like 1,3,5. Then merge both the files and sort it with SeqNum.