Header & Footer is missing

karthi_gana · Post by **karthi_gana** » Mon Mar 28, 2011 8:07 am

All,

I have designed a simple job as below

Seq File --> Transformer --> FTP Stage (Plug in)

Sample file :

UHDR|2011087
1234|201101|2.67565
1234|201101|2.67565
1234|201101|2.67565
UTRL|5

when i run the job, i see only

1234|201101|2.67565
1234|201101|2.67565
1234|201101|2.67565

in the remote machine.

i.e. Header & Trailer is missing in the file (once the file FTP'ed) . why?

I don't see any info in the log. i.e. The job ran successfully.

chulett · Post by **chulett** » Mon Mar 28, 2011 8:13 am

Assuming your metadata matches the detail records and thus the others were silently dropping during import for metadata mismatches. Define the file as one long string instead.

FranklinE · Post by **FranklinE** » Mon Mar 28, 2011 10:01 am

I deal primarily with COBOL datasets that have multiple record types in addition to header and trailer, so I've given some thought to what would be either easiest or least confusing.

Ideally, your header and trailer records would have as many delimiters as your detail records. That is usually unlikely, so the compromise is towards Craig's suggestion: first column identifies how much of the beginning of each record you need to identify what type it is, then have one more column for the rest of the record. Use the transformer to parse by type (substrings, usually, but index() works well with delimiters), branch off the header and trailer records and sequential-funnel them back just before the FTP stage.

Edit: The above assumes you have derivation or other editing to do on individual columns in the detail rows. Of course, if you are just passing data through, you don't have to have separate process branches.

chulett · Post by **chulett** » Mon Mar 28, 2011 10:04 am

Doesn't even need to be that 'complex', just read and send it as a varchar with no quotes and all of the records should be transferred intact.

karthi_gana · Post by **karthi_gana** » Mon Mar 28, 2011 11:22 am

chulett wrote:Doesn't even need to be that 'complex', just read and send it as a varchar with no quotes and all of the records should be transferred intact.

I think i dindn't give the complete information. As you said, I just stored all the values in a single column only.

i.e.

report varchar(500)

this columns will have all the values.

reports

UHDR|2011087
1234|201101|2.67565
1234|201101|2.67565
1234|201101|2.67565
UTRL|5

i used the same column in the ftp stage also. i.e. reports varchar(500).

karthi_gana · Post by **karthi_gana** » Mon Mar 28, 2011 11:24 am

chulett wrote:Define the file as one long string instead.

yes.

report varchar(500)

with no quotes.

ray.wurlod · Post by **ray.wurlod** » Mon Mar 28, 2011 11:31 am

Go back to your orignal design and add a Reject link to the Sequential File stage that reads the file. This will capture the header and trailer rows and begin a stream of processing in which you can do what you like, for example re-parse with a Column Import stage.

chulett · Post by **chulett** » Mon Mar 28, 2011 12:48 pm

Are you saying your metadata is one varchar(500) column throughout the job and yet you still lose the header & footer? That doesn't make sense to me.

karthi_gana · Post by **karthi_gana** » Tue Mar 29, 2011 12:35 am

chulett wrote:Are you saying your metadata is one varchar(500) column throughout the job and yet you still lose the header & footer?

yes.

i am unable to identify the logic hole in the job.

chulett · Post by **chulett** » Tue Mar 29, 2011 7:05 am

Then try Ray's approach to see if they are getting rejected for some reason.