DSXchange

All,

I have designed a simple job as below

Seq File --> Transformer --> FTP Stage (Plug in)

Sample file :

UHDR|2011087
1234|201101|2.67565
1234|201101|2.67565
1234|201101|2.67565
UTRL|5

when i run the job, i see only

1234|201101|2.67565
1234|201101|2.67565
1234|201101|2.67565

in the remote machine.

i.e. Header & Trailer is missing in the file (once the file FTP'ed) . why?

I don't see any info in the log. i.e. The job ran successfully.

Assuming your metadata matches the detail records and thus the others were silently dropping during import for metadata mismatches. Define the file as one long string instead.

I deal primarily with COBOL datasets that have multiple record types in addition to header and trailer, so I've given some thought to what would be either easiest or least confusing.

Ideally, your header and trailer records would have as many delimiters as your detail records. That is usually unlikely, so the compromise is towards Craig's suggestion: first column identifies how much of the beginning of each record you need to identify what type it is, then have one more column for the rest of the record. Use the transformer to parse by type (substrings, usually, but index() works well with delimiters), branch off the header and trailer records and sequential-funnel them back just before the FTP stage.

Edit: The above assumes you have derivation or other editing to do on individual columns in the detail rows. Of course, if you are just passing data through, you don't have to have separate process branches.

Doesn't even need to be that 'complex', just read and send it as a varchar with no quotes and all of the records should be transferred intact.

chulett wrote:Doesn't even need to be that 'complex', just read and send it as a varchar with no quotes and all of the records should be transferred intact.

I think i dindn't give the complete information. As you said, I just stored all the values in a single column only.

i.e.

report varchar(500)

this columns will have all the values.

reports

UHDR|2011087
1234|201101|2.67565
1234|201101|2.67565
1234|201101|2.67565
UTRL|5

i used the same column in the ftp stage also. i.e. reports varchar(500).

chulett wrote:Define the file as one long string instead.

yes.

report varchar(500)

with no quotes.

Go back to your orignal design and add a Reject link to the Sequential File stage that reads the file. This will capture the header and trailer rows and begin a stream of processing in which you can do what you like, for example re-parse with a Column Import stage.

Are you saying your metadata is one varchar(500) column throughout the job and yet you still lose the header & footer? That doesn't make sense to me.

chulett wrote:Are you saying your metadata is one varchar(500) column throughout the job and yet you still lose the header & footer?

yes.

i am unable to identify the logic hole in the job.

Then try Ray's approach to see if they are getting rejected for some reason.