Final Delimiter in Seq. File Stage of DS Server

[memrinal]

Is there an equivalent of Final Delimiter (Of Parallel Extender' Seq. File), in server's Seq. File Stage.
Actually my input file contains rows from different files, and each record's first field specifies the file to which it belongs. all the columns are delimited by Pipe (|). if there is an equivalent of final delimiter then i can take the second field to different files, depending on the file name in transformer stage.
Please send a copy of reply to my email id also.
Regards,
Mrinal
email ID- Mrinal_Kumar01@infosys.com

[davidnemirovsky]

I haven't come across a 'final delimiter' in DS Server, but from what I understand you are trying to do you don't need it.

If your first field contains 'filename' then you can output the desired results for each file with a Constraint in a transformer stage.

A second set of transformers for each different file (if the format is different for each file) can break out the pipe-delimited fields relavant to each file.

[memrinal]

all the different files have different columns, and all are pipe separated. even the input file is pipe separated only.

I haven't come across a 'final delimiter' in DS Server, but from what I understand you are trying to do you don't need it.

If your first field contains 'filename' then you can output the desired results for each file with a Constraint in a transformer stage.

A second set of transformers for each different file (if the format is different for each file) can break out the pipe-delimited fields relavant to each file.

[sonia jacob]

Hi,

If I am right, your input file is a follows

File1|data......
File2|data......
....................
....................

FileN|data......

and as your output there would be N files with the appropriate "data".

If you are sure about the number and name of output files, then I guess you could have a transformer with just two columns Col1 which would read File1, File2,... FileN and Col2 which would read the rest of the data till end of line.

If you are not sure about the number and name of output files, then you would require a routine that reads the Input file, based on the the first column create / open the file, write records / go to end of file and write record.

The previous post from David also speaks about the same, I guess.

Sonia

[crouse]

To add to Sonia's reply...

If you can detect the row type by the first column in the row via the transformer stage, then send the row to an applicable "Row Splitter" stage that can split the second column up into it's specific columns, without landing the data to a file first.

-Craig

[memrinal]

Actually the file is in this format:
File1|Field11|Field12|Field13..............|Field1N
File2|Field21|Field22|Field23|Field24........|Field2M
File1|Field11|Field12|Field13..............|Field1N
File1|Field11|Field12|Field13..............|Field1N
File2|Field21|Field22|Field23|Field24........|Field2M
File2|Field21|Field22|Field23|Field24........|Field2M
File2|Field21|Field22|Field23|Field24........|Field2M
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O

Depending on the file name in the first field, i want the otherfields into different files , File1, File2, file3.

[davidnemirovsky]

So as I stated before just output the desired results for each file with a Constraint in a transformer stage. 3 output links (1 for each file) and based on field1 the row goes to the appropriate file.

A second set of transformers for each file splits the pipe-delimited fields for that file.

Is that answering your question or am I missing the point?

[crouse]

Looks like you're getting there...

Read the entire row in one column. Use the FIELD function to find the first column and send the remainder of the row as ONE column out the appropriate link of the transform to a ROW SPLITTER stage.

The ROW SPLITTER stage will then automatically split the one column into several columns, based on the delimiter. Read up on the ROW SPLITTER stage. This is exactly the case it was created for: changing column meta data "on the fly" without landing the data to a file first, or without doing a ton of FIELD commands to walk throw the fields.

-Craig