Is there an equivalent of Final Delimiter (Of Parallel Extender' Seq. File), in server's Seq. File Stage.
Actually my input file contains rows from different files, and each record's first field specifies the file to which it belongs. all the columns are delimited by Pipe (|). if there is an equivalent of final delimiter then i can take the second field to different files, depending on the file name in transformer stage.
Please send a copy of reply to my email id also.
Regards,
Mrinal
email ID- Mrinal_Kumar01@infosys.com
Final Delimiter in Seq. File Stage of DS Server
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 85
- Joined: Fri Jun 04, 2004 2:30 am
- Location: Melbourne, Australia
- Contact:
I haven't come across a 'final delimiter' in DS Server, but from what I understand you are trying to do you don't need it.
If your first field contains 'filename' then you can output the desired results for each file with a Constraint in a transformer stage.
A second set of transformers for each different file (if the format is different for each file) can break out the pipe-delimited fields relavant to each file.
If your first field contains 'filename' then you can output the desired results for each file with a Constraint in a transformer stage.
A second set of transformers for each different file (if the format is different for each file) can break out the pipe-delimited fields relavant to each file.
Cheers,
Dave Nemirovsky
Dave Nemirovsky
what should be column definition for the first input file
all the different files have different columns, and all are pipe separated. even the input file is pipe separated only.
djhigh wrote:I haven't come across a 'final delimiter' in DS Server, but from what I understand you are trying to do you don't need it.
If your first field contains 'filename' then you can output the desired results for each file with a Constraint in a transformer stage.
A second set of transformers for each different file (if the format is different for each file) can break out the pipe-delimited fields relavant to each file.
-
- Participant
- Posts: 122
- Joined: Mon Jul 05, 2004 1:33 pm
- Location: MA
Re: what should be column definition for the first input fil
Hi,
If I am right, your input file is a follows
File1|data......
File2|data......
....................
....................
FileN|data......
and as your output there would be N files with the appropriate "data".
If you are sure about the number and name of output files, then I guess you could have a transformer with just two columns Col1 which would read File1, File2,... FileN and Col2 which would read the rest of the data till end of line.
If you are not sure about the number and name of output files, then you would require a routine that reads the Input file, based on the the first column create / open the file, write records / go to end of file and write record.
The previous post from David also speaks about the same, I guess.
Sonia
If I am right, your input file is a follows
File1|data......
File2|data......
....................
....................
FileN|data......
and as your output there would be N files with the appropriate "data".
If you are sure about the number and name of output files, then I guess you could have a transformer with just two columns Col1 which would read File1, File2,... FileN and Col2 which would read the rest of the data till end of line.
If you are not sure about the number and name of output files, then you would require a routine that reads the Input file, based on the the first column create / open the file, write records / go to end of file and write record.
The previous post from David also speaks about the same, I guess.
Sonia
Re: what should be column definition for the first input fil
Actually the file is in this format:
File1|Field11|Field12|Field13..............|Field1N
File2|Field21|Field22|Field23|Field24........|Field2M
File1|Field11|Field12|Field13..............|Field1N
File1|Field11|Field12|Field13..............|Field1N
File2|Field21|Field22|Field23|Field24........|Field2M
File2|Field21|Field22|Field23|Field24........|Field2M
File2|Field21|Field22|Field23|Field24........|Field2M
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
Depending on the file name in the first field, i want the otherfields into different files , File1, File2, file3.
File1|Field11|Field12|Field13..............|Field1N
File2|Field21|Field22|Field23|Field24........|Field2M
File1|Field11|Field12|Field13..............|Field1N
File1|Field11|Field12|Field13..............|Field1N
File2|Field21|Field22|Field23|Field24........|Field2M
File2|Field21|Field22|Field23|Field24........|Field2M
File2|Field21|Field22|Field23|Field24........|Field2M
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
File3|Field31|Field32|..........................|Field3O
Depending on the file name in the first field, i want the otherfields into different files , File1, File2, file3.
-
- Participant
- Posts: 85
- Joined: Fri Jun 04, 2004 2:30 am
- Location: Melbourne, Australia
- Contact:
So as I stated before just output the desired results for each file with a Constraint in a transformer stage. 3 output links (1 for each file) and based on field1 the row goes to the appropriate file.
A second set of transformers for each file splits the pipe-delimited fields for that file.
Is that answering your question or am I missing the point?
A second set of transformers for each file splits the pipe-delimited fields for that file.
Is that answering your question or am I missing the point?
Cheers,
Dave Nemirovsky
Dave Nemirovsky
Looks like you're getting there...
Read the entire row in one column. Use the FIELD function to find the first column and send the remainder of the row as ONE column out the appropriate link of the transform to a ROW SPLITTER stage.
The ROW SPLITTER stage will then automatically split the one column into several columns, based on the delimiter. Read up on the ROW SPLITTER stage. This is exactly the case it was created for: changing column meta data "on the fly" without landing the data to a file first, or without doing a ton of FIELD commands to walk throw the fields.
-Craig
Read the entire row in one column. Use the FIELD function to find the first column and send the remainder of the row as ONE column out the appropriate link of the transform to a ROW SPLITTER stage.
The ROW SPLITTER stage will then automatically split the one column into several columns, based on the delimiter. Read up on the ROW SPLITTER stage. This is exactly the case it was created for: changing column meta data "on the fly" without landing the data to a file first, or without doing a ton of FIELD commands to walk throw the fields.
-Craig