Hi all
i am trying to read multiple files from a folder and then writing it to a single table. is there any way to identify, if a row is rejected, which file it belonged to? my guess would be if i am able to load the name of the file while reading the file i can capture the name with the rejected row. not sure how though. maybe the name of the file in one column and the rest of its content in another?
thanks
Identification of a rejected row
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 197
- Joined: Sun Jul 15, 2007 11:45 pm
- Location: Prague
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Premium Member
- Posts: 197
- Joined: Sun Jul 15, 2007 11:45 pm
- Location: Prague
There is a property called File Name Column in sequential file stage. Try that out. You cannot propagate the field to the target stage. So probably take a steam out using copy stage and join them back with the rejects link records and then write to the target file.
"given enough eyeballs, all bugs are shallow" - Eric S. Raymond
mahadev.v,
Did you convey this message to DS_FocusGroup that
Enabling option "File Name Column" in the Sequential file Stage will require an Environment variable APT_IMPORT_PATTERN_USES_FILESET to give the File individual file name in the specified column(or default fileNameColumn) when file pattern is selected for retrieving multiple files from a folder with matching the specified pattern.
Did you convey this message to DS_FocusGroup that
Enabling option "File Name Column" in the Sequential file Stage will require an Environment variable APT_IMPORT_PATTERN_USES_FILESET to give the File individual file name in the specified column(or default fileNameColumn) when file pattern is selected for retrieving multiple files from a folder with matching the specified pattern.
After getting a field as Source file name from the Sequential file Stage, you can further use Remove Duplicate logic in transformer or Sort-filter as to seperate the duplicate records from the Source.DS_FocusGroup wrote:rejected rows can be defined as duplicate rows in the source which are dropped while being loaded or based on some lookup any rejected records.
-
- Premium Member
- Posts: 197
- Joined: Sun Jul 15, 2007 11:45 pm
- Location: Prague