Remove header and footer(first and last line)
Moderators: chulett, rschirm, roy
Remove header and footer(first and last line)
Hi all,
My source file having header in first line and tailer in last line
so i want to remove first and last record using DS(parallel Extender ) if any one know please suggest
Regards
Ujjwal
My source file having header in first line and tailer in last line
so i want to remove first and last record using DS(parallel Extender ) if any one know please suggest
Regards
Ujjwal
Write a ksh script that uses the results of a wc -l to issue the appropriate head and tail commands, streaming to stdout. Put that ksh into the filter on the Sequential stage.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
-
- Premium Member
- Posts: 614
- Joined: Fri Feb 06, 2004 3:59 pm
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I don't recall the 'reject method' causing any warnings. It will rather happily process any records that fail the metadata check into oblivion without raising any kind of a fuss.
Perhaps it's only after you actually setup a reject link? In this case, it doesn't sound like that would be needed.
![Wink :wink:](./images/smilies/icon_wink.gif)
Perhaps it's only after you actually setup a reject link? In this case, it doesn't sound like that would be needed.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
But i wonder how iam getting the following warining if i approach reject mechanism.
Is there any other smarter approach to get rid of Short input record - read.
-Kumar
Code: Select all
SEQ,0: Short input record
SEQ,0: Import warning at record 0.
SEQ,0: Import unsuccessful at record 0.
-Kumar
This is a point of style.
Relying on the stage to behave poorly (IMO), meaning to silently toss data, is poor style.
You should affirmatively handle all data. If you don't want a row, build the logic to purposely remove the remove. If yo do want specific rows, build the logic to purposely include only those rows.
Letting the job "lose" data because it's the header and footer means that potentially in the future Ascential could change the behavior of the stage and now because you don't have affirmative logic unexpected things start happening.
For example, if you use a constraint to say you only want rows where a code value is A thru Z except for X, a lazy programmer might say: inlink.column NE 'X'. This serves the purpose that Xs don't make it in, but also allows '!' and 'x' and 'XX' to go thru. The affirmative logic would be to name all values A to Z except X in the constraint.
It's style but the korn shell script is a reusable solution. It's a generic script that can always be used to strip off the header and trailer record. It's even more beneficial if the header and trailer has row counts and checksum information, because then the script can validate the file first and return a non-zero value stopping the job if the file is corrupted. See the expandabilities?
Relying on the stage to behave poorly (IMO), meaning to silently toss data, is poor style.
You should affirmatively handle all data. If you don't want a row, build the logic to purposely remove the remove. If yo do want specific rows, build the logic to purposely include only those rows.
Letting the job "lose" data because it's the header and footer means that potentially in the future Ascential could change the behavior of the stage and now because you don't have affirmative logic unexpected things start happening.
For example, if you use a constraint to say you only want rows where a code value is A thru Z except for X, a lazy programmer might say: inlink.column NE 'X'. This serves the purpose that Xs don't make it in, but also allows '!' and 'x' and 'XX' to go thru. The affirmative logic would be to name all values A to Z except X in the constraint.
It's style but the korn shell script is a reusable solution. It's a generic script that can always be used to strip off the header and trailer record. It's even more beneficial if the header and trailer has row counts and checksum information, because then the script can validate the file first and return a non-zero value stopping the job if the file is corrupted. See the expandabilities?
Last edited by kcbland on Sat Nov 19, 2005 8:46 am, edited 1 time in total.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
dspxlearn - Please don't hijack other peoples threads. If you've got an new question, start a new post.
Ken, agree on your 'point of style'. Was primarily looking for a clarification / confirmation on how the stage actually works as I don't have access to a PX server. Just have a Ray Wurlod Guinea Pig Merit Badge, Texas Chapter.![Wink :wink:](./images/smilies/icon_wink.gif)
Ken, agree on your 'point of style'. Was primarily looking for a clarification / confirmation on how the stage actually works as I don't have access to a PX server. Just have a Ray Wurlod Guinea Pig Merit Badge, Texas Chapter.
![Wink :wink:](./images/smilies/icon_wink.gif)
Last edited by chulett on Sat Nov 19, 2005 1:57 pm, edited 1 time in total.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: