To handle empty file/only hd tr file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 75
- Joined: Mon Jul 12, 2010 4:11 am
- Location: Chennai
To handle empty file/only hd tr file
Hi,
My source is fixed width file with below format.
H00014700
Ddewuuwq
T00014720
I have below scenarios for rejections. If input file falls under any one below, then I have to update another table with file name and reject reason
1. Empty file (zero rows)
2. Only header and trailer
3. Only Detail
4. More than one header/trailer
any idea how to handle?
My source is fixed width file with below format.
H00014700
Ddewuuwq
T00014720
I have below scenarios for rejections. If input file falls under any one below, then I have to update another table with file name and reject reason
1. Empty file (zero rows)
2. Only header and trailer
3. Only Detail
4. More than one header/trailer
any idea how to handle?
Ajitha S
There isn't any 1-click type of solution for this, unfortunately.
There are various ways to handle this, either from the command line using utilities such as awk or via a subroutine call which uses BASIC or a call to C-code. But if you wish to do this exclusively inside a DataStage PX job you would need to read the file and then sort and aggregate on your first 1-letter column.
- If there are more than 1 "H" rows, error
- If there are more than 1 "T" rows, error
- If the "H" row is not row number 1 then error
- If the "T" row is not the highest row number, then error
- If the "D" row count is 0, then error
There are various ways to handle this, either from the command line using utilities such as awk or via a subroutine call which uses BASIC or a call to C-code. But if you wish to do this exclusively inside a DataStage PX job you would need to read the file and then sort and aggregate on your first 1-letter column.
- If there are more than 1 "H" rows, error
- If there are more than 1 "T" rows, error
- If the "H" row is not row number 1 then error
- If the "T" row is not the highest row number, then error
- If the "D" row count is 0, then error
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
External source stage, call:
This shoudl give zero if zero detail records
Similarly and
for your file, these will give you counts of headers and trailers into your jobs data stream - use a funnel to combine these streams ( There could be a creative way in unix itself , using echo & awk to call all 3 greps in a single line) and parse it in a transformer
maybe like :
I would advise you not to waste more time on development using sorts aggregators etc - when shell one-liners do it much faster.
Code: Select all
grep -c ^D #srcfile#
Similarly
Code: Select all
grep -c ^H
Code: Select all
grep -c^T
maybe like :
Code: Select all
echo "DETAIL" `grep -c ^D file.txt` '\n' "HEADER" `grep -c ^H file.txt` '\n' "TRAILER" `grep -c ^T file.txt`
Last edited by rameshrr3 on Fri May 16, 2014 11:46 am, edited 1 time in total.
The unix test -s tests if the specified file exists and has a size greater than zero.
Code: Select all
if [ -s /path/file ] then ...
Choose a job you love, and you will never have to work a day in your life. - Confucius
Code: Select all
touch filename.txt | wc -l
empty file will return 0.
Why? There's no need to create a file that isn't needed nor a need to count all of the rows in it. The test Eric posted will tell you in a simple true/false manner if the file exists and is not empty, with true moving on to more complex tests and false causing a reject. If you need to differentiate between 'exists' and 'empty' do a simple test for existence first, then test for empty as noted.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers