Page 1 of 1
To handle empty file/only hd tr file
Posted: Fri May 16, 2014 4:33 am
by ajithaselvan
Hi,
My source is fixed width file with below format.
H00014700
Ddewuuwq
T00014720
I have below scenarios for rejections. If input file falls under any one below, then I have to update another table with file name and reject reason
1. Empty file (zero rows)
2. Only header and trailer
3. Only Detail
4. More than one header/trailer
any idea how to handle?
Posted: Fri May 16, 2014 5:09 am
by ArndW
There isn't any 1-click type of solution for this, unfortunately.
There are various ways to handle this, either from the command line using utilities such as awk or via a subroutine call which uses BASIC or a call to C-code. But if you wish to do this exclusively inside a DataStage PX job you would need to read the file and then sort and aggregate on your first 1-letter column.
- If there are more than 1 "H" rows, error
- If there are more than 1 "T" rows, error
- If the "H" row is not row number 1 then error
- If the "T" row is not the highest row number, then error
- If the "D" row count is 0, then error
Posted: Fri May 16, 2014 6:17 am
by chulett
Also realize that you cannot recognize an empty file inside a job to process it and note that somehow. That one you'll need to check separately. Elsewhere.
Posted: Fri May 16, 2014 11:26 am
by rameshrr3
External source stage, call:
This shoudl give zero if zero detail records
Similarly
and
for your file, these will give you counts of headers and trailers into your jobs data stream - use a funnel to combine these streams ( There could be a creative way in unix itself , using echo & awk to call all 3 greps in a single line) and parse it in a transformer
maybe like :
Code: Select all
echo "DETAIL" `grep -c ^D file.txt` '\n' "HEADER" `grep -c ^H file.txt` '\n' "TRAILER" `grep -c ^T file.txt`
I would advise you not to waste more time on development using sorts aggregators etc - when shell one-liners do it much faster.
Posted: Fri May 16, 2014 11:36 am
by qt_ky
The unix test -s tests if the specified file exists and has a size greater than zero.
Posted: Sun May 18, 2014 6:07 pm
by ssnegi
This will create an empty file if it does not exist and then do a line count.
empty file will return 0.
Posted: Sun May 18, 2014 9:23 pm
by chulett
Why? There's no need to create a file that isn't needed nor a need to count all of the rows in it. The test Eric posted will tell you in a simple true/false manner if the file exists and is not empty, with true moving on to more complex tests and false causing a reject. If you need to differentiate between 'exists' and 'empty' do a simple test for existence first, then test for empty as noted.