blank rows in the file

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
madhukar
Participant
Posts: 86
Joined: Fri May 20, 2005 4:05 pm

blank rows in the file

Post by madhukar »

hi all
hw to identify and remove blank rows from the source file
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Hello madhukar,

with Px you can either use the filter command to run it through the shell script or command of your choice, or you can read the empty row and then remove it from the output stream with a filter stage, or a transform stage, or a modify stage.
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Does a blank row imply row with spaces to the record length or not even the spaces?
elavenil
Premium Member
Premium Member
Posts: 467
Joined: Thu Jan 31, 2002 10:20 pm
Location: Singapore

Post by elavenil »

Filter command can be executed in the Server edition as well.

Regards
Saravanan
dsxdev
Participant
Posts: 92
Joined: Mon Sep 20, 2004 8:37 am

Post by dsxdev »

Hi madhur,
Can you be clear whether your balnk line means something like this

XXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXAAA

or
XXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXAAA

In first case no need to wory as these records will be rejected by the sequential file stage if you want these records to be trapped then
use a reject link from sequntial files.

seq---------->Trns---------->.......
|
|
|
V
seq (reject)

Reject file contains all teh blank records.
Happy DataStaging
sonia jacob
Participant
Posts: 122
Joined: Mon Jul 05, 2004 1:33 pm
Location: MA

Post by sonia jacob »

I was searching for a solution on a similar problem and found this post. In my experiance the job still throws warnings even when there is a reject link! Would that be right or am i doing something wrong.

dsxdev wrote:Hi madhur,
Can you be clear whether your balnk line means something like this

XXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXAAA

or
XXXXXXXXXXXXXXXXXX

XXXXXXXXXXXXXXXAAA

In first case no need to wory as these records will be rejected by the sequential file stage if you want these records to be trapped then
use a reject link from sequntial files.

seq---------->Trns---------->.......
|
|
|
V
seq (reject)

Reject file contains all teh blank records.
Regards
Sonia Jacob
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Use the filter command and use the sed operation to find and delete empty lines.
Put this in the filter command box

Code: Select all

sed -e '/^$/d' 
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Post Reply