Custom Delimiters

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
narasimha
Charter Member
Charter Member
Posts: 1236
Joined: Fri Oct 22, 2004 8:59 am
Location: Staten Island, NY

Custom Delimiters

Post by narasimha »

I get a few log files which have delimiters like

Code: Select all

|.|
Is there a simple way of parsing the records, or do we have to go to the regular expression route?
Narasimha Kade

Finding answers is simple, all you need to do is come up with the correct questions.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

If you mean column delimiter, then you could try using the pipe '|' character as the delimiter, and ignore the columns with just the decimals. Treat them as marker fields, just ignore them.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
narasimha
Charter Member
Charter Member
Posts: 1236
Joined: Fri Oct 22, 2004 8:59 am
Location: Staten Island, NY

Post by narasimha »

Ken,

I had tried your approach, but was curious to know if there is a way to actually use |.| as a column delimiter ??
Narasimha Kade

Finding answers is simple, all you need to do is come up with the correct questions.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Nothing inherent to the Sequential File stage. Off the top of my head, you may be able to leverage the Filter option with perhaps 'tr' or some other command to translate your 'multiple character delimiter' into a single character. :?

Or go old school and bring in the record as one long string and then use something like Field to bust it up. I'm not saying that the Field command supports multi-character delimiters, I've never had to deal with anything like that, but it would be interesting to investigate the possibility.

Probably best to stick with Ken's approach, it seems the most... straight-forward. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
narasimha
Charter Member
Charter Member
Posts: 1236
Joined: Fri Oct 22, 2004 8:59 am
Location: Staten Island, NY

Post by narasimha »

Your right Craig, Ken's approach does give me the desired results.
I was looking into other possibilities, like -

Code: Select all

sed -e 's/|.|/|/g'
works perfectly on the OS level, but it complains when I put it in the Filter Command.
Narasimha Kade

Finding answers is simple, all you need to do is come up with the correct questions.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

And the complaint would be?
-craig

"You can never have too many knives" -- Logan Nine Fingers
narasimha
Charter Member
Charter Member
Posts: 1236
Joined: Fri Oct 22, 2004 8:59 am
Location: Staten Island, NY

Post by narasimha »

I get the error

Code: Select all

LogFile..Copy_of_Sequential_File_0.DSLink3: ds_seqopen() - Error in filter command "/usr/bin/sed -e 's/" - 
sed: Function s/ cannot be parsed.

LogFile..Copy_of_Sequential_File_0.DSLink3: DSD.SEQOpen GCI $DS.SEQOPEN error 11.
Narasimha Kade

Finding answers is simple, all you need to do is come up with the correct questions.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Try it in the before job subroutine.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
rwierdsm
Premium Member
Premium Member
Posts: 209
Joined: Fri Jan 09, 2004 1:14 pm
Location: Toronto, Canada
Contact:

Post by rwierdsm »

narasimha,

I'd be very interested in your eventual solution. I will be receiving some files that have multiple 32k wide columns separated by a 3 character delimiters.

Please post your findings!
Rob Wierdsma
Toronto, Canada
bartonbishop.com
Post Reply