more than one final delimiter?
Moderators: chulett, rschirm, roy
more than one final delimiter?
hi all
i have a really simple question, but just can't fix it!
how to specify multiple final delimiters in the sequential file!? for example I want both pipeline (|) OR 'end' characters are the finial delimiters for a record. what's the syntax!?
sorry, I am really frustrated right now, i read docs and google it, just can't find the answer!
thanks for the help
Abyss
i have a really simple question, but just can't fix it!
how to specify multiple final delimiters in the sequential file!? for example I want both pipeline (|) OR 'end' characters are the finial delimiters for a record. what's the syntax!?
sorry, I am really frustrated right now, i read docs and google it, just can't find the answer!
thanks for the help
Abyss
-
- Premium Member
- Posts: 376
- Joined: Sat Jan 07, 2012 12:25 pm
- Location: Piscataway
Are you saying that your file could look like this ?
or like this?
Code: Select all
col1,col2,col3
a,b,c|
l,m,n
x,y,z~
Code: Select all
col1,col2,col3
a,b,c|~
l,m,n|~
x,y,z|~
Jerome
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
Data Integration Consultant at AWS
Connect With Me On LinkedIn
Life is really simple, but we insist on making it complicated.
like this:
if i can set finial delimiter to end OR | then i can get all records
Code: Select all
col1,col2,col3
a,b,c| l,m,n|x,y,z|~
1,2,3|e,r,t
c,x,z
-
- Participant
- Posts: 9
- Joined: Tue Sep 20, 2005 2:10 am
I don't know a way to *really* use several delimiters in one sequential file stage.
But perhaps you can cope with this by using a Filter in the sequential file stage. Here you can define a filter program that for example would change all pipebars in the file into end-characters, so instead of two delimiters you only have one.
But with large files I suspect that this is not a very fast/efficient solution.
Kind regards,
Johannes.
But perhaps you can cope with this by using a Filter in the sequential file stage. Here you can define a filter program that for example would change all pipebars in the file into end-characters, so instead of two delimiters you only have one.
But with large files I suspect that this is not a very fast/efficient solution.
Kind regards,
Johannes.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
It's not clear (from your examples, anyway) which data belongs in which column.
You may be better off, once you've identified all the variants, to read each line as a single VarChar and do the parsing in a Transformer stage.
If nothing else, your parsing will be executing in parallel.
You may be better off, once you've identified all the variants, to read each line as a single VarChar and do the parsing in a Transformer stage.
If nothing else, your parsing will be executing in parallel.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
hi, thanks for the reply guys. sorry i didn't make the requirement clear. I was in the hurry.
problem is quite simple, as you know in sequential file we need to define "finial delimiter". it's normally end of line , tab or something else.
what's i want is to set BOTH end of line character AND pipeline (|) as finial delimiter.
for example:
regards
abyss
problem is quite simple, as you know in sequential file we need to define "finial delimiter". it's normally end of line , tab or something else.
what's i want is to set BOTH end of line character AND pipeline (|) as finial delimiter.
for example:
Code: Select all
the input file may like this:
record1|record2|record3
record4|
record5
(each record use comma as delimiter)
the output should be:
record1
record2
record3
record4
record5
abyss
Last edited by abyss on Thu Sep 18, 2014 8:17 pm, edited 1 time in total.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Some other options:
(1) get "them" to supply cleaner data
(2) specify pipe as Record Delimiter property value and pre-filter the newline characters from the file (perhaps using tr -d command as the Filter command in the Sequential File stage)
(1) get "them" to supply cleaner data
(2) specify pipe as Record Delimiter property value and pre-filter the newline characters from the file (perhaps using tr -d command as the Filter command in the Sequential File stage)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.