Is delimiter can be parameterized at job level
Moderators: chulett, rschirm, roy
Is delimiter can be parameterized at job level
Hi,
I have a requirement that the sequential file can come with data of any delimiter.I can see delimiter can be selected from list of options in properties of sequential stage but this is not promoting to have parameter to be included in the list. Can any one help me on whether the delimiter can be parameterized at job level, if Yes can you tell me how?
Thanks
LK
I have a requirement that the sequential file can come with data of any delimiter.I can see delimiter can be selected from list of options in properties of sequential stage but this is not promoting to have parameter to be included in the list. Can any one help me on whether the delimiter can be parameterized at job level, if Yes can you tell me how?
Thanks
LK
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
Re: Is delimiter can be parameterized at job level
You Can:
If you create a USER DEFINE STAGE.
DS User
If you create a USER DEFINE STAGE.
DS User
Re: Is delimiter can be parameterized at job level
Is it not as simple as i thought. But i am trying now.
DS User
DS User
-
- Participant
- Posts: 597
- Joined: Fri Apr 29, 2005 6:19 am
- Location: Singapore
That sounds like an unusual requirement. I have not tried it, but I think you should be able to read your delimited file in with no delimiter or quote character using a single VarChar column. Then in a Transformer stage put in logic to columnize each record based on your delimiter parameter.
Choose a job you love, and you will never have to work a day in your life. - Confucius
Hi,
I've already done this using a logic as following:
- In sequence job, detect your delimiter and other properties
- Optionally call either a command line, script or job to modify a template schema file properties (delim_string='#Your Char#')
- Then run you job using the correct schema file passed as a job parameter
Schema file properties overwrite all stage properties, so it should work like this.
Regards,
Eric
I've already done this using a logic as following:
- In sequence job, detect your delimiter and other properties
- Optionally call either a command line, script or job to modify a template schema file properties (delim_string='#Your Char#')
- Then run you job using the correct schema file passed as a job parameter
Schema file properties overwrite all stage properties, so it should work like this.
Regards,
Eric
Hi Eric,
It is a very good design approach.The current design which we have is the same.Execution time taking is more as we have so many sequential files with windows batch scripts.So I am looking for the option something in data stage.Can you please help me on this if you have any other?
Thanks,
NL
It is a very good design approach.The current design which we have is the same.Execution time taking is more as we have so many sequential files with windows batch scripts.So I am looking for the option something in data stage.Can you please help me on this if you have any other?
Thanks,
NL
Two Erics have suggested two options, both in DataStage. Which ones have you tried? Columnization that I mentioned (Column Import stage does the same thing) will take the task of dividing rows into columns in parallel across your nodes. When you do it the default way in the Sequential File stage, it happens sequentially. I think if you can get this approach to work, it will be faster, unless your server is overloaded to begin with.
Choose a job you love, and you will never have to work a day in your life. - Confucius