Is delimiter can be parameterized at job level

[NL]

Hi,

I have a requirement that the sequential file can come with data of any delimiter.I can see delimiter can be selected from list of options in properties of sequential stage but this is not promoting to have parameter to be included in the list. Can any one help me on whether the delimiter can be parameterized at job level, if Yes can you tell me how?

Thanks
LK

[priyadarshikunal]

Welcome aboard!!

Did you try delimiter string property? don't remember what it is actually called but should be similar.

[NL]

Once we choose the type of delimeter can not be changed.I want to send delimeter during runtime.Help me if you have any idea on this.

Thanks & regards,
NL

[pandeesh]

Are you NL or LK?
Your first and recent post differs in signature.
Are two people using the same account? :D

[chulett]

I don't believe that it can be parameterized but perhaps you could use schema files to control that?

[SURA]

You Can:

If you create a USER DEFINE STAGE.

DS User

[SURA]

Is it not as simple as i thought. But i am trying now.

DS User

[NL]

I don't believe that it can be parameterized but perhaps you could use schema files to control that?

Craig can you elaborate me more on controlling using schema files.
Thanks

[kandyshandy]

Schema file will not be an easy option... Do you know the possible delimiters (in this world) that would come in your file? Is it like 2 or 3 or less than 5?

[qt_ky]

That sounds like an unusual requirement. I have not tried it, but I think you should be able to read your delimited file in with no delimiter or quote character using a single VarChar column. Then in a Transformer stage put in logic to columnize each record based on your delimiter parameter.

[eph]

Hi,

I've already done this using a logic as following:
- In sequence job, detect your delimiter and other properties
- Optionally call either a command line, script or job to modify a template schema file properties (delim_string='#Your Char#')
- Then run you job using the correct schema file passed as a job parameter

Schema file properties overwrite all stage properties, so it should work like this.

Regards,
Eric

[qt_ky]

I think Eric has identified a good solution.

[NL]

Hi Eric,

It is a very good design approach.The current design which we have is the same.Execution time taking is more as we have so many sequential files with windows batch scripts.So I am looking for the option something in data stage.Can you please help me on this if you have any other?

Thanks,
NL

[qt_ky]

Two Erics have suggested two options, both in DataStage. Which ones have you tried? Columnization that I mentioned (Column Import stage does the same thing) will take the task of dividing rows into columns in parallel across your nodes. When you do it the default way in the Sequential File stage, it happens sequentially. I think if you can get this approach to work, it will be faster, unless your server is overloaded to begin with.