Wildcards in output sequential file...

[kaps]

Hi

Can we use wildcards in the file name option of output sequential file stage ?

I tried like this in file name

/path/myfile*.txt

It gives me error code as status = -1. I think this means it is not able to open the seq file.

does the wildcards allowed in the file name ?

Thanks for your inputs...

[chulett]

does the wildcards allowed in the file name ?

Nope, not allowed in Input nor Output filenames. Not really sure how you would think that would even work on output...

[ketfos]

Hi,
i wrote a job exactly as mentioned in the initial message.

It worked fine.

My output file name was defined as

uv/Projects/workfiles/x*.txt

The file was populated correctly. Input was Oracle table.

Ketfos

[kcbland]

Creating files with embedded reserved characters is usually a no-no. These would be chars such as:

Code: Select all

/\&' "!`;,>|

[ray.wurlod]

What would it mean???

The Sequential File stage uses a C function to write to file. If the file is to be created, what gets created?

If the file doesn't have to be created, does it write to every file whose name matches the wildcard pattern? This is weird.

[chulett]

My output file name was defined as

uv/Projects/workfiles/x*.txt

The file was populated correctly.

It's not about it being 'populated' correctly, it's about the notion of using 'wildcards' in an output filename. Are you expecting it to literally create a two character filename - an 'x' followed by a '*', an asterisk? Or is the expectation that it gets expanded somehow? As noted, an asterisk or splat (or any other metacharacter) is not necessarily allowed as a valid character in a filename.

The question back to kaps is what was the motivation behind the question? Why wondering about using wildcards in the output filename? Unless you just literally want that character in the name, perhaps if you stated what your end result should be, people here could help you figure out how to get there.

[kaps]

Thanks for all the inputs. I think I misstated what I wanted to say...
It's a input sequential file stage not output.

I was wondering If I can read more than one file using seq file stage in a single datastage job.I don't want to use folder stage or don't want to merge the files before reading them using cat commands etc...

Based on the replies here I think wildcards are not supported on seq file stage.

Sorry about the confusion and thanks for the inputs.

[chulett]

I was wondering If I can read more than one file using seq file stage in a single datastage job.I don't want to use folder stage or don't want to merge the files before reading them using cat commands etc...

Well, you've eliminated the two most common ways of doing that.

As, as you've surmised, wildcards are not support in the Sequential File stage.

[kaps]

wow...what a quick reply. Thanks for the lightning reply and that too during 10.30 PM central time and I believe it's 9.30 for you...
This is the greatness of this forum...Hats of to you guys...

[rleishman]

Technically, if you didn't care about elegant coding, and you hated the support guys, and they hated you, you could:

- Put /dev/null in the File Name
- Put cat /path/*.txt in the Filter Command

[chulett]

Ross, curious... have you actually tried that and had it work? I assumed it would way back when, but was unable to actually make it work in my 7.0.1 version. Had to fall back to the 'cat them all together before job' approach.

[rleishman]

It worked a lot better in my head. Is it my fault DataStage can't keep up?

It seems that DS somehow switches globbing off !?! Even if you try to expand a * in a shell script it doesn't expand.

So, if DS can't/won't glob, get someone else to do it for you; like my old friend Perl.

Sequential File Stage:
- Set filters on
- Filename = /dev/null
- Filter command = ....

Code: Select all

perl -pe 'BEGIN{@ARGV=</path/*.txt>}'

Now this solution is even harder to understand than my orininal one; but I tested this one. At least the other one only had a 'cat' - until nit-pickety Craig got hold of it.

If anyone uses it, please do 2 things:
1. Add a comment/description to the Seq File stage saying what it does.
2. Don't mention my name, or where you got the code.

[chulett]

Excellent! Have to keep this 'work around' in mind for next time.