DSXchange

I am working with flat fixed length ASCII files in UNIX. I have the non-DataStage processes that need to know the current file layout (field name, start position, size).
I would like to be able to retrieve this info dynamically from the most current version of a Data Stage job that creates this file.
Only this way I can insure that the changes to the file layout (fields sequence, field sizes and such) will not affect the next process.
Can anybody give me an idea?

Are you asking about how to make a job publish a file format? Or how to extract the file definitions from the job design so that you can distribute a data file with a format file?

You should be saving the file definitions for your output files in DS Manager. You do have options at this point on how to get this information out.

The approved tool for doing this is MetaStage. From MetaStage, you can extract file definitions in DS Manager, and then publish this information in whatever format you want.

Another option is the Reporting Assistant (search this forum). Another option is to reverse engineer the internal job design metadata table (search this forum). There is no inherent functionality for generating an output map of a data file.

Yes, I would like to have a data file with a file format.
But I would like to get file format dynamically at runtime.
Knowing a job name, a file name and a link name is it possible to access the job design at runtime and dump the file map into sum file where I can grep on field name and get a displacement info?
In fact we are doing it now but we pars the DSX files which are secondary. We made some assumptions on the structures of the DSX files. And it worked for a while. It may break with any new changes to the DSX file structures.
I would like to try to get it right from the source (DS job in database) at runtime.
And we do not have MetaStage.

kcbland wrote:Are you asking about how to make a job publish a file format? Or how to extract the file definitions from the job design so that you can distribute a data file with a format file?

You should be saving the file definitions for your output files in DS Manager. You do have options at this point on how to get this information out.

The approved tool for doing this is MetaStage. From MetaStage, you can extract file definitions in DS Manager, and then publish this information in whatever format you want.

Another option is the Reporting Assistant (search this forum). Another option is to reverse engineer the internal job design metadata table (search this forum). There is no inherent functionality for generating an output map of a data file.

A rough check on line length might suffice.
Line length is the quotient after dividing characters in file (wc -c file) by lines in file (wc -l file) which you can do using the UNIX command expr.
Check out the man pages for the wc, cut and expr commands.

I am sorry. It is not an answer to my question. Did you try to answer somebody else?

ray.wurlod wrote:A rough check on line length might suffice.
Line length is the quotient after dividing characters in file (wc -c file) by lines in file (wc -l file) which you can do using the UNIX command expr.
Check out the man pages for the wc, cut and expr commands.

leomauer wrote:Yes, I would like to have a data file with a file format.
But I would like to get file format dynamically at runtime.
Knowing a job name, a file name and a link name is it possible to access the job design at runtime and dump the file map into sum file where I can grep on field name and get a displacement info?

Yes, you could do this. You do realize that this is unsupported, and you have to address the fact that Ascential can/does changed the underlying storage model from one release to another?

You can search this forum for discussions on the internal job design storage file DS_JOBOBJECTS. Your efforts are tremendous for someone not experienced in the internals of this product. This is a trivial task for one of the Inner Circle members to do for you, so if you want to give an expert a try I sure one of us can knock this out for you (shameless plug). Otherwise, expect only guidance and pointers from us, as we don't debug code. :D

Good luck!

leomauer wrote:I am sorry. It is not an answer to my question. Did you try to answer somebody else?

ray.wurlod wrote:A rough check on line length might suffice.
Line length is the quotient after dividing characters in file (wc -c file) by lines in file (wc -l file) which you can do using the UNIX command expr.
Check out the man pages for the wc, cut and expr commands.

I was suggesting a simple way to check that a changed structure "will not affect the next process" (to quote from your original post) "from outside DataStage" (to quote your subject line). While this method does not report the changed structure, a changed structure is likely to yield a different line length.