DSXchange

I am using Column Propagation because I have 19 output Sequential files, all defined exactly the same way. I would like to maintain a single schema file and use it with column propagation. Works great except DS cannot generate a reliable schema from a table definition. This is extremely aggravating.
Here is an example of two fields from the schema file that appear to me to be identically defined;

AD_Tool:nullable string[max=5] {null_field='',prefix=2};
Alloc_Unit_ID:nullable string[max=5] {null_field='',,prefix=2};

In fact, I have exported the table definition and manually updated the definition of the second field to actually BE identical to the first (except for the name).
Upon import, the result is as above. The second field - Alloc_Unit_ID has an extra comma that prevents the schema file from being usable.

I'm forced to build a sed script to remove the extra commas.

Anybody know why this is the case?
How I can generate a schema file from a Table Definition that works?

You can make schema files from tables in any database.
When you goto the option to Import the table definitions -> Orchestrate Schema file, there you have to option to connect any where.
Instead if you have the ER diagram get the ddl, generate those tables in Sandbox and import the metadata. There you are!
IHTH.

Thank you Mr. Guru. You might take a peek at my questions as they don't appear to correlate to your reply. I apologize if I didn't state my problem succintly.
I have a schema file, generated from a table definition, that is incorrect. DataStage is failing when attempting to use it with column propagation when writing to many sequential files. It is failing because there are extra commas in the schema file. I'd like to automate this process and eliminate the manual process. This means I would like to eliminate the shell script I have written to correct the schema file manually in order to make it usable by DataStage.

Has anyone experienced this problem?

Why are there extra commas and how can I fix it?

Hi trobinson,
I never faced this kind of problem and I can't reproduce ( simulate this error). It would be great if you post create script of this table, that might help us to look into details of this problem.

Thanks
Girija

trobinson wrote: I have a schema file, generated from a table definition, that is incorrect. DataStage is failing when attempting to use it with column propagation when writing to many sequential files.

I dont get this. If the table definition is incorrect, then its a problem with the way the table was created, not with the schema importer.

I think the table you are getting your metadata from, is not created appropriately. Try using those table as target and switch off RCP.

Ah, the light bulb goes off. When I say Table Definition I was referring to a DataStage object called a Table Definition. These things do not need to be spawned from a dbms object, they can be native to DataStage. This particular one comes from the column tab from a Copy stage. It results from saving the columns of a link. It has nothing to do with any dbms ddl.

This is straight Datastage folks no smoke and no mirrors. I save the columns to a Table Definition and attempt to generate a schema file from that saved Table Definition. The purpose in doing so is to maintain the Table Definition in a single place and use it in many places via a schema file.

And now...the rest of the story;
I have a job that needs to write 19 different sequential files that ALL have the same format but contain different data. Think of buckets. I have 19 buckets. Using Switch and filter stages, I route data to the proper buckets. Let's say I then want to add a field to all the files. Do I add this 19 times? Heck no! I define the Sequential stages with the exact same schema file which I maintain as a Table Definition. Theoretically, this would enable me to add a field to this table definition, generate a schema file, save it to the OS and boom! the new field shows up in all 19 files because I am also using column propagation. Holy Master Metadata Batman! Gotta love it. If it worked, that is, which it don't. I have to constantly massage the dang schema file. Hence this post. What time is it in Australia?

Here's the run-time error;

main_program: Fatal Error: Caught ORCHESTRATE exception in main program: APT_ParseError: In field "Alloc_Unit_ID": In property list: Expected property name, value, or "}", got: ",", line 6

I first need to understand how is this occuring in order to see where the problem is.
As said by you, the schema file results from the columns tab in the copy stage, did you try to view what is being generated.

There can be two possiblities where the problem is occuring.
1. The schema file is not generated properly.
2. The importer is not operating as it should be.

I first need to understand how is this occuring in order to see where the problem is.
As said by you, the schema file results from the columns tab in the copy stage, did you try to view what is being generated.

There can be two possiblities where the problem is occuring.
1. The schema file is not generated properly.
2. The importer is not operating as it should be.

And yea I am not in Australia.
My existance is in USA.

1. Open any Table Definition.
2. Go to the Layout tab.
3. Select the Parallel radio button.
4. Still Clueless?

Hi,
I'm following everything up to where you 'generate a schema file' from the table definition. I have 7.5, how do you do this?

.... Theoretically, this would enable me to add a field to this table definition, generate a schema file, save it to the OS and boom! the new field shows up in all 19 files because I am also using column propagation.

Carter[/quote]

You mean what is step 5? That's easy. Copy and paste the results of step 3 and paste it into a UNIX file. Yes, it would be oh so sweet to do a save as and FTP it to UNIX but baby steps, baby steps. I'm concerned with generating it correctly. Actually saving it as a schema file is a bell and whistle until I get the first problem resolved.

Oh !
So you are just copy pasting the schema to a unix file.

And that extra comma is showing up on the parallel tab ?
If yes, did you try changing the nullabilty of the field?