Help with Delimited, required_column_missing problem

[gateleys]

My job is as follows-

ODBC--->SeqFile----->AGG.....etc

The ODBC has a user defined SQL. SeqFile is used to append current records to previous ones (so, it has been set to append rows). The job runs fine.

Now, I need to add one field 'xcount' to ODBC (which undergoes a SUM in the user-defined SQL). The underlying database contains the new field definition (but I did not import it, rather just typed it). I have made changes in the SQL and added the column with the proper datatype, and also made changes to the subsequent stages. However, now when I run the job, it gives me the following error-

MyJob..penddwMyjob.frompend: read_delimited() - row 1, column xCount, required column missing

Where did I go wrong? Please help.[/code]

[chulett]

That's because you appended rows with X+1 columns in them onto the end of rows with only X columns.

If you want to keep the old rows in the file and still be able to read them in, in the reader stage scroll the Columns tab over to the right and change the value of Incomplete Columns. It defaults to Error - change it to 'Retain' or 'Replace'.

[ray.wurlod]

The metadata must be the same (including the additional column) on both the input to and the output from the Sequential File stage.

Your SQL must deliver exactly the same number of columns as are specified on the output link from the ODBC stage.

There's some good material in on-line help; next time you're in the Columns grid, particularly of a Sequential File stage, click Help.

[gateleys]

That's because you appended rows with X+1 columns in them onto the end of rows with only X columns.

If you want to keep the old rows in the file and still be able to read them in, in the reader stage scroll the Columns tab over to the right and change the value of Incomplete Columns. It defaults to Error - change it to 'Retain' or 'Replace'.

Hi Chulett,
It did not work when I set the Incomplete column to 'Retain'...but VOILA!! it works with 'Replace'. But, I want to know the consequences it will have to the previous rows (with X cols) that were already there. Please explain. And yes...thanx a lot... i just saved loads of time on my job.

[chulett]

The field will be populated based on the setting of the Missing Columns Action on the Format tab. By default, it gets mapped to a NULL, I do believe. Check the online help available from within the stage.