Loading duplicate rows using teradata connector

nikhil_bhasin · Post by **nikhil_bhasin** » Tue Jul 23, 2013 3:22 am

Hi,

I am migrating a job from 8.1 version to 8.7. This job loads data into a teradata table which is having UPI on key columns and source data is expecetd to have duplicate rows, so in 8.1, multiload stage in sequential mode is used to counter this situation, However in 8.7 using connector I am not able to execute this job, as it gets aborted due to constraint violation. I have set the connector to sequential mode, even then it gets aborted.

ray.wurlod · Post by **ray.wurlod** » Tue Jul 23, 2013 4:44 am

Running in Sequential mode does not eliminate duplicates - it merely processes the duplicates in one node. Eliminating duplicates must be coded for, for example using a Remove Duplicates stage.

chulett · Post by **chulett** » Tue Jul 23, 2013 6:09 am

Sorry, don't have a good way to merge duplicate posts so here we go:

prasson_ibm wrote:If your table is defined with UPI then how come table can have duplicate records.

rameshrr3 · Post by **rameshrr3** » Tue Jul 23, 2013 5:29 pm

Since you are using a Teradata Connector in 8.7 , Can you try setting the partitioning method to hash and sort by the keys with 'unique' option selected. I see that the Teradata connector doesn't allow you to specify a sequential execution method like the older stages.

nikhil_bhasin · Post by **nikhil_bhasin** » Tue Jul 23, 2013 10:06 pm

@ Ray - I agree to your point of removing duplicates beforehand, but since it is running fine in 8.1/multiload, we are told not to do any changes execpt for that of connector

.

@ prasson - Table doesn't have duplicates, they are trying to retain the last update as per requirement.

@ ramesh - we can change the execution mode to sequential in connector, you just have to click on the connector icon, once you open properties, instead of link (which opens by default)