DSXchange

Job failed with the error saying "Required column missing"

It might be the describe failed error.
Please check the presence of the missing column in source as well as target with same metadata

Got it, it was the metadata error, with more quotes in other columns,
issue resolved, running the job again with Sort and Agg....post the results soon.

Okay the input link to the SORT stage has processed all the rows but the output link doesnt show any rows processed from a long time, does this mean it is not configured correctly????

Here is the settings i am using.

Rows coming in 1.2 million rows
Max Rows in VM = 49000
Max Open Files = 20
Temp File = explicit Full Path mentioned(i can also see files getting created therr)
tracing level = 0
Stable sort = no
esc char= \
column seperator = ,
Sort spec = Col1 ASC,COl2 ASC,Col3 ASC ...so on

Still Running and no rows on the Sort Output link Yet, running from 3 hours.

running from 4 hours.

You will never get any rows output from a Sort stage until all the input rows have been consumed. This is common sense - it does not know where the final row will be in the sorted order until it has been processed.

Nagaraj wrote:Okay the input link to the SORT stage has processed all the rows but the output link doesnt show any rows processed from a long time, does this mean it is not configured correctly????

Don't confuse my advice about how a properly configured Aggregator will run with what you see in the Sort stage, it must take in everything first. And the Server sort stage is pretty slow, in my experience I found an O/S level sort to be approximately 10x faster. YMMV.

Okay will try to sort at the OS level and check the Processing in Job.