DSXchange

Hi Everybody, Wishing you all a very happy new year. My datasource is a dataset (DS1) created by previous job. I need to add another columns and pass on the new dataset (DS2) (DS1 + new column) This new column is numeric. It is like appending another column with value 1 to DS1 to all the rows. Could...

Hi All, I got to execute 30 jobs which are infact loading the data into fact and summary tables. Configuration: 12 CPU, each CPU duo core solaris Logical nodes: 12 I was just wondering that 1. If the larger number of job running parallel will effect the overall performance or 2, Should I restrict nu...

Thanks All,
You guys are wonderful.
Regards,
Munish

I am reading from sequential file.
The load is so high (200 gig) that it has to done on fly.

This information will be written to log file.

Any suggestions??

Thx,
Munish

Hi There, Scenario My records look like AccountNo. BillNo. Item_id 1 1 167 1 1 168 1 2 6 1 2 7 1 2 8 2 1 6 Now, I need to record log for record count By account, Bill, RecordCount so it will be Accountno Billno RecordCount 1 1 2 1 2 3 2 1 1 I searched thru the DSX, there were articles about KeyMgtGe...

Hi All, You have been wonderful. I would like to seek your kind advice upon: Scenario: Our ETL batch window process 200 gig sequential file in 8 hours. ETL batch is a part of big continuous batch window. There may be a possibility when our ETL might be running during day time i.e. peak reporting per...

You are right.
That is exactly the problem is.
I am just wonering how to refresh my lookup.
If I get that particular new value again, I do not want to insert it.

Any thoughts??

regards,
Munish

Thanks for your advices, However, I am still not clear about how to 1. If I use Change Control, Will the latest values which are inserted into Dim table will be updated in Change control 2. If not what might be the best option like Hash Look up, Sparse Look up etc. 3. We are writing to DB and readin...

Hi there, This is in continuation of my previous doubts regarding TELCO ETL. We are parsing 200 gig of sequential file where the data load in our biggest table is to the tune of more thatn 200 mills every day. Now, the scenario is that we are reading sequential file and we have to 1. First, Do looku...

Thanks for the reply.
Please advice:
1. How to shift this to Parallel Forum.
Cheers,
Munish

Thanks for the reply.

It definitely makes sense.
However,
What are your thoughts regarding scenario
like such a key might be strange/ very big say after 5-10 years.

The doubt might be silly but I thought of expressing it.

Hi There, We have to create key during ETL. This can be done by surrogate key generator. However, Fact table will be loaded with more than 100 millions of rows everyday. Please advice (TELCO project) what might be the better solution and why: 1. Will the simple key like incremtal key is fine enough ...

Thanks Buddies, Your replies are informative and in line with our thoughts. Does Datastage gives any sort of confirmation in log or in any other way of successful load or is there any property that confirms that Load is successful without going to Database for rechecking. Since, in our case data fro...

Hi There, We are loading data from sequential file into Oracle DB using OCI-Load, If we use Upsert, it provides us facility of Reject file. However, Load property does not provide that functionality. Since the data is massive, we can not use Upsert. Could any one please advice about the different sc...