ETL Batch Process & Timings

[Munish]

Hi All,

You have been wonderful.

I would like to seek your kind advice upon:

Scenario:
Our ETL batch window process 200 gig sequential file in 8 hours. ETL batch is a part of big continuous batch window. There may be a possibility when our ETL might be running during day time i.e. peak reporting period.

We are using Oracle and our OLAP tables are partitioned on the basis of ETL_Key.

What might be the performance implications as for reporting:

1. During ETL_Batch, we will be writing in a new partition and our reports will be reading from previous partition.
Will it effect reporting performance.

2. During ETL_Batch, we will be writing in a new partition and our reports will be reading from same partition.
Will the reading and writing in same partition at same time will effect reporting performance.

I look forward to your replies.

Thanks and regards,
Munish

[kumar_s]

2nd method may not pull out the precise result as the data and the indices been updated/inserted by ETL process. Due to the same reason, the performace might even get affected as the data gets loaded as the local index gets updated (based on the partitioning key).