Active and On-line History Tables

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
suneeth
Participant
Posts: 11
Joined: Fri Jun 18, 2004 1:06 am

Active and On-line History Tables

Post by suneeth »

Hi All,
My Requirement is I need to split the data into Active which holds the current day's data and On-line history data which should contain the history. I need to perform this on daily, weekly and Monthly basis. I also have reports accessing this data any point of time. Can anyone help me in the best way to do this.
I am using DB2 database.

Suneeth--
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

suneeth,

your requirements are to write a simple DataStage job that does this. You would write a job that takes an input stream through a transform stage which would use constraints to direct the flow to two output streams, one for your current data and one to the on-line history. Both of these would reside in your DB2 tables (accessed through the DB/2 stage or a ODBC conneciton). On the face of it a simple process, though no doubt there are additional constraints, perhaps you can explain.
suneeth
Participant
Posts: 11
Joined: Fri Jun 18, 2004 1:06 am

Re: Active and On-line History Tables

Post by suneeth »

suneeth wrote:Hi All,
My Requirement is I need to split the data into Active which holds the current day's data and On-line history data which should contain the history. I need to perform this on daily, weekly and Monthly basis. I also have reports accessing this data any point of time. Can anyone help me in the best way to do this.
I am using DB2 database.

Suneeth--
Arnd,
Thanks for the reply. Writing the data into two tables is good suggestion. My concern is we are duplicating the data. I am thinking abt using the bulk load which will load the data into History table and delete the data from the active table. Does this work or you have better solution.

Suneeth--
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

suneeth - your approach sounds like what most of us would do; but it is difficult to know the details specific to your actual situation.

Am I correct in assuming that you are only trying to archive older data in the same form as your original tables? If so, why not look at doing it at a database level - partitioning your tables based on date; then any archiving can be done by the DBA without actually having to move any data...
suneeth
Participant
Posts: 11
Joined: Fri Jun 18, 2004 1:06 am

Post by suneeth »

Arnd,
Sounds like its a good idea. Thanks for that. Let me check with the DBA.

cheers,
suneeth--
Post Reply