pros and cons of using Flat file as source/target

[sengs]

Hi all,

I am process of preparing document with Pros n cons of using flat file as source / target.

Kindly post ur replies n if u hav any doc for the same. Plz share it here.

[ArndW]

The pros and cons of using a sequential file can only be discussed when you have alternatives. Are you comparing using flat files to a database? Hashed Files? Printouts?

[sengs]

Compare it with oracle database.

[ArndW]

In DataStage you seldom have the luxury of being able to choose your source data or target systems. The question is quite irrelevant unless you have specific circumstances that you haven't mentioned.

Flat Files
- compact
- can be given any name you wish and be put anywhere on your machine
- can be written to and read from without additional software

Oracle
- Has colorful documentation
- Is expensive
- can store data for random access

[sengs]

thanks a lot for your reply.
In our project we are going to use flat file as input. so we are analysing the adv and dis advantages of the same.

[sumesh.abraham]

Hi All,

In our project, the following scenario arises.

Data needs to be transformed as per the business rules. The source can be a flat file or an Oracle table. What are the prons and cons of using a
1) flat file for storing the data 2) Oracle table for storing the data.

I guess the parameters for the selection may include
1) Ease of extraction
2) Performance of the extraction process
3) Avoidance of extra validation (E.g: If an Oracle table is used for storing the data, there may be constraints which will make sure that a non-integer value is not present for a field of type integer. But if a flat file is used,we may need validation for the data).

I would appreciate if you can suggest the pros and cons of using
1) Flat file
2) Oracle table

Thanks,
Sumesh

[chulett]

To me, it's simple: use flat files for 'staging' when your processes can support sequential media / processing. There's way too much overhead to use a database for simple staging, as a general rule I only do that when I need to use the staged data in sql joins.

[sumesh.abraham]

Chulett,

Thanks for the reply. Can you be little more specific. I need to create a formal document with the pros and cons of each.

Thanks,
Sumesh

[sumesh.abraham]

Chulett,

Thanks for the reply. Can you be little more specific. I need to create a formal document with the pros and cons of each.

Thanks,
Sumesh

[chulett]

What kind of specifics? That's about as specific as I'm willing to get this early in the morning. Recognize that this is hardly a DataStage topic, this is something anyone doing pretty much any kind of data manipulation has to face and resolve.

Suggest you Google for the topic, I can't imagine it would be all that hard to find more formal or complete discussions of this topic out there. Heck, I seem to recall Kimball's books discussing this.