Hi,
Is there a way to compress .ds dataset files to different location?
Thanks,
Compress dataset files
Moderators: chulett, rschirm, roy
No directly; but you could do a
Code: Select all
orchadmin dump {dataset} | gzip -c > zipfile.gz
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
No. The .ds files themselves are tiny in any case, and are already in a binary format. The actual data of a Data Set resides elsewhere, on the resource disks specified in your configuration file. Again, these are already in a binary format. Attempting to compress them will not produce any gain worth having. And they could not be used in/from the other location.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Total dataset size is 800 GB per day
Hi All
We have huge volume of Data over total dataset size is 800 GB per day.
so i am plaining to compress dataset with following command
orchadmin dump {dataset} | gzip -c > zipfile.gz
How to uncompress it back
not sure abt compress & uncompress stage whether i can use it
Thanks
Sanjay
We have huge volume of Data over total dataset size is 800 GB per day.
so i am plaining to compress dataset with following command
orchadmin dump {dataset} | gzip -c > zipfile.gz
How to uncompress it back
not sure abt compress & uncompress stage whether i can use it
Thanks
Sanjay
Is there are reason why you are not compressing the dataset within the DataStage job itself? Use the compress stage in the job that creates the dataset and the expand stage in the job(s) that read the dataset. These can use with the Unix 'compress/uncompress' programs or 'gzip/gunzip'.
Compressing and uncompressing the data adds a little overhead, but can significantly reduce the I/O. In the end, the jobs will probably run as fast as they did before, maybe even faster due to the reduced I/O. Better yet, the data footprint will be significantly reduced.
Hope this helps.
Brad
Compressing and uncompressing the data adds a little overhead, but can significantly reduce the I/O. In the end, the jobs will probably run as fast as they did before, maybe even faster due to the reduced I/O. Better yet, the data footprint will be significantly reduced.
Hope this helps.
Brad
It is not that I am addicted to coffee, it's just that I need it to survive.
With DataSets the actual disk space used for an unbounded Varchar column is smaller than that used in a bounded one.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>