Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.
Moderators: chulett , rschirm , roy
harryhome
Participant
Posts: 112 Joined: Wed Oct 18, 2006 7:10 am
Post
by harryhome » Mon Oct 11, 2010 11:42 am
Hi, In job doing lookup with .ds files using looking stage. Lookup has 5 million row , number of lookups are 2.
Getting error as below;
lkp_ISR_HST,0: Error writing table file "/tech/dsnode1/Datasets/lookuptable.20101011.mv1mb1b": No space left on device
anbu
Premium Member
Posts: 596 Joined: Sat Feb 18, 2006 2:25 am
Location: india
Post
by anbu » Mon Oct 11, 2010 11:58 am
You dont have enough space in /tech/dsnode1/Datasets to write lookup file.Remove the old files from /tech/dsnode1/Datasets.
Use /bin/df to check how much free space is there in /tech/dsnode1/Datasets
You are the creator of your destiny - Swami Vivekananda
harryhome
Participant
Posts: 112 Joined: Wed Oct 18, 2006 7:10 am
Post
by harryhome » Mon Oct 11, 2010 11:58 am
Is there any Limit on Loopupstage, in terms of number of rows.
harryhome
Participant
Posts: 112 Joined: Wed Oct 18, 2006 7:10 am
Post
by harryhome » Mon Oct 11, 2010 12:01 pm
anbu wrote: You dont have enough space in /tech/dsnode1/Datasets to write lookup file.Remove the old files from /tech/dsnode1/Datasets.
Use /bin/df to check how much free space is there in /tech/dsnode1/Datasets
Thanks, can please elaborate more on use of /bin/df
anbu
Premium Member
Posts: 596 Joined: Sat Feb 18, 2006 2:25 am
Location: india
Post
by anbu » Mon Oct 11, 2010 12:21 pm
As far as I know, there is no limit on rows in Lookupstage
df command displays filesystem space. Check man df to know more about this command
You are the creator of your destiny - Swami Vivekananda
harryhome
Participant
Posts: 112 Joined: Wed Oct 18, 2006 7:10 am
Post
by harryhome » Tue Oct 12, 2010 7:30 am
df -a Datasets
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/datastage_dg-tech_dsnode1
2064208 572584 1386768 30% /tech/dsnode1
This tells still 70% of space is available. If there is no limit on lookup stage then not sure why giving "No space left on device" message
ArndW
Participant
Posts: 16318 Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:
Post
by ArndW » Tue Oct 12, 2010 7:59 am
There is indeed a limit on the reference data size of a lookup stage in PX. If you are on AIX then
this thread might help solve the problem, but only within certain limits.
vivekgadwal
Premium Member
Posts: 457 Joined: Tue Sep 25, 2007 4:05 pm
Post
by vivekgadwal » Tue Oct 12, 2010 8:02 am
Is this job, that is causing you grief, running as part of a sequence? The disk may be getting full at the time the job is getting triggered. Check with your Unix admins to see the disk usage at a point in time or you monitor that and see what is happening.
Lookup is dependent on the memory. As long as you have enough memory, you should be fine. Your error doesn't denote this, as far as I can understand it.
Vivek Gadwal
Experience is what you get when you didn't get what you wanted