Stange behaviour...file size in Unix(AIX) is 0

kaps · Post by **kaps** » Tue Oct 02, 2007 12:57 pm

Hi

I am seeing a stange behaviour while running a server job. I can see the statistics on Designer showing that over million records are written to the file but in unix the file size is 0. Job is still running.

Job design is:

MQ Stage---> Transformer--->SeqFile

I was thinking the file size will grow but it's 0. Can someone tell me what could be the reason ?

Thanks

DSguru2B · Post by **DSguru2B** » Tue Oct 02, 2007 1:24 pm

Are you looking at the right file in the right directory on the right server?
Are you getting any warnings?

DeepakCorning · Post by **DeepakCorning** » Tue Oct 02, 2007 1:30 pm

Are you using any FTP softwares to look into the file??

ray.wurlod · Post by **ray.wurlod** » Tue Oct 02, 2007 2:39 pm

Please post two things:

the value of the File property in the Sequential File stage (that is, the pathname copied/pasted from the job design)

an ls -l listing of that pathname from UNIX

chulett · Post by **chulett** » Tue Oct 02, 2007 2:40 pm

Some of that kind of information isn't flushed to disk until the file is closed. As far as I know, it can depend on your O/S and be affected by kernel or disk settings as well.

kaps · Post by **kaps** » Tue Oct 02, 2007 2:56 pm

I am looking at right everything...I am not using any ftp...

Ray -
File name from job
#$prm_project_path#/fin_transaction/EfinFin_Q_Dump_ext.msg

ls -l output
-rw-rwSr-- 1 a136940 dstage 0 Sep 28 16:02 EfinFin_Q_Dump_ext.msg

Craig - This is the first time I see this problem. Do you think something got changed in server ?

Thanks

ArndW · Post by **ArndW** » Tue Oct 02, 2007 3:15 pm

Delete your empty file. Re-run the job with the correct parameter setting (confirm the setting in the director log entry for the job). Is the empty file created again? Do you have an after-job subroutine that might touch the file?

kaps · Post by **kaps** » Tue Oct 02, 2007 3:34 pm

I can't rerun the job as this job is doing destructive read from MQ. If I stop the job all the messages will be gone and I don't have a way to recover them.

Thanks

ArndW · Post by **ArndW** » Tue Oct 02, 2007 3:51 pm

if the job is still running then your data might be buffered and not flushed to disk. You won't know the actual file contents until the job completes.
Try doing a 'fuser -fux {filename}' to see if your DataStage job actually has this file open for write; you might be looking at the wrong file.

kaps · Post by **kaps** » Tue Oct 02, 2007 4:23 pm

I see the file size growing in other jobs even when they are running. what could be the reason for that ?

When I did fuser -fux filename I did not get any reply back. It simply prrints the filename with the : at the end...what does that means ?

Problem here is that this job will be running all the time. I need to run another job to stop this job. I am afraid If I run the stop job I will loose the records.

What Am I missing here ?

Thanks for your help

ArndW · Post by **ArndW** » Tue Oct 02, 2007 5:43 pm

kaps - when fuser doesn't return anything it means that no process has the file open. Check your job parameters again to see what the output file should be. If you are certain that you are looking at the correct file then you should check to see if the job is actually still running by using "ps -ef | grep {yourjobname}".

DeepakCorning · Post by **DeepakCorning** » Tue Oct 02, 2007 6:04 pm

Or try to view data in the job and check.. it will reveal tht it is locked by the process or not...

kaps · Post by **kaps** » Wed Oct 03, 2007 9:27 am

Arnd - When I check the job using ps -ef command it gives me the following result...

dsadm 368878 3334210 0 12:34:50 - 0:00 phantom DSD.StageRun edw_ft_FinancialTransactionQueue_ext_DR. edw_ft_FinancialTransactionQueue_ext_DR.trn_chk_success 46 0/50/1
dsadm 2322518 1 1 Sep 28 - 0:28 phantom DSD.RUN edw_ft_FinancialTransactionQueue_ext_DR 0/50/1/0/0
dsadm 2822356 2711788 2 10:19:35 pts/4 0:00 grep edw_ft_FinancialTransactionQueue_ext_DR
dsadm 3002612 2322518 0 Sep 28 - 81:10 phantom DSD.StageRun edw_ft_FinancialTransactionQueue_ext_DR. edw_ft_FinancialTransactionQueue_ext_DR.trn_chk_success 4 0/50/1
dsadm 3334210 3268822 1 12:34:50 - 0:05 phantom DSD.RUN edw_ft_FinancialTransactionQueue_ext_DR. 0/50/1/0/0

So I believe the job is running...

Deepak - I tried to do the view data. It says data source is empty...

Please let me know what's going on here...

Thanks

DeepakCorning · Post by **DeepakCorning** » Wed Oct 03, 2007 12:24 pm

So the job is still running and when u say show file data it does not give u an error saying the file in Use?
If that is the case then i think the job is not doing anything (did not even trigger) , so u may want to stop it and replace the seq file with an hashed file and check it out.

ArndW · Post by **ArndW** » Wed Oct 03, 2007 3:06 pm

Kaps, login to a terminal session as dsadm (that isn't a good userid to run jobs as) and do a "truss -p 3002612" and see what that process is doing. It might be doing nothing, i.e. calls to SLEEP.