Page 1 of 1
Error while File Set creation
Posted: Tue Mar 04, 2008 12:17 am
by tkbharani
Dear All
While creating simple FileSet we got the following warning
"When checking operator: No nodes with a disk in the "export" resource pool; using the default disk pool instead."
After seraching and reading document We changed the
Config file/Default.apt as follows.
resource disk "/u07/PX/ds7.5/Ascential/DataStage/Datasets" {pools "" "export"}
After adding export in resource disk, the job was not givig any warnings and completed successfully.
Can any one tel me why we use export in default.apt/resource disk. Do some stages need some special parameter like this in default.apt
Thanks
TKB
Posted: Tue Mar 04, 2008 3:00 am
by ArndW
This message normally will not appear unless you modify your stage and change the Stage -> Advanced tab and specifying "export". Did you do that?
No I did not change any setting in FileSet stage.
Posted: Tue Mar 04, 2008 4:09 am
by tkbharani
No I did not change any setting in FileSet stage.
More over the FileSet Stage--> Advanced Tab , we have only one active dropdown taht is buffering mode.
Is that ok to use export in config/default.apt
Posted: Tue Mar 04, 2008 4:24 am
by ArndW
I'm confused about how buffering mode plays a role here.
Ahh, I just reread the post and realized you are using file sets. The Parallel Job Advanced Developer Guide quotes
File sets
The export operator can generate and name exported files, write them to their destination, and list the files it has generated in a file whose extension is .fs. The data files and the file that lists them are called a file set. They are established by means of the fileset option. This option is especially useful because some operating systems impose a 2 GB limit on the size of a file and you must distribute the exported files among nodes to prevent overruns.
When you choose -fileset, the export operator runs on nodes that contain a disk in the export disk pool. If there is no export disk pool, the operator runs in the default disk pool, generating a warning when it does. However, if you have specified a disk pool other than export (by means of the -diskpool option), the operator does not fall back and the export operation fails.
This means you can run without an "export" pool and deprecate the warning with no worries or you can assign a pool of nodes to include "export"
Posted: Tue Mar 04, 2008 4:41 am
by tkbharani
Thanks for that ArndW
If am not using "export" in disk pools, my job will always complete with WARNING, that it has used default disk pool. So in order to avoid that I am using export.
Or is there any way to suppress the warning other than changing default.apt.
Posted: Tue Mar 04, 2008 5:24 am
by ArndW
PX allows you to control the "type" of many messages. I suggested that you "deprecate" the warning message and turn it into a normal, informational, one in my original post.
This is done by using "message handlers" in PX.
Posted: Tue Mar 04, 2008 6:12 am
by ray.wurlod
There are several stages that expect a particular node pool or disk pool to exist, and which log a message indicating that they will use the default pool when the expected pool is not found. For example, as you've found, a File Set stage expects a pool called import or export (depending on what you're doing with it), the DB2 Enterprise stage expects a db2 pool, and so on. Others expect but are silently disappointed - for example a tsort operator expects to find a sort pool.
Message handler
Posted: Wed Mar 05, 2008 12:22 am
by tkbharani
Dear All
For the above warning message I am trying to write Message Handler to "Demote to Infomational". But still I am getting warning.
I am creating a new rule for the handler through DS Director--> selecting Log message and the --> Add rule to message handler.
My message handler :
-----------------------------------
Message ID: TFOP000000
Action : Demote To Informational
Example: File_Set_6: When checking operator: No nodes with a disk in the "export" resource pool; using the default disk pool instead.
Posted: Wed Mar 05, 2008 2:27 am
by ray.wurlod
You will still get the text, but it should now have a green icon when you run the job. Also in the log will be two more entries, one (near the beginning of the job run) showing that message handlers have been attached to the job, the other (near the end) showing that message handlers were used.
Posted: Wed Mar 05, 2008 3:29 am
by tkbharani
Hi Ray
The log color is not changing ,its still yellow in color. And also in the beginning of the job there is no message handler logs.
Posted: Wed Mar 05, 2008 3:32 am
by ArndW
I just tested it and the demote works. I'm at V8 so my message-id is "IIS-DSEE-TOIX-00031" but I've used the message handling so much at V7.x that I know it works correctly.
You need to re-run the job after adding the handler for it to take effect.
its working
Posted: Wed Mar 05, 2008 3:47 am
by tkbharani
Its working.
The thing is I created Message handler ----> Edit Choose message handler--> New.
Thats where it was NOT Working.
-----
When I created Edit Local runtime handler for job. It was working.
Why is this...?
Posted: Wed Mar 05, 2008 4:01 am
by ray.wurlod
Because you probably missed some vital step in the New phase. It is really difficult to reconstruct the exact history of steps you took.
Warning in Fileset Stage while running the job in V8.5
Posted: Fri Jun 14, 2013 1:26 am
by KVSNR_123
The soution straight forward....without touching the COnfiguration files.
Open FIleset stage-->Input tab-->Options-->Slect Diskpool and assign 'LRF' to it. It should be work fine and wokring for me fine(DS8.5 version)
Posted: Fri Jun 14, 2013 4:58 am
by ray.wurlod
Welcome aboard. Are you sure these options were available in the release extant in 2008 when this thread was running? Your "solution" will only work if there is a disk pool named "LRF" in the configuration file. There is no evidence that such is the case for the original questioner.