Passing values of a text file to User variables.

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
samratisking
Participant
Posts: 37
Joined: Tue Jan 29, 2008 6:03 am
Location: Guntur
Contact:

Passing values of a text file to User variables.

Post by samratisking »

Hi Friends,

I have a requirement in which i need to pass values in different columns in a txt file or an csv file to a User Variables stage. How can I achieve this? Please help me with this issue. Thankyou.
Samratisking
nayanpatra
Participant
Posts: 41
Joined: Sat Jun 06, 2009 11:13 pm
Location: Kolkata

Sequential file read

Post by nayanpatra »

You can perform the read with the sequential file stage. And then can move the column values into your user variables. There are many stages with which you can do this. If you are simply moving the data into user variables then you can use a copy stage. If you are transforming the data you can use a transformer. Just check the requirement and use the correct stage accordingly.
Nayan
samratisking
Participant
Posts: 37
Joined: Tue Jan 29, 2008 6:03 am
Location: Guntur
Contact:

Re: Sequential file read

Post by samratisking »

nayanpatra wrote:You can perform the read with the sequential file stage. And then can move the column values into your user variables. There are many stages with which you can do this. If you are simply moving the data into user variables then you can use a copy stage. If you are transforming the data you can use a transformer. Just check the requirement and use the correct stage accordingly.
Hi Nayan,

I wanted to do this in Sequences.. So I cant use transformers etc.. Is there a way that I can get the values in a text file to the User Variables stage, from where I can pass those values to corresponding jobs??? Thankyou.
Samratisking
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

You will need to shell out to UNIX, perform the selection there and then return the results to your job sequence. What utility you use to do this depends upon how complex your selection is.
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Will the same job run in more than one invocation with different parameters at the same time?

If no, you can try the following
1.) take copy of DSParams to some default
2.) append your list of parameters and make it DSParam

This way, you can use environment variables which will be assigned by your external text file.

Alternatively there are several routines and shell scripts floating around which can be used to extract parameters from flat file or tables.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

:!: I for one would never suggest anyone monkey with their DSParam files in this manner or try replacing it with a job specific list. There's no need for any kind of 'trick' like this, never mind the fact that it can be... dangerous.

As noted, getting values from a flat file can be as simple as a 'cat' to echo them to standard out or as complicated of a custom routine you may need. We would need more details to give more detailed help, however. How many fields from this flat file do you need? Are they all individual job parameter values? If so, perhaps this post will help.
-craig

"You can never have too many knives" -- Logan Nine Fingers
nayanpatra
Participant
Posts: 41
Joined: Sat Jun 06, 2009 11:13 pm
Location: Kolkata

Post by nayanpatra »

Yeah, you can try it with the Unix commands. But if any kind of manipulation is required then you may go for a shell script.
Nayan
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

A simple cat command (Execute Command) upstream of the User Variables stage should suffice. The user variable derivations can use Field() functions to extract the relevant columns from the row that was read.
(I've assumed that the file has one line. Without that assumption the requirement makes little or no sense.)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
tbtcust
Premium Member
Premium Member
Posts: 230
Joined: Tue Mar 04, 2008 9:07 am

Post by tbtcust »

Hi All

I have set up a Sequence Job that uses a shared parm set. The Sequence Job looks like this:
Execute_Command---> UserVariables ---> Job_Activity

I execute the cat command in the Execute_Command
The output is trapped in the UserVariables via Execute_Command.$CommandOutput
I can't get the output into the Job_Activity shared parm set.

Can someone help?

Thanks in advance for any help.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Rather than hijack / sidetrack someone else's slightly related post, please start your own.
-craig

"You can never have too many knives" -- Logan Nine Fingers
tbtcust
Premium Member
Premium Member
Posts: 230
Joined: Tue Mar 04, 2008 9:07 am

Post by tbtcust »

Apologies chulett. Since this seems to be a solution I could use I thought it made sense to continue the conversation. Will send a new post.

Thanks.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

tbtcust wrote:this seems to be a solution I could use
Somewhat but now we know you have an 8.x/PX question unlike this 7.x/Server issue you posted into. And a parameter set doesn't really equate well to the User Variables stage. Plus you can decide when the issue is resolved over there. So... new post. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply