Fold Basic Routine, Limit on string length?

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Fold Basic Routine, Limit on string length?

Post by roy »

Hi Everyone,
I've had to implement Base64 decoding in DS Basic.
In short every 4 characters represent 3 characters,
so I used the Fold function to create sets of 4 characters to work with.
In one case where I convert up to 7k+ characters the job hangs after 50 rows.
10 rows run ok.
Does anyone know if the Fold function has limits or perhaps the Basic proccess reaches a certain limit?
I'm reading from a file and writing to a file
taking 3 columns:
1. @INROWNUM
2. Len(Data)
3. Ereplace(Base64Decode(Data[1,10000]), Char(13):Char(10), " ", 0 , 0)
Any Ideas would be welcome ...
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

No clue but if you suspect the issue lies with the Fold function, seems to me you could replace it with substring rather easily.
Last edited by chulett on Sun Mar 11, 2012 1:49 pm, edited 1 time in total.
-craig

"You can never have too many knives" -- Logan Nine Fingers
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

Have you tried the FoldDP() function for NLS?

If that doesn't help, can you provide enough detail, with before/after example, to recreate the problem?
Choose a job you love, and you will never have to work a day in your life. - Confucius
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

The source data is in Base64 format!
Therefore it has no NLS characters in it!
the output data after coversion is another thing!
This is the whole idea of Base64; to keep data managable for DB or movement via data protocols while nutralizing the whole NLS issues.
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Wel I got the Fold out but still no change :cry:
Guess this wraps the thread...
I did get to a limit on testing the input string via the routine test option and hit the old 8192 string length limit notification, actual was 8180 anf actual length with no data problems was around 8144 characters
IHTH (I Hope This Helps) ...
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Re: Fold Basic Routine, Limit on string length?

Post by qt_ky »

NLS was mentioned in the original post, so I had to ask.

Is your job still hanging, or is it able to process around 8144 characters at a time, no problem?
Choose a job you love, and you will never have to work a day in your life. - Confucius
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Re: Fold Basic Routine, Limit on string length?

Post by roy »

qt_ky wrote:NLS was mentioned in the original post, so I had to ask.

Is your job still hanging, or is it able to process around 8144 characters at a time, no problem?
The 8144 characte limit was experianced in the routine test environment,
the actual data tested had less then 8,000 charaters.

I ended up keeping the regular ascii values and manually converting the NLS values I got to unicode ones that my computer could handle and transformed everythig else to spaces.
This means I might loose some info,but in order to support anything else I need exact instructions on how to transform givven input.
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

So Roy, is this Resolved or Worked Around or are you still futzing with this? :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply