DSXchange

Posted: **Tue Feb 01, 2005 11:52 pm**

I have a requirement to split a file, into N multiple files. N is dependent on size. Say one day I have 100MB file. I want to split the files of say 10MB(This value will be passed as parameter,will be constant) each. So I will have 10, 10MB files.Tommorrow if I have 60MB file, than I will have 6,10MB files. Each File should have name like "FileName1","FileName2"...so on.
I have written small Basic program, for that. But it's not working, it get's compiled, but while testing, It throws error.
---------------------------
TEST #1
*******

Arg1 = C:\Orders\
Arg2 = jcOrders.txt
Arg3 = 10

Test failed.

DSLogWarn called from : SplitFiles
Message to be logged is...
> Cannot open C:\Orders\jcOrders.txt
DSLogWarn called from : SplitFiles
Message to be logged is...
> Error while Reading File C:\Orders\jcOrders.txt status=-1
Program "DSU.SplitFile": Line 17, Improper data type.
---------------------------
Though The file "C:\Orders\jcOrders.txt" is there. Every input given is proper.

---------------------------
*Function SplitFile(FilePath,FileName,SizeOfSplit)
RoutineName='SplitFile'
$INCLUDE UNIVERSE.INCLUDE FILEINFO.H
Ans=''
OpenSeq FileName To FileVar Else
Call DSLogWarn("Cannot open ":FilePath:FileName, "SplitFiles")
ErrorCode=1
End
cnt=1
i=1
loop
ReadBlk FileLine From FileVar ,SizeOfSplit Else
Call DSLogWarn("Error while Reading File ":FilePath:FileName:" status=":Status(),"SplitFiles")
ErrorCode=1
End

WRITESEQ FileVar TO FilePath:FileName:i Else
Call DSLogWarn("Unable to write into":FilePath:FileName:i :" status=":Status(),"SplitFiles")
ErrorCode=1
End
i=i+1
ErrorCode=0
Repeat
If ErrorCode=1 then
Ans='FAILED'
End Else
Ans='SUCCESS'
End
Return(Ans)

Posted: **Wed Feb 02, 2005 12:04 am**

Further improvement please help in this, ignor the previous one. I spotted the knot, it was silly mistake.

EST #1
*******

Arg1 = C:\Orders\
Arg2 = jcOrders.txt
Arg3 = 10

Test failed.

Program "DSU.SplitFile": Line 17, Improper data type.
---------------------------
The code looks like this
---------------------------
*Function SplitFile(FilePath,FileName,SizeOfSplit)
RoutineName='SplitFile'
$INCLUDE UNIVERSE.INCLUDE FILEINFO.H
Ans=''
OpenSeq FilePath:FileName To FileVar Else
Call DSLogWarn("Cannot open ":FilePath:FileName :" SplitFiles")
ErrorCode=1
End

i=1
loop
ReadBlk FileLine From FileVar ,SizeOfSplit Else
Call DSLogWarn("Error while Reading File ":FilePath:FileName:" status=":Status() :" SplitFiles")
ErrorCode=1
End

WriteBlk FileVar TO FilePath:FileName:i Else
Call DSLogWarn("Unable to write into":FilePath:FileName:i :" status=":Status():" SplitFiles")
ErrorCode=1
End
i=i+1
ErrorCode=0
Repeat
If ErrorCode=1 then
Ans='FAILED'
End Else
Ans='SUCCESS'
End
Return(Ans)

Posted: **Wed Feb 02, 2005 7:13 am**

You need to open the file you are writing to. Just like you opened the one you are reading.

Posted: **Wed Feb 02, 2005 8:58 am**

Here's my own version I wrote for an after-job routine. It is fixed at 10 output files, but that easily can be a calculation based on the size of the file:

Code: Select all

      DEFFUN WordCount(FileName) Calling "DSU.WordCount"

      SplitFactor = 10
      InputFileName = InputArg
      WriteBufferCacheSize = 10000

      RoutineName = "SplitFileNT"

      Openseq InputFileName TO InputFileHandle Then

         LineCount = WordCount(InputFileName)

         LinesPerFile = OCONV(LineCount/SplitFactor, "MD0") + 1

         Suffix = 0

         For x = 1 to SplitFactor
            OutputFileName = InputFileName:"_":x
            Call DSU.OutputFile(OutputFileName, DSJ.ERRFATAL)
         Next x

         SourceFileExhausted = @FALSE
         CreateNextFile = @TRUE
         Loop
         Until SourceFileExhausted Do
            Readseq line From InputFileHandle Else SourceFileExhausted = @TRUE
*If SourceFileExhausted = @FALSE Then Call DSLogInfo(line, "Msg")
            If CreateNextFile Then
               Suffix += 1
               OutputFileName = InputFileName:"_":Suffix
               Openseq OutputFileName To OutputFileHandle Then
                  Call DSLogInfo("Opening output file: ":OutputFileName, RoutineName)
               End Else
                  Create OutputFileHandle Then
                     Call DSLogInfo("Creating output file: ":OutputFileName, RoutineName)
                  End Else
                     Call DSLogFatal("Unable to create file: ":OutputFileName, RoutineName)
                  End
               End
               CreateNextFile = @FALSE
               WriteBuffer = ""
               RowsWrittenThisFile = 0
               RowsWrittenThisBuffer = 0
            End
            If NOT(SourceFileExhausted) Then
               WriteBuffer<-1> = line
               RowsWrittenThisFile += 1
               RowsWrittenThisBuffer += 1
*Call DSLogInfo("Adding to buffer  RowsThisBuffer [":RowsWrittenThisBuffer:"] File [":RowsWrittenThisFile:"]":@AM:WriteBuffer, "Msg")
            End
            If RowsWrittenThisBuffer > WriteBufferCacheSize OR RowsWrittenThisFile > LinesPerFile OR SourceFileExhausted Then
               CONVERT @AM TO CHAR(10) IN WriteBuffer
               Writeseq WriteBuffer TO OutputFileHandle Else
                  Call DSLogFatal("Unable to write to file: ":OutputFileName, RoutineName)
               End
*Call DSLogInfo("Writing Buffer  RowsThisBuffer [":RowsWrittenThisBuffer:"] File [":RowsWrittenThisFile:"]":@AM:WriteBuffer, "Msg")
               WriteBuffer = ""
               RowsWrittenThisBuffer = 0
            End
            If RowsWrittenThisFile > LinesPerFile or SourceFileExhausted Then
               Call DSLogInfo("Closing output file: ":OutputFileName:" after ":RowsWrittenThisFile:" rows", RoutineName)
               WEOFSEQ OutputFileHandle
               Closeseq OutputFileHandle
               CreateNextFile = @TRUE
            End
         Repeat
         Closeseq InputFileHandle
      End Else
         Call DSLogFatal("Unable to open input file: ":InputFileName, RoutineName)
      End

Here's the WordCount function:

Code: Select all

      FullyQualifiedFileName = DOWNCASE(Arg1)
      FunctionName = "WordCount"
      SampleSize = 100000
      NTcmd = "dir ":FullyQualifiedFileName
      Call DSExecute("NT", NTcmd, ScreenOutput, SystemReturnCode)

      ScreenOutput = DOWNCASE(ScreenOutput)
      SlashCount = Count(FullyQualifiedFileName, "\")
      If SlashCount = 0 Then
         FileName = FullyQualifiedFileName
      End Else
         FileName = FIELD(FullyQualifiedFileName,"\", SlashCount+1)
      End
      FileNamePos = INDEX(ScreenOutput, FileName, 1)
      FileSize = TRIM(ScreenOutput[FileNamePos-14,14])
      CONVERT "," TO "" IN FileSize

      ApproxNumberOfLines = 0
      OPENSEQ FullyQualifiedFileName TO F.FILE THEN
         Done = @FALSE
         LineCnt = 0
         TotalSize = 0
         Loop
            READSEQ line From F.FILE Else Done = @TRUE
         Until Done Do
            size = LEN(line)
            TotalSize += size
            LineCnt += 1
            If LineCnt >= SampleSize Then Done = @TRUE
         Repeat
         CLOSESEQ F.FILE
         If LineCnt < SampleSize Then
            ApproxNumberOfLines = LineCnt
         End Else
            ApproxNumberOfLines = OCONV(FileSize / (TotalSize/SampleSize), "MD0")
         End
         AverageRowSize = OCONV((TotalSize/ApproxNumberOfLines), "MD0")
      END ELSE
         Call DSLogFatal("Unable to open file: ":FullyQualifiedFileName, FunctionName)
      End
      Ans = ApproxNumberOfLines
      Call DSLogInfo(FullyQualifiedFileName:" Size: [":FileSize:"] Avg Row Size: [":AverageRowSize:"] Apprx Row Count [":ApproxNumberOfLines:"]", FunctionName)

DSXchange

Can anyone Help me in reviewing My Basic Code.

Can anyone Help me in reviewing My Basic Code.