I have a requirement to split a file, into N multiple files. N is dependent on size. Say one day I have 100MB file. I want to split the files of say 10MB(This value will be passed as parameter,will be constant) each. So I will have 10, 10MB files.Tommorrow if I have 60MB file, than I will have 6,10MB files. Each File should have name like "FileName1","FileName2"...so on.
I have written small Basic program, for that. But it's not working, it get's compiled, but while testing, It throws error.
---------------------------
TEST #1
*******
Arg1 = C:\Orders\
Arg2 = jcOrders.txt
Arg3 = 10
Test failed.
DSLogWarn called from : SplitFiles
Message to be logged is...
> Cannot open C:\Orders\jcOrders.txt
DSLogWarn called from : SplitFiles
Message to be logged is...
> Error while Reading File C:\Orders\jcOrders.txt status=-1
Program "DSU.SplitFile": Line 17, Improper data type.
---------------------------
Though The file "C:\Orders\jcOrders.txt" is there. Every input given is proper.
---------------------------
*Function SplitFile(FilePath,FileName,SizeOfSplit)
RoutineName='SplitFile'
$INCLUDE UNIVERSE.INCLUDE FILEINFO.H
Ans=''
OpenSeq FileName To FileVar Else
Call DSLogWarn("Cannot open ":FilePath:FileName, "SplitFiles")
ErrorCode=1
End
cnt=1
i=1
loop
ReadBlk FileLine From FileVar ,SizeOfSplit Else
Call DSLogWarn("Error while Reading File ":FilePath:FileName:" status=":Status(),"SplitFiles")
ErrorCode=1
End
WRITESEQ FileVar TO FilePath:FileName:i Else
Call DSLogWarn("Unable to write into":FilePath:FileName:i :" status=":Status(),"SplitFiles")
ErrorCode=1
End
i=i+1
ErrorCode=0
Repeat
If ErrorCode=1 then
Ans='FAILED'
End Else
Ans='SUCCESS'
End
Return(Ans)
Can anyone Help me in reviewing My Basic Code.
Moderators: chulett, rschirm, roy
-
- Charter Member
- Posts: 199
- Joined: Tue Jan 18, 2005 2:50 am
- Location: India
-
- Charter Member
- Posts: 199
- Joined: Tue Jan 18, 2005 2:50 am
- Location: India
Further improvement please help in this, ignor the previous one. I spotted the knot, it was silly mistake.
EST #1
*******
Arg1 = C:\Orders\
Arg2 = jcOrders.txt
Arg3 = 10
Test failed.
Program "DSU.SplitFile": Line 17, Improper data type.
---------------------------
The code looks like this
---------------------------
*Function SplitFile(FilePath,FileName,SizeOfSplit)
RoutineName='SplitFile'
$INCLUDE UNIVERSE.INCLUDE FILEINFO.H
Ans=''
OpenSeq FilePath:FileName To FileVar Else
Call DSLogWarn("Cannot open ":FilePath:FileName :" SplitFiles")
ErrorCode=1
End
i=1
loop
ReadBlk FileLine From FileVar ,SizeOfSplit Else
Call DSLogWarn("Error while Reading File ":FilePath:FileName:" status=":Status() :" SplitFiles")
ErrorCode=1
End
WriteBlk FileVar TO FilePath:FileName:i Else
Call DSLogWarn("Unable to write into":FilePath:FileName:i :" status=":Status():" SplitFiles")
ErrorCode=1
End
i=i+1
ErrorCode=0
Repeat
If ErrorCode=1 then
Ans='FAILED'
End Else
Ans='SUCCESS'
End
Return(Ans)
EST #1
*******
Arg1 = C:\Orders\
Arg2 = jcOrders.txt
Arg3 = 10
Test failed.
Program "DSU.SplitFile": Line 17, Improper data type.
---------------------------
The code looks like this
---------------------------
*Function SplitFile(FilePath,FileName,SizeOfSplit)
RoutineName='SplitFile'
$INCLUDE UNIVERSE.INCLUDE FILEINFO.H
Ans=''
OpenSeq FilePath:FileName To FileVar Else
Call DSLogWarn("Cannot open ":FilePath:FileName :" SplitFiles")
ErrorCode=1
End
i=1
loop
ReadBlk FileLine From FileVar ,SizeOfSplit Else
Call DSLogWarn("Error while Reading File ":FilePath:FileName:" status=":Status() :" SplitFiles")
ErrorCode=1
End
WriteBlk FileVar TO FilePath:FileName:i Else
Call DSLogWarn("Unable to write into":FilePath:FileName:i :" status=":Status():" SplitFiles")
ErrorCode=1
End
i=i+1
ErrorCode=0
Repeat
If ErrorCode=1 then
Ans='FAILED'
End Else
Ans='SUCCESS'
End
Return(Ans)
Here's my own version I wrote for an after-job routine. It is fixed at 10 output files, but that easily can be a calculation based on the size of the file:
Here's the WordCount function:
Code: Select all
DEFFUN WordCount(FileName) Calling "DSU.WordCount"
SplitFactor = 10
InputFileName = InputArg
WriteBufferCacheSize = 10000
RoutineName = "SplitFileNT"
Openseq InputFileName TO InputFileHandle Then
LineCount = WordCount(InputFileName)
LinesPerFile = OCONV(LineCount/SplitFactor, "MD0") + 1
Suffix = 0
For x = 1 to SplitFactor
OutputFileName = InputFileName:"_":x
Call DSU.OutputFile(OutputFileName, DSJ.ERRFATAL)
Next x
SourceFileExhausted = @FALSE
CreateNextFile = @TRUE
Loop
Until SourceFileExhausted Do
Readseq line From InputFileHandle Else SourceFileExhausted = @TRUE
*If SourceFileExhausted = @FALSE Then Call DSLogInfo(line, "Msg")
If CreateNextFile Then
Suffix += 1
OutputFileName = InputFileName:"_":Suffix
Openseq OutputFileName To OutputFileHandle Then
Call DSLogInfo("Opening output file: ":OutputFileName, RoutineName)
End Else
Create OutputFileHandle Then
Call DSLogInfo("Creating output file: ":OutputFileName, RoutineName)
End Else
Call DSLogFatal("Unable to create file: ":OutputFileName, RoutineName)
End
End
CreateNextFile = @FALSE
WriteBuffer = ""
RowsWrittenThisFile = 0
RowsWrittenThisBuffer = 0
End
If NOT(SourceFileExhausted) Then
WriteBuffer<-1> = line
RowsWrittenThisFile += 1
RowsWrittenThisBuffer += 1
*Call DSLogInfo("Adding to buffer RowsThisBuffer [":RowsWrittenThisBuffer:"] File [":RowsWrittenThisFile:"]":@AM:WriteBuffer, "Msg")
End
If RowsWrittenThisBuffer > WriteBufferCacheSize OR RowsWrittenThisFile > LinesPerFile OR SourceFileExhausted Then
CONVERT @AM TO CHAR(10) IN WriteBuffer
Writeseq WriteBuffer TO OutputFileHandle Else
Call DSLogFatal("Unable to write to file: ":OutputFileName, RoutineName)
End
*Call DSLogInfo("Writing Buffer RowsThisBuffer [":RowsWrittenThisBuffer:"] File [":RowsWrittenThisFile:"]":@AM:WriteBuffer, "Msg")
WriteBuffer = ""
RowsWrittenThisBuffer = 0
End
If RowsWrittenThisFile > LinesPerFile or SourceFileExhausted Then
Call DSLogInfo("Closing output file: ":OutputFileName:" after ":RowsWrittenThisFile:" rows", RoutineName)
WEOFSEQ OutputFileHandle
Closeseq OutputFileHandle
CreateNextFile = @TRUE
End
Repeat
Closeseq InputFileHandle
End Else
Call DSLogFatal("Unable to open input file: ":InputFileName, RoutineName)
End
Code: Select all
FullyQualifiedFileName = DOWNCASE(Arg1)
FunctionName = "WordCount"
SampleSize = 100000
NTcmd = "dir ":FullyQualifiedFileName
Call DSExecute("NT", NTcmd, ScreenOutput, SystemReturnCode)
ScreenOutput = DOWNCASE(ScreenOutput)
SlashCount = Count(FullyQualifiedFileName, "\")
If SlashCount = 0 Then
FileName = FullyQualifiedFileName
End Else
FileName = FIELD(FullyQualifiedFileName,"\", SlashCount+1)
End
FileNamePos = INDEX(ScreenOutput, FileName, 1)
FileSize = TRIM(ScreenOutput[FileNamePos-14,14])
CONVERT "," TO "" IN FileSize
ApproxNumberOfLines = 0
OPENSEQ FullyQualifiedFileName TO F.FILE THEN
Done = @FALSE
LineCnt = 0
TotalSize = 0
Loop
READSEQ line From F.FILE Else Done = @TRUE
Until Done Do
size = LEN(line)
TotalSize += size
LineCnt += 1
If LineCnt >= SampleSize Then Done = @TRUE
Repeat
CLOSESEQ F.FILE
If LineCnt < SampleSize Then
ApproxNumberOfLines = LineCnt
End Else
ApproxNumberOfLines = OCONV(FileSize / (TotalSize/SampleSize), "MD0")
End
AverageRowSize = OCONV((TotalSize/ApproxNumberOfLines), "MD0")
END ELSE
Call DSLogFatal("Unable to open file: ":FullyQualifiedFileName, FunctionName)
End
Ans = ApproxNumberOfLines
Call DSLogInfo(FullyQualifiedFileName:" Size: [":FileSize:"] Avg Row Size: [":AverageRowSize:"] Apprx Row Count [":ApproxNumberOfLines:"]", FunctionName)
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle