Split a big folder

[ongtran]

Hi all,

I have a big folder its size is 3G I like to split it to multiple folders.Everyone has a suggestion please give me. Thanks

[Killer42]

I have a big folder its size is 3G I like to split it to multiple folders.Everyone has a suggestion please give me. Thanks

Can you provide some more details about your situation?

My first reponse would be, why don't you just use Windows Explorer to create another folder, and move half your files into it?

Does this question have anything to do with Visual Basic?

[ongtran]

Good Morning Mod,

I have no clue to do. Please show me the way.

I like to split a big folder to 10-20 small folders. I like to use VB or any kind software.

Browse a folder---->split button (command) .... It will create multiple folders on the path that is assigned and transfer files to these folders.

Have a good weekend.

[ongtran]

My folder is big including over 40000 files and everytime it is opened and taken time on that. I like to divide it to mulitple folders and each folers has around 1000 files depends on the size of each file.

I like to set for each folder has a limit size 100M.

The first reason I don't do manually that I like to learn how to split, second I will have more big folders so I like to save time in the future.

[kadghar]

Good Morning Mod,

I have no clue to do. Please show me the way.

I like to split a big folder to 10-20 small folders. I like to use VB or any kind software.

Browse a folder---->split button (command) .... It will create multiple folders on the path that is assigned and transfer files to these folders.

Have a good weekend.

Well, you can always use VBA as a file manager, I have no idea what you want it for, but i had fun making it. here you go : (change NumFolders in order to make 10 or 20 or whatever u want)

Sub carpetas()
Dim i As Integer
Dim j As Integer
Dim Str1 As String
Dim Int1 As Integer
Dim Obj1 As Object
Dim NumFolders As Integer
 
NumFolders = 10
 
 
With Application.FileDialog(msoFileDialogFolderPicker)
    .AllowMultiSelect = False
    .Show
    If .SelectedItems.Count > 0 Then
        Str1 = Dir(.SelectedItems.Item(1) & "\" & "*.*")
        Set Obj1 = CreateObject("Scripting.FileSystemObject")
        For j = 1 To NumFolders
            Obj1.CreateFolder .SelectedItems.Item(1) & "\Folder" & j
        Next
            j = 1
        While Str1 <> ""
            Obj1.CopyFile .SelectedItems.Item(1) & "\" & Str1, .SelectedItems.Item(1) & "\" & "Folder" & j & "\" & Str1, False
            Obj1.DeleteFile .SelectedItems.Item(1) & "\" & Str1, True
            Str1 = Dir()
            j = j + 1
            If j = NumFolders + 1 Then
                j = 1
            End If
        Wend
    End If
End With
 
End Sub

[CoMPMStR]

My folder is big including over 40000 files and everytime it is opened and taken time on that. I like to divide it to mulitple folders and each folers has around 1000 files depends on the size of each file.

I like to set for each folder has a limit size 100M.

The first reason I don't do manually that I like to learn how to split, second I will have more big folders so I like to save time in the future.

I was thinking more of something like this, it may take a while with 40000+ files but it works just fine...

    Private Sub sortFolder(ByVal selFolder As String, ByVal newFolName As String)
        On Error Resume Next
 
        'set the selected folder
        Dim selPath As String = selFolder
 
        'set the size variable to calculate the total size of the new folder
        Dim newSize As Integer = 0
 
        'get the directory info for the selected folder
        Dim dInfo As New System.IO.DirectoryInfo(selPath)
 
        'iterate through the the first thousand files in the selected folder
        For i As Integer = 0 To 999
 
            'if we reach the end of the list, exit the for loop
            If i >= dInfo.GetFiles.Length Then Exit For
 
            'set the folder name (you can change to whatever you want)
            Dim folderName As String = newFolName
 
            'set the new folder path
            Dim curFolder As String = selPath & "\" & folderName
 
            'make the new folder
            MkDir(curFolder)
 
            'if we reach the end of the list then we exit
            If i >= dInfo.GetFiles.Length Then Exit For
 
            'set the current file
            Dim curFile As String = dInfo.GetFiles.GetValue(i).ToString
 
            'copy the file to the new folder
            FileCopy(selPath & "\" & curFile, curFolder & "\" & curFile)
 
            'delete the old file
            Kill(selPath & "\" & curFile)
 
            'set the new folder directory info and size to zero
            Dim newFolderInfo As New System.IO.DirectoryInfo(curFolder)
            newSize = 0
 
            'iterate through the files in the new folder and adds the size of each
            For n As Integer = 0 To newFolderInfo.GetFiles.Length - 1
                newSize += FileLen(curFolder & "\" & newFolderInfo.GetFiles.GetValue(n).ToString)
            Next
 
            'if the new folder's size is greater than or equal to 100MB we exit the loop
            If newSize >= 104857600 Then
                Exit For
            End If
        Next
 
        'display complete message
        MsgBox("File sort complete" & vbCrLf & "New Folder - " & newFolName & " - " & _
        FormatNumber(newSize / 1024 / 1024, 2) & " MB")
    End Sub
 

Put this sub in your project and it will go through the first thousand files in selFolder, create a new folder in the selected folder named newFolName. All the files will go in this folder until it reaches 1000 files or 100MB.

If you want to do it for all of the files in the folder you should use it like the following: (put in a button_click event)

        Dim folBrowse As New FolderBrowserDialog
        folBrowse.ShowDialog()
 
        Dim dirInfo As New System.IO.DirectoryInfo(folBrowse.SelectedPath)
        Dim index As Integer = 0
 
        Do Until dirInfo.GetFiles.Length = 0
            sortFolder(folBrowse.SelectedPath, "newfolder" & index)
            index += 1
        Loop
 
        MsgBox("Completely done")
 

This little piece will open a folder browser dialog and go through the selected folder until the file count reaches 0. You can change newfolder to anything you wish, but it will append a number to the end of the folder to make it unique. Hopefully this helps.

[Killer42]

Thanks for that, CoMPMStR. Just one quibble - won't you overflow newSize rather quickly?

I generally discourage the use of the Integer type for anything, unless you desperately need to save memory. The Long type is actually slightly faster to process, at least on a 32 bit processor.

[CoMPMStR]

Thanks for that, CoMPMStR. Just one quibble - won't you overflow newSize rather quickly?

I generally discourage the ue of the Integer type for anything, unless you desperately need to save memory. The Long type is actually slightly faster to procss, at least on a 32 bit processor.

I'm always glad to help if I can. I just hope he's using .NET and not VB6. =/ I didn't know that the variable type would determine how fast an execution is processed. I always thought it was the CPU speed that determined process speed. The newSize variable will only overflow if the total filesizes in the new folder are greater than or equal to 2GB right? Or is it 4GB in .NET? So that means the next file in the list would have to be really large, like the size of a DVD image. It will never get that big because it always resets back to zero before adding the filesizes, and also, the 100MB check. If newSize is greater than 100MB it will exit, this should prevent it from any overflow errors. If not, it can simply be changed.

[Killer42]

I'm always glad to help if I can. I just hope he's using .NET and not VB6. =/ I didn't know that the variable type would determine how fast an execution is processed. I always thought it was the CPU speed that determined process speed.

Certainly the processor speed is a much bigger factor. But different instructions can take different numbers of cycles to complete, of course. A program which does twice the work will always take twice as long, regardless of how long that is.

The plain fact is that on a 32 bit processor, obviously, processing a 32 bit (four-byte) chunk of data takes one instruction (however many cycles that may require). But to work with a 16-bit (two-byte) Integer (I'm talking about VB6 now) requires extra instructions to convert back and forth. (The 32-bit "long integer" is thus referred to as the "native" data type). It's not a huge difference, of course, but I've always figured that unless you have a good reason to overlook it, why not use the quicker type? Or to put it another way, when weighing up which way to go, a small boost in processing speed (32-bit Long) is generally worth more than a very slight reduction in memory useage (16-bit Integer).

The newSize variable will only overflow if the total filesizes in the new folder are greater than or equal to 2GB right? Or is it 4GB in .NET? So that means the next file in the list would have to be really large, like the size of a DVD image. It will never get that big because it always resets back to zero before adding the filesizes, and also, the 100MB check. If newSize is greater than 100MB it will exit, this should prevent it from any overflow errors. If not, it can simply be changed.

I don't know about VB.Net, suppose we'll have to check up on that. But in VB6 the Integer data type is a 16-bit integer, which will overflow at 32,768. This is why I brought up the subject, though it looks as though I may be simply out of date.

[Killer42]

Ok, further info from MSDN.

You're right, Integer, in VB.Net, is now a 32-bit integer. See http://msdn2.microsoft.com/en-us/lib...z3(VS.71).aspx for a table listing the differences. But in brief, VB6 "Integer" = VB.Net "Short" while VB6 "Long" = Vb.Net "Integer".

[ongtran]

Thanks for your responses. I will try to do my stuff with all your ideas.

[ongtran]

What is the different .net and VB? do i need to get software for .net? I'm just a beginner in programming. Thanks

[Killer42]

What is the different .net and VB? do i need to get software for .net? I'm just a beginner in programming. Thanks

.Net is Microsoft's new framework for doing things in Windows. I don't know how to describe it better - go to MS, or maybe Wikipedia would be better, for more info.

VB has been around for a long time, and was up to version 6 before MS introduced the .Net framework. So VB6 is the last "pre-dotNet" version. And in the opinion of certain people, still the best available. VB.Net refers to the versions created to work with the .Net framework. I can't really comment on it, since I still use VB6.

If you are planning to learn one, I would definitely recommend you go with the latest version (VB 2005, I believe). Although I love VB6, it is at least 9 years old. It will make much more sense to learn an up-to-date development environment. (Plus, VB 2005 Express edition is available as a free download, apparently.)

[ongtran]

Thanks all of you help me to know more VB.Net

Dim selPath As String = selFolder
Dim newSize As Integer = 0
Dim i As Integer

Dim dInfo As New System.IO.DirectoryInfo(selPath)
Dim folderName As String = newFolName
Dim curFile As String
Dim curFolder As String = selPath & "\" & folderName


MkDir(curFolder)

For i = 0 To 9
If i >= dInfo.GetFiles.Length Then
Exit For

End If




curFile = dInfo.GetFiles.Clone(i).ToString
FileCopy(selPath & "\" & curFile, curFolder & "\" & curFile) Kill(selPath & "\" & curFile)


Next 'end of of i

If I have 6 files and I assign i is reached to 9. It will created 3 folders that contain 3 files, 2 files and 1 file.

When I watch a value of i =0,1,2 then these steps happen

curFile = dInfo.GetFiles.Clone(i).ToString
FileCopy(selPath & "\" & curFile, curFolder & "\" & curFile)
Kill(selPath & "\" & curFile)

Then i =3..9, curFile still get a value of i=2.

How can I fix it. Please give me a hint. Thanks.

[evanderburg]

I was out looking for information on how to do just what you guys are doing. Thanks so much for the code. This is a big help. I am going to work on creating a program to automate one of our in house processes at my company where we have to split folders into smaller chunks for processing.

[Killer42]

I was out looking for information on how to do just what you guys are doing. Thanks so much for the code. This is a big help. I am going to work on creating a program to automate one of our in house processes at my company where we have to split folders into smaller chunks for processing.

Glad you found this helpful. :)

That's what is so great about an open forum like this. Apparently at some sites you can read the questions, but have to pay to read the answers - ugh!