How to get around batch file processing limit
- by Patrick Cuff
I have a Windows batch file that processes all the files in a given directory. I have 206,783 files I need to process:
for %%f in (*.xml) do call :PROCESS %%f
goto :STOP
:PROCESS
:: do something with the file
program.exe %1 > %1.new
set /a COUNTER=%COUNTER%+1
goto :EOF
:STOP
@echo %COUNTER% files processed
When I run the batch file, the following output is written:
65535 files processed
As part of the processing, an output file is created for each file procesed, with a .new extension. When I do a dir *.new it reports 65,535 files exist.
So, it appears my command environment has a hard limit on the number of files it can recognize, and that limit is 64K - 1.
Is there a way to extend the command environment to manage more than 64K - 1 files?
If not, would a VBScript or JavaScript be able to process all 206,783 files?
I'm running on Windows 2003 server, Enterprise Edition, 32-bit.
UPDATE
It looks like the root cause of my issue was with the built-in Windows "extract" command for ZIP files.
The files I have to process were copied from another system via a ZIP file. My server doesn't have a ZIP utility installed, just the native Windows commands. I right-clicked on the ZIP file, and did an "Extract all...", which apparently just extracted the first 65,535 files.
I downloaded and installed 7-zip onto my server, unzipped all the files, and my batch script worked as intended.
