compress-archive task failing in SSIS with 1.25GB file

Question

I am using the 'execute process task' to call the powershell compress-archive command.

For small files it works fine.

For a file of 1.25GB it is failing. It starts processing then 'finishes' but no file is created.

Given that the compress-archive should work with files up to 2GB, why is this occurring and what other options do I have?

7zip is not an option due to having no control over server installs.

Answer 1

Compress-Archive is a simplified wrapper around the System.IO.Compression.ZipArchive .Net library. Some features are not supported, such as storing relative path. Exception handling and reporting is weak. I usually choose to use the .Net library directly for any zipping task that requires robust exception handling.

The code isn't much more complicated with a transcript (always a good idea for unattended jobs) and error handling. This may not solve your problem, but it will likely give you more information:

$ZipPath = 'C:\Temp\test.zip'
$LogPath = "c:\Temp\Archiving-$(get-date -f yyyy-MM-dd).log"
$FilesToZip = (Get-ChildItem 'C:\Temp\*.csv' -File) #trying to compress the zip or log file will result in error: 'process cannot access the file ... in use'
@( 'System.IO.Compression','System.IO.Compression.FileSystem') | % { [void][Reflection.Assembly]::LoadWithPartialName($_) }
Try{
    Start-Transcript $LogPath -Append
    $WriteArchive = [IO.Compression.ZipFile]::Open( $ZipPath, 'Update')#Update mode also creates a zip
    ForEach($file in $FilesToZip){
        $ArchivedFile = [IO.Compression.ZipFileExtensions]::CreateEntryFromFile($WriteArchive, 
            $File.FullName, $file.Name, 'Optimal')
        Write-Host "Archived: $($File.FullName) bytes: $($file.Length)"
    }
}Catch [Exception]{
    Write-Error $_.Exception
}Finally{
    $WriteArchive.Dispose() #close the zip file so it can be read later     
    Stop-Transcript 
}

The roughly equivalent code using Compress-Archive:

Compress-Archive -Path 'C:\Temp\*.csv' -DestinationPath 'C:\Temp\test.zip' -Update

However Compress-Archive overwrites entries inside the .Zip archive on subsequent runs. CreateEntryFromFile creates duplicate entries within the archive.