Robocopy/7-zip bat file with high compression ratio



  • Re: Backup plan using Robocopy + 7-zip command line (7za.exe)
    @@@@@@@@@@@@ Thought I should share @@@@@@@@@@@@

    This script was adapted from another sourcecode but modified script to accommodate Robocopy.
    does the following:
    Initial setup provides whether to have "full", "weekly incremental" or "alternate day" backups.
    1.) copies the folders using robocopy with multi-thread,
    2.) 7z the files with highest compression,
    3.) adds the checksum at the end of the archive as a separate text file.

    
    @echo off
    ::**** start debug syntax ***
    ::call :main >debug_Logfile_BackupScript_revC.txt 2>&1
    ::pause
    ::exit/b
    ::**** end debug syntax ***
    ::**** start INI file  ***
    IF EXIST "TS-Backup_revD.ini" DEL /f /q "TS-Backup_revD.ini"
    call :build-config >TS-Backup_revD.ini 2>&1 
    :build-config
    echo # Enter file and folder names, one per line. > TS-Backup_revD.ini
    echo C:\21EC2020 >> TS-Backup_revD.ini
    ::echo C:\21EC2020\writeup\folder to back-up >> TS-Backup_revD.ini
    ::**** end of INI file ***
    REM BackupScript
    REM Version 1.01, Updated: 2008-05-21
    REM By Jason Faulkner (articles[-at-]132solutions.com)
    :::::http://www.pcmech.com/article/windows-batch-script-to-backup-data/
    :::::::::http://jasonfaulkner.com/RandomChars.aspx
    
    REM Performs full or incremental backups of folders and files configured by the user.
    
    REM Usage---
    REM   > BackupScript
    
    ::: -- Version History GÇô-
    :::(Major version).(Minor version).(Revision number).(Build number)
    :::          XX.XXX         YYYYMMDD Author      Description
    SET "version=01.1-beta  &:20080521 J.Faulkner  initial version, providing the framework
    SET "version=01.0.1"    &:20121031 C.Reyes     framework ready
    SET "version=01.0.1"    &:20121031 C.Reyes     Set default settings
    SET "version=01.0.2"    &:20141208 C.Reyes     Added windows 7 parameters
    SET "version=01.1.0"    &:20160828 C.Reyes     modded xcopy to robocopy windows 7 parameters
    SET "version=01.1.1"    &:20160828 C.Reyes     changed date format to accomomdate robocopy windows 7 parameters
    SET "version=01.1.2"    &:20160830 C.Reyes     Added Log parameter from robocopy
    SET "version=01.2.0"    &:20191030 C.Reyes     increased compression size x10
    SET "version=01.2.1"    &:20191101 C.Reyes     Added timestamp
    SET "version=01.2.2"    &:20191101 C.Reyes     Added HASH (SHA256) file for user checksum validation
    SET "version=01.2.2"    &:20191114 C.Reyes     updated 7z to highest compression ratio syntax to all compatible 7z versions
    SET "title=%~nx0 - version %version% #####  Compressing Backup. DO NOT CLOSE   #####"
    title %title%
    SET SUBDIR=%~dp0
    SETLOCAL ENABLEEXTENSIONS ENABLEDELAYEDEXPANSION
    :main
    ::mode 50,3
    color 0e
    echo.
    echo.   This will take awhile. Please Wait....
    echo.
    REM ---Configuration Options---
    
    REM Folder location where you want to store the resulting backup archive.
    REM This folder must exist. Do not put a '\' on the end, this will be added automatically.
    REM You can enter a local path, an external drive letter (ex. F:) or a network location (ex. \\server\backups)
    :: windows XP OS
    ::SET BackupStorage=C:\Bkup_assembly_logfile
    :: Windows 7 OS
    SET BackupStorage=%subdir%
    
    REM Which day of the week do you want to perform a full backup on?
    REM Enter one of the following: Sun, Mon, Tue, Wed, Thu, Fri, Sat, *
    REM Any day of the week other than the one specified below will run an incremental backup.
    REM If you enter '*', a full backup will be run every time.
    SET FullBackupDay=*
    rem only used for major large backups over 1 terabyte 
    ::SET FullBackupDay= Tue
    
    REM Location where 7-Zip is installed on your computer.
    REM The default is in a folder, '7-Zip' in your Program Files directory.
    SET InstallLocationOf7Zip=%ProgramFiles%\7-Zip
    
    set hh=%time:~0,2%
    if "%hh:~0,1%" == " " @set hh=0%hh:~1,1%
    set min=%time:~3,2%
    set ss=%time:~6,2%
    set timeStamp=-%hh%.%min%.%ss%
    
    REM +-----------------------------------------------------------------------+
    REM | Do not change anything below here unless you know what you are doing. |
    REM +-----------------------------------------------------------------------+
    
    REM Usage variables.
    Set logs=%subdir%
    SET exe7Zip=%InstallLocationOf7Zip%\7z.exe
    SET dirTempBackup=%TEMP%\backup
    SET filBackupConfig=%subdir%\TS-Backup_revD.ini
    IF EXIST "%logs%%nowdate%%timeStamp%_Userdrive_log.txt" DEL /f /q "%logs%%nowdate%%timeStamp%_Userdrive_log.txt"
    REM Validation.
    IF NOT EXIST %filBackupConfig% (
      ECHO No configuration file found, missing: %filBackupConfig%
      GOTO End
    )
    IF NOT EXIST "%exe7Zip%" (
      ECHO 7-Zip is not installed in the location: %dir7Zip%
      ECHO Please update the directory where 7-Zip is installed.
      GOTO End
    )
    
    REM Backup variables.
    FOR /f "tokens=1,2,3,4 delims=/ " %%a IN ('date /t') DO (
      SET DayOfWeek=%%a
      SET NowDate=%%d-%%b-%%c
      SET FileDate=%%d%%b%%c
      rem SET FileDate=%%b-%%c-%%d
    )
    
    IF {%FullBackupDay%}=={*} SET FullBackupDay=%DayOfWeek%
    IF /i {%FullBackupDay%}=={%DayOfWeek%} (
      SET txtBackup=Full
      SET swXCopy=/e 
    ) ELSE (
      SET txtBackup=Incremental
      SET swXCopy=/s /maxage:%FileDate%
    )
    
    ECHO Starting to copy files.
    IF NOT EXIST "%dirTempBackup%" MKDIR "%dirTempBackup%"
    FOR /f "skip=1 tokens=*" %%A IN (%filBackupConfig%) DO (
      SET Current=%%~A
      IF NOT EXIST "!Current!" (
        ECHO ERROR! Not found: !Current!
      ) ELSE (
        ECHO Copying: !Current!
        SET Destination=%dirTempBackup%\!Current:~0,1!%%~pnxA
        REM Determine if the entry is a file or directory.
        IF "%%~xA"=="" (
          REM Directory. Ensure the /L is removed before final archive otherwise the zipfile is empty.
          roboCOPY "!Current!" "!Destination!" *.ini *.rcj *.wsf *.ttl *.cmd *.bat *.HTA *.vbs *_CSCcode_*.txt /NDL /NFL /xo /nc /np /ns /ia:RASHCE /r:4 /w:5 /xx /is /MT:128 %swXCopy% /log+:"%logs%%nowdate%%timeStamp%_Userdrive_log.txt"
        )
      )
    )
    ECHO. Done copying files.
    ECHO.
    
    SET BackupFileDestination=%BackupStorage%\TS-scripts_Backup_%FileDate%%timeStamp%_%txtBackup%.7z
    set BackupFileDestination-hash=%BackupStorage%\TS-scripts_Backup_%FileDate%%timeStamp%_%txtBackup%.txt
    
    REM If the backup file exists, remove it in favor of the new file.
    IF EXIST "%BackupFileDestination%" DEL /f /q "%BackupFileDestination%"
    IF EXIST "%BackupFileDestination-hash%" DEL /f /q "%BackupFileDestination-hash%"
    
    ECHO Compressing backed up files. (New window)
    REM Compress files using 7-Zip in a lower priority process.
    
    Rem This method may have a High "CPU usage"
    :: 7z switch definitions https://superuser.com/questions/281573/what-are-the-best-options-to-use-when-compressing-files-using-7-zip/1449735
    START "Compressing Backup. DO NOT CLOSE" /low /W /B "%exe7Zip%" a -mx=9 -mfb=273 -ms=on -r "%BackupFileDestination%" "%dirTempBackup%\"
    ECHO. Done compressing backed up files.
    ECHO.
    
    :: added hash file
    :: Pass the file to hash as the only arg
    Set "MD5="
    For /f "skip=1 Delims=" %%# in (
      'certutil -hashfile "%BackupFileDestination%" MD5'
    ) Do If not defined MD5 Set MD5=%%#
    Set MD5=%MD5: =%
    Echo %MD5% > %BackupFileDestination-hash%
    
    ECHO Cleaning up.
    IF EXIST "%dirTempBackup%" RMDIR /s /q "%dirTempBackup%"
    ECHO.
    echo. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
    echo. File was Archived on Computer: >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
    echo.                                %computername% >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
    echo. File used for backup is: >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
    echo.                                %title% >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
    echo. xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx >> %logs%%nowdate%%timeStamp%_Userdrive_log.txt
    ::mode 30,15
    color 0e
    cls
    echo   ??????????????????????????
    echo   ?                        ?
    echo   ?       DONE...          ?
    echo   ?                        ?
    echo   ?   Thank You for        ?
    echo   ?    your patience.      ?
    echo   ??????????????????????????
    :Pause
    :End
    ECHO Finished.
    ECHO.
    ENDLOCAL
    
    


  • try to use those

    simple put ``` at the start and end to make block
    

    And that looks very complex, robocopy is meant to be easy I used it back in the day

    simply

    robocopy /mir /mt source target
    

    anytime you want to make full copy make 7zip archive of it, like snapshot.



  • @Emad-R said in Robocopy/7-zip bat file with high compression ratio:

    try to use those

    simple put ``` at the start and end to make block
    

    And that looks very complex, robocopy is meant to be easy I used it back in the day

    simply

    robocopy /mir /mt source target
    

    anytime you want to make full copy make 7zip archive of it, like snapshot.

    I use /mir /mt: most often. Works fine. I sometimes forget /mt: and immediately see slowness with large jobs.



  • @Emad-R
    Thanks for the tip. updated.


Log in to reply