This is an automated archive.
The original was posted on /r/datahoarder by /u/HTWingNut on 2024-01-22 01:00:01+00:00.
I did a thing! I wrote my own PowerShell hashing script called... POWERHASH! (cue the dramatic music - PowerShell + Hash = POWERHASH). It does more than just hash your files though.
Windows does a crap job at offering built in tools to validate your data, so you have to resort to third party tools. Unfortunately there are limited third party tools out there, and many older ones that are no longer maintained or supported or cost money. So I decided to write my own. I am not a developer or programmer, so pardon the mess, but as far as I can tell the script is fully functional and efficient. I'm open to feedback from testing, however.
The biggest thing I wanted from a hashing program is to be able to update hashes of changed or added files to a folder without having to rehash the entire folder. I did not find this feature in the handful of programs I evaluated. So I implemented that feature. This way you can hash some files, then run the UPDATE function to update hashes and remove hashes of files removed from the folder. But it can do more than that as well. See below for features.
The project was first started by using stock PowerShell 5.1 that comes with Windows, because I like to use stock apps whenever I can. But I quickly realized that PS 5.1 is antiquated and the latest PowerShell Core 7 (currently version 7.4.1) is so much better, offers a ton of cmdlets to prevent having to load or program your own functions, and is very efficient with file comparisons. It's free and easy to install too.
****** FEATURES ******
So what does PowerHash do?
- Generate SHA256 (or MD5) hashes of files in a folder recursively (makes use of
Get-FileHash cmdlet)
- Omit folders and files from checksum operation by using exclusion keywords for folders and files separately
- Update existing hash log file with only files that have changed and/or been deleted from a folder so a full hash of all files does not need to be completed
- Scrub a folder against an existing POWERHASH log file to check for discrepancies
- Compare two log files for discrepancies (so you can hash two locations then scrub them based on the log files)
- Find duplicate files based on matching hashes
- Run from interactive menu or use command line flags (so you can schedule updates and scrubs with Task Scheduler)
****** REQUIREMENTS ******
PowerShell 7 (Core) is required to run this script, but it's free and easy to install. Instructions here:
Or just type in cmd prompt or powershell prompt: winget install --id Microsoft.Powershell --source winget
This will install the latest PowerShell version (currently 7.4.1 as of this writing, which is what this program was validated with).
****** DOWNLOAD ******
While I recommend you read my diatribe on how to use the program, you can download it from github:
It's currently considered BETA. You can nab the powerhash.ps1 over on the right under "releases".
I have a video on how to use the program as well here:
****** INSTRUCTIONS ******
The program runs in both an interactive menu mode through cmd prompt or powershell prompt. It's best to CD to the folder where your powerhash.ps1 folder resides and run from there.
To run: pwsh .\powerhash.ps1
By default it will generate SHA256 hashes of files, but you can use the -MD5 flag to have it generate and manage MD5 hash checksums instead (i.e. pwsh .\powerhash.ps1 -md5)
This will take you to the interactive menu. I will first take you through the interactive menu, then show you how to run the same commands using command line mode.
It may be a good idea to have a Windows Explorer window open so you can click/drag folder and file names to the cmd windows to save keystrokes. Just remember to click the command windows after dragging file names/paths over.
You can get command line help at any time by typing: pwsh .\powerhash.ps1 -help or even a full readme with -readme flag. Or use -readme flag with specific function you want. Those available readme's are shown in the -help.
The script will generate a MAIN HASH LOG that will contain SHA256 hash values, file name, file size, last modified date of every file hashed. This is the MAIN HASH LOG that will be set to READ ONLY using the naming convention:
SHA256_[FOLDER NAME]_[DATE TIME STAMP].log
Example: User enters folder D:\DATA, the log file will be 'SHA256_DATA_20240117_190211.log'
Let's call this '[hashlog].log'
You can rename [hashlog].log whatever you want to after it's made, but the file will be set to READ ONLY. This file should not be hand edited or it could make it not work properly with the script.
All operations that touch the MAIN HASH LOG will be summarized in the '[hashlog]_history.log' file. This file you can make notes in if desired as its reference only.
There will be other supplementary log files generated depending on what function you use. The log files that can be generated are:
HISTORY: ‘[hashlog]_history.log’ Maintains summary of all actions performed on [hashlog].log
UPDATED: ‘[hashlog]_updated.log’ Details of file changes when using ‘UPDATE’ fn
COMPARE: ‘[hashlog]_compare.log’ Details of results when comparing two logs ‘COMPARE’ fn
SCRUB: ‘[hashlog]_scrub.log’ Details of results after running ‘SCRUB’ fn
DUPLICATES: ‘[hashlog]_duplicates.log’ Details of duplicate files list after running ‘DUPLICATES’ fn
EXCLUSIONS: ‘[hashlog]_excluded.log’ Lists files Excluded from exclusion keywords set by user
PREVIOUS: ‘[hashlog]_previous.log’ Copy of [hashlog].log as “undo” after running ‘UPDATE’ fn
****** MAIN MENU ******
The Main Menu will present you with multiple options:
=POWERHASH SHA256= by HTWingNut v2024.01.17
Type q from any menu to return here
Choose from the following:
[G]enerate New SHA256 Hash Log
[U]pdate Hash Log
[C]ompare Hash Logs
[S]crub Folder with Log
[D]uplicate File Check
[Q]uit
CHOICE:
****** GENERATE (OR CREATE) ******
BEFORE YOU CAN DO ANYTHING with any of the other functions you must create a MAIN HASH LOG ([G] in menu or -create flag in command line). You point the program to the folder that you want to generate hashes from.
You can then assign any folder and file exclusion parameters you want. These are in the fomr of keywords or keyphrases and must be entered in a specific format. Folder and File exclusions are independent of each other. Folder means path without the file. File means just the file name and extension.
The format to use - single quotes around each keyphrase with multiple keyphrases separted by a comma:
Enter FOLDER Exclusion: '\LinuxISO','Temp','\Recycle Bin'
No wildcards allowed and IS NOT case sensitive. This example would exclude any file path starting with LinuxISO, any folder path containing the word temp, and any file path starting with Recycle Bin
`Enter FILE Exclusion: '.pdf','Thumbs.db'
This would exclude any files with a .pdf extension (or if file contains .pdf in the filename itself) and anything named Thumbs.db.
Command Line
To create this log from command line use:
pwsh .\powerhash.ps1 -create -path "D:\Data"
if you want to add exclusions:
pwsh .\powerhash.ps1 -create -path "D:\Data" -excludefolders "'\LinuxISO','Temp','\Recycle Bin'" -excludefiles "'.pdf','Thumbs.db'"
Note the exclusion lists have to be wrapped completely in double quotes.
The results will be stored in '[hashlog].log'.
If exclusions were added it will generate a '[hashlog]_exclusion.log' file which will list the file names of all files excluded from hashing along with the rule that excluded it.
And of course everything will be summarized in the '[hashlog]_history.log' file.
****** UPDATE HASH LOG ******
This function will allow you to update an existing hash log with only files that have been changed, added/new to the folder, or deleted from the folder. It will also provide potentially renamed or moved files. You will have to provide the log file to update and the file path where to look for updated files.
Folder and File exclusions can also be added, modified, or removed at this time. It will remove any existing log entries that match the exclusion profiles provided by the user.
Command Line
The command line version of this would look something like this:
pwsh .\powerhash.ps1 -update -log "SHA256_DATA_20240117_190211.log' -path "D:\DATA"
And like the create/generate hash prompt you can exclude files as well.
pwsh .\powerhash.ps1 -update -log "SHA256_DATA_20240117_190211.log' -path "D:\Data" -excludefolders "'\LinuxISO','Temp','\Recycle Bin'" -excludefiles "'.pdf','Thumbs.db'"
If you want to clear exclusions from command line you can use the '-excludeclear' command:
pwsh .\powerhash.ps1 -update -log "SHA256_DATA_20240117_190211.log' -path "D:\Data" -excludeclear
The main log file '[hashlog].log' will be updated with any changes detected in the folder.
Details of the update will be stored in '[hashlog]_updated.log'.
And of course everything will be summarized in the '[hashlog]_history.log' file.
****** SCRUB FOLDER ******
The...
Content cut off. Read original on https://www.reddit.com/r/DataHoarder/comments/19cj4bb/powerhash_a_powershell_sha256md5_hashing_script/
