Photo De-Duplication
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
Robert 205081fb97 rm db.sqlite3 11 months ago
webapp rm db.sqlite3 11 months ago
.gitignore rm db.sqlite3 11 months ago
0.jpg dups 11 months ago
1.jpg dups 11 months ago
2.jpg init 2 11 months ago
3.jpg init 2 11 months ago
4.jpg init 2 11 months ago
5.jpg init 2 11 months ago
BadImageFormat.jpg Check for tooSmall <1KB and tooLarge >10MB files. 11 months ago
LICENSE init 2 11 months ago
README.md django web server 11 months ago
Screenshot2025-04-26.png Screen update 11 months ago
Screenshot2025-04-26Non-DupFound.png updated photos 11 months ago
TooSmall.jpg Check for tooSmall <1KB and tooLarge >10MB files. 11 months ago
a.jpg speed boost round 1 11 months ago
b.jpg speed boost round 1 11 months ago
coordinates.py GPS Within 10 Feet checking... 11 months ago
dedup.py django web server 11 months ago
delibs.py django web server 11 months ago
do_dups.inc Args -scores and -forreal 11 months ago
get_dups.bat . 11 months ago
get_dups.sh Args -scores and -forreal 11 months ago
japan-river.jpg geo-data 11 months ago
london-bridge.jpg geo-data 11 months ago

README.md

dedup

Photo De-Duplication

Install Windows

cd dedup
python -m venv C:\Users\$env:USERNAME\Common\env_dedup

Check to see if C:\Users$env:USERNAME\Common\env_dedup\Scripts OR bin folder exists in Windows...

& C:\Users\$env:USERNAME\Common\env_dedup\Scripts\Activate.ps1

If you get an error like: "running scripts is disabled on this system" It means your PowerShell execution policy is too restrictive. You can temporarily allow scripts by running:

Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass

After that repeat the Activate Script command:

If that does not work, then TRY:
& C:\Users\$env:USERNAME\Common\env_dedup\bin\Activate.ps1

Here are the packages that need to be installed by pip:

pip install xxhash opencv-python pillow

Usage Windows:

Again, verify if the folder exists Scripts or bin:

& C:\Users\$env:USERNAME\Common\env_dedup\Scripts\Activate.ps1
python dedup.py 0.jpg 1.jpg

Windows Automated Directory use, not TESTED YET!:

get_dups.bat %USERPROFILE%\Pictures

Install Linux

cd dedup
python3 -m venv myenv
source myenv/bin/activate
pip install xxhash opencv-python pillow

Useage Linux:

cd dedup
source myenv/bin/activate
python dedup.py 0.jpg 1.jpg

Get more details on scores:

This command will give more details: Matrix deviation score, Decomposed similarity, Combined similarity, and general score...

python dedup.py 0.jpg 1.jpg -scores

Linux Automated Directoy use:

-forreal will move duplicate images into dups directory.

./get_dups.sh .
  OR: 
./get_dups.sh $HOME/Pictures

Files made by get_dups Scripts:

error level: 0 = NOT a Dup, 1 = Duplicate, 2 = Close Match, 5 = Same GPS GEO-location, 8 = Invalid Image, 9 = File Too small/big. Possible files: dups.txt, alike.txt, sameGPS.txt, invalid.txt, size.txt.

Run website:

First activate the python environment, then:

pip install django
cd webapp
python manage.py runserver

Visit http://127.0.0.1:8000 Login as bob, password: guest Wait a minute...for program to run it will compair a.jpg and b.jpg. See: authapp/views.py

[Image of ScreenShot]