Metadata about author and title will help sift things further. File sizes probably aren't that reliable. File hashes are a good start, which is what DupeGuru is doing. There are libraries that can access all those formats. For images, the duplicate check tools like Duplicate File Finder likely implement modern image comparison algorithms but even they miss similar files (false negatives) and have high false positive rates.įor ebooks specifically you're better off looking at the metadata. If you're doing it with text look at NLP methods like tf-idf or technologies like BERT. You are asking for an algorithmic approach to something people would give different answers to when asked if two images are similar. Gcc -L/usr/local/lib shash.o simi.o simiw.o lookup3.o -o shashĭefine "similar". Gcc -O2 -std=c99 -I/usr/local/include -c -o lookup3.o lookup3.c Gcc -O2 -std=c99 -I/usr/local/include -c -o simiw.o simiw.c Gcc -O2 -std=c99 -I/usr/local/include -c -o simi.o simi.c Trouble building it under FreeBSD and Linux: gcc -O2 -std=c99 -I/usr/local/include -c -o shash.o shash.c These docs are not identical, but they're similar enough that I wouldn't Shash is "a sample implementation of Charikar's hash for identification rw-r-r- 1 vogelke mis 1897832 0 18:21:52 gnuplot-5.2.8.pdf dupeGuru is a cross-platform (Linux, OS X, Windows) GUI tool to find duplicate files in a system. Versions of the GNUPlot documentation: me% cd /src/graphics/gnuplot/doc Like a similarity hash to compare the output. I think your best bet would be to extract just the text and then run something I have tried searching and tried other apps, but I am unable to find anything that can solve my problem.
#DUPEGURU UBUNTU COMMAND LINE SOFTWARE#
Is there any software that can find similar files (that search the content of the file) but may have a slight difference, like an extra page or cover, which is close to being a duplicate, but not 100%? I have also ran the duplicate plug-in in Calibre and it is also not flagging the files as dupes. Looking at the files through Calibre reader shows the file looks exactly the same to my eyes.
I have 3 files with the same file name, format and size (Example: Alice In Wonderland.epub size 17.5MB)ĭupeGuru is not flagging these as dupes. I am running DupeGuru scan type for “Content”.įor example. However my issue is that I am running into very SIMILAR files (not exact dupes) which DupeGuru is not flagging. I have been using DupeGuru (been using it for years) and it finds exact duplicates, which is great. To learn more about some of the command's other options, check out this post on converting and manipulating image files on the Linux command line.I am in the process of cleaning up and organizing 150GB worth of ebooks in various formats (i.e. The convert command makes resizing image files extremely easy.
The resultant files might look like this: $ ls -l dog* # get filetype and base name from argumentįiletype=`echo $img | awk -F. Note how it extracts the file extension from the filename so that it can build the new filename. The first script shown below would create a "smile_2.jpg" file from a "smile.jpg" file using the 1200x800 resolution. If you intend to convert a number of images or will be resizing images often, it's a good idea to use a script. Asking for it to be saved as a 2000x1200 will result in one that is only 1440x1200. Generating a 1200x1000 image from a 2400x2000 is one thing. Note that if the numbers aren't numerically related to the current dimensions of the image, the resultant resolution might not be what you expect. The resolution should be expressed as the desired width (in pixels), followed by an "x" and then the desired height. The syntax is "convert -resize resolution currentfile newfile".
#DUPEGURU UBUNTU COMMAND LINE INSTALL#
Dupeguru is not in the Software Manager, which installs programs with just a click, so how do I install it please I would be grateful for line-by line detailed instructions as Linux is still a mystery to me. $ convert -resize 1200x1000 smile.jpg smile-2.jpg I already have FSlint installed but want to try any other duplicate file finders with a GUI also.
0 kommentar(er)
