Saturday, November 24, 2018

fdupes -- taming duplicate files in Linux

Posts for the usage of fdupes in linux:

Man Page for fdupes:

In Ubuntu:

In General (see text below from this page):

fdupes – A Command Line Tool to Find and Delete Duplicate Files in Linux
by Editor | Published: August 8, 2015 | Last Updated: April 5, 2017

It is a common requirement to find and replace duplicate files for most of the computer users. Finding and removing duplicate files is a tiresome job that demands time and patience. Finding duplicate files can be very easy if your machine is powered by GNU/Linux, thanks to ‘fdupes‘ utility.

Find and Delete Duplicate Files in Linux
Fdupes – Find and Delete Duplicate Files in Linux

What is fdupes?
Fdupes is a Linux utility written by Adrian Lopez in C programming Language released under MIT License. The application is able to find duplicate files in the given set of directories and sub-directories. Fdupes recognize duplicates by comparing MD5 signature of files followed by a byte-to-byte comparison. A lots of options can be passed with Fdupes to list, delete and replace the files with hardlinks to duplicates.

The comparison starts in the order:

size comparison > Partial MD5 Signature Comparison > Full MD5 Signature Comparison > Byte-to-Byte Comparison.

Install fdupes on a Linux

Installation of latest version of fdupes (fdupes version 1.51) as easy as running following command on Debian based systems such as Ubuntu and Linux Mint.

$ sudo apt-get install fdupes
On CentOS/RHEL and Fedora based systems, you need to turn on epel repository to install fdupes package.

# yum install fdupes
# dnf install fdupes    [On Fedora 22 onwards]
Note: The default package manager yum is replaced by dnf from Fedora 22 onwards…

How to use fdupes command?
1. For demonstration purpose, let’s a create few duplicate files under a directory (say tecmint) simply as:

$ mkdir /home/"$USER"/Desktop/tecmint && cd /home/"$USER"/Desktop/tecmint && for i in {1..15}; do echo "I Love Tecmint. Tecmint is a very nice community of Linux Users." > tecmint${i}.txt ; done
After running above command, let’s verify the duplicates files are created or not using ls command.

$ ls -l

total 60
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint10.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint11.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint12.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint13.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint14.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint15.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint1.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint2.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint3.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint4.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint5.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint6.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint7.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint8.txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint9.txt
The above script create 15 files namely tecmint1.txt, tecmint2.txt…tecmint15.txt and every files contains the same data i.e.,

"I Love Tecmint. Tecmint is a very nice community of Linux Users."
2. Now search for duplicate files within the folder tecmint.

$ fdupes /home/$USER/Desktop/tecmint

3. Search for duplicates recursively under every directory including it’s sub-directories using the -r option.

It search across all the files and folder recursively, depending upon the number of files and folders it will take some time to scan duplicates. In that mean time, you will be presented with the total progress in terminal, something like this.

$ fdupes -r /home

Progress [37780/54747] 69%
4. See the size of duplicates found within a folder using the -S option.

$ fdupes -S /home/$USER/Desktop/tecmint

65 bytes each:                         
5. You can see the size of duplicate files for every directory and subdirectories encountered within using the -S and -r options at the same time, as:

$ fdupes -Sr /home/avi/Desktop/

65 bytes each:                         

107 bytes each:
6. Other than searching in one folder or all the folders recursively, you may choose to choose in two folders or three folders as required. Not to mention you can use option -S and/or -r if required.

$ fdupes /home/avi/Desktop/ /home/avi/Templates/
7. To delete the duplicate files while preserving a copy you can use the option ‘-d’. Extra care should be taken while using this option else you might end up loosing necessary files/data and mind it the process is unrecoverable.

$ fdupes -d /home/$USER/Desktop/tecmint

[1] /home/tecmint/Desktop/tecmint/tecmint13.txt
[2] /home/tecmint/Desktop/tecmint/tecmint8.txt
[3] /home/tecmint/Desktop/tecmint/tecmint11.txt
[4] /home/tecmint/Desktop/tecmint/tecmint3.txt
[5] /home/tecmint/Desktop/tecmint/tecmint4.txt
[6] /home/tecmint/Desktop/tecmint/tecmint6.txt
[7] /home/tecmint/Desktop/tecmint/tecmint7.txt
[8] /home/tecmint/Desktop/tecmint/tecmint9.txt
[9] /home/tecmint/Desktop/tecmint/tecmint10.txt
[10] /home/tecmint/Desktop/tecmint/tecmint2.txt
[11] /home/tecmint/Desktop/tecmint/tecmint5.txt
[12] /home/tecmint/Desktop/tecmint/tecmint14.txt
[13] /home/tecmint/Desktop/tecmint/tecmint1.txt
[14] /home/tecmint/Desktop/tecmint/tecmint15.txt
[15] /home/tecmint/Desktop/tecmint/tecmint12.txt

Set 1 of 1, preserve files [1 - 15, all]:
You may notice that all the duplicates are listed and you are prompted to delete, either one by one or certain range or all in one go. You may select a range something like below to delete files files of specific range.

Set 1 of 1, preserve files [1 - 15, all]: 2-15

   [-] /home/tecmint/Desktop/tecmint/tecmint13.txt
   [+] /home/tecmint/Desktop/tecmint/tecmint8.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint11.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint3.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint4.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint6.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint7.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint9.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint10.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint2.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint5.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint14.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint1.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint15.txt
   [-] /home/tecmint/Desktop/tecmint/tecmint12.txt
8. From safety point of view, you may like to print the output of ‘fdupes’ to file and then check text file to decide what file to delete. This decrease chances of getting your file deleted accidentally. You may do:

$ fdupes -Sr /home > /home/fdupes.txt
Note: You may replace ‘/home’ with the your desired folder. Also use option ‘-r’ and ‘-S’ if you want to search recursively and Print Size, respectively.

9. You may omit the first file from each set of matches by using option ‘-f’.

First List files of the directory.

$ ls -l /home/$USER/Desktop/tecmint

total 20
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint9 (3rd copy).txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint9 (4th copy).txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint9 (another copy).txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint9 (copy).txt
-rw-r--r-- 1 tecmint tecmint 65 Aug  8 11:22 tecmint9.txt
and then omit the first file from each set of matches.

$ fdupes -f /home/$USER/Desktop/tecmint

/home/tecmint/Desktop/tecmint9 (copy).txt
/home/tecmint/Desktop/tecmint9 (3rd copy).txt
/home/tecmint/Desktop/tecmint9 (another copy).txt
/home/tecmint/Desktop/tecmint9 (4th copy).txt
10. Check installed version of fdupes.

$ fdupes --version

fdupes 1.51
11. If you need any help on fdupes you may use switch ‘-h’.

$ fdupes -h

Usage: fdupes [options] DIRECTORY...

 -r --recurse      for every directory given follow subdirectories
                  encountered within
 -R --recurse:    for each directory given after this option follow
                  subdirectories encountered within (note the ':' at
                  the end of the option, manpage for more details)
 -s --symlinks    follow symlinks
 -H --hardlinks    normally, when two or more files point to the same
                  disk area they are treated as non-duplicates; this
                  option will change this behavior
 -n --noempty      exclude zero-length files from consideration
 -A --nohidden    exclude hidden files from consideration
 -f --omitfirst    omit the first file in each set of matches
 -1 --sameline    list each set of matches on a single line
 -S --size        show size of duplicate files
 -m --summarize    summarize dupe information
 -q --quiet        hide progress indicator
 -d --delete      prompt user for files to preserve and delete all
                  others; important: under particular circumstances,
                  data may be lost when using this option together
                  with -s or --symlinks, or when specifying a
                  particular directory more than once; refer to the
                  fdupes documentation for additional information
 -N --noprompt    together with --delete, preserve the first file in
                  each set of duplicates and delete the rest without
                  prompting the user
 -v --version      display fdupes version
 -h --help        display this help message

see for complete article.

Saturday, November 17, 2018

This has worked great for initial updates of windows machines before connecting to local WSUS Server

WSUS Offline Update ...since security, time and bandwidth are money.

How to: Deploy WSUS to clients without AD/Domain/GP using the registry

Good information here on how to add non-domain computers to WSUS.


We have a workgroup environment here and I needed a solution to provide our internal WSUS server to the clients.
So I used the way to add registry keys to our machines.

Note: the values used here working for my environment. Yours may differ. You can change them to match your needs. Please read more about it at #5 of this how-to.

9 Steps total
Step 1: Create a *.reg file (wsus-client.reg) containing this:
Windows Registry Editor Version 5.00



Step 2: Edit the lines:
- "WUServer"="http://your-WSUS-server:port"; and
- "WUStatusServer"="http://your-WSUS-server:port";
to match the IP address (or FQDN) of your WSUS server. IMPORTANT: remove the ";" from the end of that lines!

"WUServer"=" "
"WUStatusServer"=" "

"WUServer"=" "
"WUStatusServer"=" "

"WUServer"=" http://intranet.local:8080 "
"WUStatusServer"=" http://intranet.local:8080 "

The first key is named WUServer. This registry key holds a string value which should be entered as the WSUS server’s URL.

By default, in Windows Server 2012, WSUS 4.0 uses port 8530. However, WSUS 3.0 uses port 80, by default
(How to Determine the Port Settings Used by WSUS -

The other key that you will have to change is a string value named WUStatusServer. The idea behind this key is that the PC must report its status to a WSUS server so that the WSUS server knows which updates have been applied to the PC. The WUStatusServer key normally holds the exact same value as the WUServer key.

Step 3: Store the *.reg file
where the computers have access to it.

Step 4: Apply the *.reg file by:
- double click on it from the client machine (admin rights required)
- put in into the login script so it will be applied at every login

Here an example for using it in a Novell Login Script (Drive Z: is here the drive needed for the login):

#regedit.exe /s Z:\WSUS\wsus-client.reg
@wuauclt.exe /resetauthorization /detectnow

Step 5: Options:
Here some links to get more informations about the options used in the *.reg file:

- (German - sorry)

Step 6: How to remove that settings from the client?
Maybe it`s needed to remove that settings for various reasons. To get the default values in the registry, just follow the next step.

Step 7: Create a *.reg file (remove-wsus-client.reg) containing this:
Windows Registry Editor Version 5.00



Step 8: Store the *.reg file
where the computers have access to it.

Step 9: Apply the *.reg file by:
- double click on it from the client machine (admin rights required)

That`s all. I don`t think it`s useful to add this in a login/logout script ;-) but it would work too.

I use this method for setting up a new machine even the machine will be used outside of our network after setup.
In this way the new client gets all needed updates in a shorter time frame than downloading it all via internet from Microsoft.
After finishing the setup, I remove the settings as described from #6 on.

Published: Jul 6th, 2011 · Last Updated: Aug 31st, 2018
WSUS Step by Step

Troubleshoot activation errors for Office for Mac 2011

Monday, September 17, 2018

Linux Mint 19 and Windows network shares

I was wondering why my Linux Mint 19 box was not letting me browse my windows network shares the same way it worked in Linux Mint 18.  Well, here are the answers, well explained.


Mint 19 and Samba File Sharing Changes
Post  by altair4 » Sat Jun 02, 2018 7:52 am

Mint 19 and Samba File Sharing Changes 

The last few releases of Ubuntu introduced a number of changes to Samba itself, its Linux kernel component, and even how Ubuntu configured it.

Mint 19 as a Samba Client

[A] Are you trying to get a list of all the samba / smb servers on your network from your file manager?

Samba changed the maximum client smb dialect that it can use from SMB1 to SMB3 so that it could connect to servers that have disabled SMB1 like Win10. This inadvertently disables netbios host discovery so if you go to Network > Windows Network you will see something like Folder is Empty in Ubuntu and just a blank screen in Cinnamon, Xubuntu, etc,..

You can restore it's discovery ability by editing /etc/samba/smb.conf and right under the workgroup = WORKGROUP line add this one:

client max protocol = NT1
And then reboot. Yes, reboot.

But: If you set the max back to NT1 you will be able to see a server that has disabled SMB1 ( Win10 ) but you will not be able to access it through the file manager.

** With the default setting of SMB3 you can however still access it but you have to do so explicitly by name in your file manager ( smb://windows-host-name ) or ip address ( smb:// ), or by using Connect to Server. 

If you have a problem with that please see this: Samba Browsing Problems Checklist.

** If the other machine runs Win10 you can also use this method: Win10, Linux, mDNS, and Samba File Sharing

** This change to smb3 does not impact any machines that have registered mDNS for their samba / smb shares. So if the other machine runs Ubuntu 17.10 or newer, another Mint 19 machine, or macOS you will automatically see those machines under Network - but not under Windows Network since it's not using a Windows discovery protocol.

** If the other Linux machine is using an earlier Mint or Ubuntu I would suggest adding to those machines an avahi/samba service announcement: Samba Browsing in an all Linux or Linux/OSX network

[ B] Are you using CIFS to mount your server?

The Linux kernel starting with 4.13 changed the default smb dialect that CIFS uses to SMB3 from SMB1. After 4.13.5 it's set to negotiate the version used with the server between 2.1 to 3.02 ( 3.11 in linux kernel 4.17 ). All modern SMB servers ( Win10, macOS, and Linux ) will accept SMB3 so this should not be an issue.

** What can be an issue is if you are running something with a very old version of samba like most NAS devices and there you would have to add an option to your cifs mount statement forcing it back to SMB1:

Mint 19 as a Server:

First: Remember Mint doesn't install the samba server package by default any longer so install it now:

sudo apt install samba
[C1] Are the clients to this server running Linux or macOS?

Stating with Ubuntu 17.10 Samba has been configured correctly for mDNS. As soon as you install Samba your machine will become visible to all the other Linux and macOS machines by default. If you created shares on Mint19 those shares will also be visible. There is no need to make any other changes

[C2] Are the clients to this machine running Windows?

A Windows machine will have the same success or failure at discovering your machine that it did in earlier Mint versions. If you have problems see: Samba Browsing Problems Checklist.

If it is running Win10 it can always use \\mint-host-name.local or \\mint-ip-address.

Wednesday, September 12, 2018

Running batch files in windows task scheduler

Good information for starting batch files in windows ... be sure to read the comments at each page for further clarification.

From here:

I faced the same problem, but I found another solution without having to modify my batch script.

The only thing that I missed out is at the 'Action' settings - "Start in (Optional)" option.

Go the task properties --> Action tab --> Edit --> Fill up as below:

Action: Start a program
Program/script: path to your batch script e.g. C:\Users\beruk\bodo.bat
Add arguments (optional):
Start in (optional): Put the full path to your batch script location e.g. C:\Users\beruk\(Do not put quotes around Start In)
Then Click OK

It works for me. Good Luck!

And from here :

In the Scheduled Task dialog, just before the name of the batch file it's going to run (it's labeled Program/script. You now have something like:

Change it to

Add the following to the **Add Arguments (optional) entry:

/k "C:\My Batch File Folder\MyScript.bat"
Tested on my system (Win7 64-bit), and it worked perfectly. I'm looking at the open command window it created as I type this text. :-)

Helped me!

Tuesday, September 04, 2018

Deleting too long file names in Windows 8 and larger

Good information here:

Assuming you are on windows:

What I always do is use subst to create a drive letter association with part of the path.

Then go to the new drive letter and navigate to the files that have long names. You should now be able to rename/delete/etc them. The reason this works is because the path itself is no longer containing >255 chars.

To create a subst folder
Start a command prompt (no admin privileges needed)
Use cd to navigate to the folder you want to go (you can use tab to autocomplete names
type subst j: . to create the driveletter association. (instead of the . you can also type the entire path)
Now in Explorer, you have a new drive letter in This PC. Go to it and do whatever you need to do to the .cache files.
Return to your cmd window and type subst /d j: to remove the drive or alternatively, restart your pc.
EDIT: A trick I learned which can shorten this quite a bit. From the Explorer window where you have trouble, click the address bar so you can type in it, then type cmd and press Enter to start a command prompt at that location, then proceed at step 3. :)

EDIT2: Okay, this trick can be expanded even further. If you are in explorer at the right path, just type in subst j: . in the address bar and press enter. A command window will pop up, execute the command and close again, and there you go, a new drive pops up in explorer.

Friday, June 01, 2018

Netgear A6100 USB Network Adaptor in Ubuntu / Mint


Bus 001 Device 006: ID 0846:9052 NetGear, Inc.

led me to.
and this led to..
Which shows your device 0846:9052 NetGear
has the rtl8812au chip
and finally the chilli555 fix can be found,,
Please first do..

sudo apt-get install --reinstall linux-headers-$(uname -r) build-essential
Then go here ..

From the site just above:

Please check here: On the right hand side, click 'Download ZIP.' Assuming downloads go to your Downloads folder, open the folder, right-click the zip file and select 'Extract Here.' Open a terminal and do:

cd Downloads/rtl8812AU_8821AU_linux-master
sudo make install
sudo modprobe 8812au
It 'makes' with a couple of warnings on my 13.10 system, but no errors. I can test no further since I haven't the device. This is a rather new device and we'll be very interested in your report.
Whenever a later kernel version, also known as linux image is installed by Update Manager, after you reboot, you will need to re-compile the driver:

cd Downloads/rtl8812AU_8821AU_linux-master
make clean
sudo make install
sudo modprobe 8812au
From Linux mint site:
The 8811au should be supported but the source code might be old, try

sudo apt install git dkms
git clone
cd rtl8812AU_8821AU_linux
sudo make -f Makefile.dkms install

Thursday, May 31, 2018

Windows update for non-domained computers using WSUS

(Some of this might not be Win10 friendly and needs to be sifted through.)

Wednesday, May 30, 2018

Interesting Linux apps outside of distro channels

Virtual Volumes View


Virtualbox ... newer than distribution channels for the most part


Angry IP Scanner

NoMachine remote desktop

Google Applications

Zoom Meetings


Installing Virtualbox guest additions in Debian guest


Follow these steps to install the VirtualBox Guest Additions on your Linux Debian virtual machine:
  1. Open terminal and become root (su); if you have sudo enabled, you may also do for example sudo -i; see man sudo for all options):
  2. Update APT cache:
    apt-get update
  3. Install the latest security updates:
    This step will upgrade all your packages, so be wise about it. Try the following steps first and they might be enough to work if not, then perform upgrade and retry.
    apt-get upgrade
  4. Install these required packages:
    Packages build-essential and module-assistant are both required to be able to compile the kernel modules and so when installing the VirtualBox Linux Guest Additions you must have them. So, this command will get the headers and packages (compilers and libraries) required to work. Notice, that after installing your VirtualBox Linux Guest Additions you will leave behind some packages as well as Linux headers, which you might delete afterwards or not. In my case they didn't hurt, but for the sake of system tidiness, you might want to pick up after playing.
    apt-get install build-essential module-assistant
  5. Configure your system for building kernel modules:
    m-a prepare
  6. Insert Guest Additions ISO:
    Expand menu Devices and click on Insert Guest Additions CD image...
  7. It should mount automatically, but if for any reason it doesn't:
    mount /media/cdrom
  8. Run the following script:
    sh /media/cdrom/
  9. Restart the system:

Installing Brother HL-L2300D in Mint Linux 18.3


--------------------------------------- ... _as&os=128

Go there and download the first 3 files. The LPR, Cups, and installer (they just have those letters and words as part of the name)

Put them in the download folder.

Right click the installer file and extract it here and set it to run as executable using right click.

Make sure to be online for this

Then right click on the download window and open a terminal

type sudo bash linux-brprinter-installer-2.2.0-1 HL-L2300D  << This is the key that makes this work >>

then type your password and select yes by typing "y" and pressing enter

If you are using the USB cable the type "n" and press enter when it asks you for the URI. Otherwise select Y and type in the URI number. I have no idea what a URI number is.

Then it will ask you to test print.

Select yes.

For some bent reason it will say "cannot access no such file or directory" along with jibberish. Which makes no sense to me. But it will print the linux mint test page.

Then you can print as normal.