Windows File Junctions, Symbolic Links and Hard Links

Linux and Unix for a long time have had the marvellous ability for you to create hard links. For Window users unfamiliar with hard links, they not are unlike file/folder shortcuts. Except for one critical difference, they are sorted and registered by the operating system’s file partition just like real files and directories. This means that unlike a shortcut, any program or application (including Windows Explorer and the Command Prompt) that looks that the hard link will always treat it like it is the original file or folder the symbolic link is pointing to.

Now this is advantageous for a number of reasons, especially if you use multiple partitions on your computer. I for example keep all my work related files on a separate partition labelled with the W: drive. This allows me to optimise searches and backups, as well as keeping the partition more secure from problems arising from operating system errors.

Part of my work involves web development, and often that requires the files I am working on to be located on the web server’s root directory. This can be a problem if the web server application is installed on the C: drive. This is where a hard link comes in as with it I can simply create a link from the web server’s root directory and point it to the folder containing the HTML files on my work drive (W:). With the link though the web server is none the wiser and once I have finished developing, testing and troubleshooting. I can remove the hard link and the web server will then assume the directory with the work has been removed. But this has been done without any directory refactoring or moving.

The Windows NTFS file system has supported some form of symbolic linking since Windows 2000. But unfortunately each revision of Windows up until Windows Vista has used a different method of implementing the links themselves. So for this article I will just focus on the Windows Vista and the identical Windows 7 implementation.

So in Windows what is the difference between a short-cut and symbolic link (sym-link) and a hard link?

Well a short cut is basically a file that points to another file. It is an antiquated, pointing system from the Windows 95 days. Shortcuts not only use up space on your hard drive, they also linger around after the item they are pointing to has been deleted and break if the item is renamed.

A symbolic link is like a short cut, but instead of being saved as a file, they are registered to the file system. This means they do not use hard disk space; all programs recognise and can read where the link is pointing to. A symbolic link can point to any file, folder either locally on the computer or over a network using a SMB path.

A file hard link and the directory junction are a little different. It not only points to the item but duplicates it, but does so without taking up the extra hard disk space required by a copied file. Also if you have a hard link pointing to a file then delete that original file, the hard link will still retain a copy. A limitation of the file hard link though is that the link can only be made on the same file partition as the file.

Finally a junction is a hard link for directories. To me they are the most useful and unlike file hard links, you can create junctions on different partitions to where the original folder is located. Again a junction is stored on the file system, does not take up space and is treated by the operating system and programs as a local folder.

Windows Vista/7 uses the command line program called mklink to create these symbolic links. It has 3 arguments and then requires both a link name and target.

mklink /D /H /J [LINK NAME] [TARGET]

Now there are 4 modes for mklink and they all counteract each other meaning you can only use one argument or none at all.

No arguments creates file symbolic link which is like a shortcut that is registered on the file system instead of stored in a file.

/D creates a directory symbolic link which is like a shortcut that is registered on the file system instead of stored in a file.

/H creates what Windows refers to as a hard file link. A hard file link is used where you need multiple copies of a file, but wish to save space by keeping only one physical copy on disk.

/J creates a Windows Junction which is a directory hard link. If you need duplicate copies of a directory but don’t wish to use up the extra hard drive space you can create a junction. Though be careful as any changes you make to a file or folder in one of the junction will affect all the others.

mklink /J WindowsVista C:\Windows

Would make a Windows Junction titled WinVista linking to the directory C:\Windows

Below is a screen capture of the 4 different types of symbolic and hard links. The yellow is a hard file link that doesn’t use up any extra hard drive space. The blue is a symbolic link to a file. The green is a symbolic link to a file. The read is a hard link to a directory which is known as a Windows directory junction.

SyncToy the GUI Alternative to Robocopy

In my last post I spoke about the benefits of using a Microsoft tool known as Robocopy for large file transfers, backing up and synchronisation. Well for those who are not comfortable with using a command line interface program or those that simply prefer the convenience of using a GUI. Well Microsoft has released an alternative, though less feature rich, free PowerToy known as SyncToy.

SyncToy currently comes in two editions, 1.4 and a 2.0 that is currently in beta. This article will cover the 2.0 beta which is more feature complete and should be stable enough for most user environments. It is currently available for both Windows XP and Vista and can be downloaded from Microsoft Download Centre.

Some of SyncToy 2’s benefits include

• The ability to mirror two directories including the renaming and deletion of files and directories
• The ability to copy from left to right directory or vise versa
• Support for network UNC paths
• Support for external devices such as flash drives, cameras, or even music players
• Ability to preview changes before they are committed to
• The option of copying all deleted files into the Recycling Bin
• Filtering by file names, extensions and attributes

How to use SyncToy 2

When you run SyncToy 2 you are first greeted by a menu displaying your folder pairs.

So select Create New Folder Pair and this dialog should appear.

This enables you to select the two directories you wish to synchronise. Generally Left Folder should be your source and Right Folder should be your destination. You will then be asked what do you wish to do with these files. Synchronize, Echo or Contribute. Hovering your mouse over each radio text will give you a a brief description. But normally if you want to clone directories use Synchronize . Otherwise use Echo. If you don’t ever want files to be deleted during the echo synchronizing then you should use Contribute.

Name your folder pair and that will be it.

If you want you can customise the options for your pair including file filtering.

Else you can just preview it to make sure it works as expected.

Now if the preview was successful you can press Run and your synchronisation will now take place. That is all there is too it!

Robocopy Basics

When it comes to copying large amounts or large in size files and directories many people don’t realise there are better tools for the job then the built in File Explorer used by Windows. Robocopy is a free command line application created by Microsoft that is perfect for copying large or even huge amounts of data between hard drives or over networks.

Robocopy or ‘Robust File Copy’ was first introduced as part of the Windows Resource Kit but has now been included as default in copies of Windows Vista. So what makes robocopy a better choice then either Windows Explorer, copy or the old DOS xcopy command?

Pros:

• It can tolerate network outages, retry and attempt to resume the file transfer a set number of times making it perfect for wireless connections.
• Can copy all NTFS attributes including owner and auditing information.
• Has a mirror mode to synchronise the target directory with the source making them identical.
• Uses a detailed verbose mode that keeps you up to date with all the active transfers.
• Is robust and quick, enabling the transferring of many gigabytes worth of data.
• You can throttle it’s bandwidth usage so the file transfers do not hog the network connection.
• Can copy copy destination content of symbolic links.

Cons:

• It uses the command prompt and has a myriad of options that can be confusing for the first time user.
• You can only copy directories, not individual files (though you can filter out or in types and names of files you don’t want transferred).

Here are a few common attributes I use with Robocopy.

Mirroring a directory with a UNC (network connection) target
robocopy "c:\my source directory" "\\Server\my target directory" /MIR /XF *.bak *.tmp /XA:ST /W:5 /R:5

/MIR MIRrors a directory
/XF *.bak or *.tmp Excludes files, in this case anything that has the extensions .bak or .tmp
/XA:ST Exclude files with either System or Temporary attributes
/W:5 Wait 5 seconds between retries (default is 30 seconds)
/R:5 Attempt up to 5 retries on a failed copy (default is normally a million)

If you have shortcuts within your directories and you don’t don’t their destination content transferred you can also use the /XJ attribute which ‘excludes Junction points’

And if you wanted to copy all the file and directory security details you can use /SEC

Copying files within an age bracket
robocopy "c:\my photos" "d:\my recent photos" /NOCOPY /MAXAGE:30 /R:0
Now while the first example used an UNC target but this doesn’t have to be the case. Here we are copying to a different hard drive on the same computer and we’re filtering the files by an age group.

/NOCOPY Ignores the individual file attributes and copies only the files themselves.
/MAXAGE:30 Only copies files that are 30 days old or less.
/R:0 Zero number of retries means if an error is encountered robocopy will skip it and just continue onto the next file.

This is result after I copied over 11,600 images (nearly 28gigs worth) over a 100Mbps ethernet connection using robocopy