Run DOS on the Raspberry Pi – Use rpix86 to turn your Pi into a 1980s super-computer.

Posted by Ben in Raspberry Pi, rpix86 on April 23, 2013

Please feel free to jump straight to the ‘DOS on the Pi‘ if you have no interest in a computer history lesson.

What is DOS?

DOS was a semi-open standard that included a common operating system for desktop computers in the 1980s and early 1990s. The hardware used Intel x86-compatible CPUs while the operating system for this architecture was DOS the Disk Operating System. Note that the DOS acronym was also used for a number of incompatible and unrelated computer systems such the Apple DOS and Atari DOS.

As DOS was a primitive, text only operating system that was easy to reverse-engineer. A number of companies made their own editions of DOS and for the most part they were all compatible with each other. The most common x86-edition of DOS was Microsoft‘s MS-DOS but IBM also had their own PC-DOS as did Digital Research with DR-DOS.

MS-DOS was Microsoft’s precursor to its Windows operating system. In the 1990s many editions of Windows were simply a layer of software designed to load up after MS-DOS or later contained MS-DOS built-in.

Off the shelf x86-compatible hardware combined with the DOS operating system became the prominent computer gaming system in North America from the late 1980s until Windows 95. Other regions such as Europe preferred cheaper, propriety systems such as the 8-bit Commodore 64, Amstrad CPC, Sinclair Spectrum. Or the equally as expensive but more user-friendly 16-bit Atari ST and Commodore Amiga lines. In these parts of the world x86-compatible computers only became popular after the early 1990s.

How did DOS come to be?

DOS started out as a clone of the early microcomputer operating system CP/M by Digital Research. CP/M was created in 1973 to run on the Intel Intellec-8 development system which happen to be the first microcomputer sold in the USA.

By the end of the 1970′s CP/M was the most popular microcomputer operating system for professionals and business. Once IBM decided to enter this burgeoning market they wanted to use CP/M on their future products but they couldn’t come to an amicable agreement with Digital Research.

So IBM asked Microsoft to develop a clone of CP/M for their first microcomputer that they dubbed the Personal Computer or the PC. But at this early stage of its life Microsoft had never developed an operating system but bluffed IBM to believe otherwise.

Microsoft discovered a small software company known as Seattle Computer Products who had a x86-compatible CP/M clone called 86-DOS. Microsoft signed a non-exclusive contract to market and distribute this operating system and on-sold it to IBM. Just weeks before IBM’s major PC public announcement Microsoft obtained near-exclusive rights for Seattle’s 86-DOS.

In August of 1981 IBM launched the IBM PC. A machine that bought mainstream attention to microcomputers and made them respectable in the eyes of the public. Every IBM PC sold was operating off a royalty-free copy of Seattle Computer Products 86-DOS operating system re-branded as PC-DOS.

IBM had always been a hardware company and believed there was little money in software. Apple to this day follows that same ethos. What IBM failed to realise was they had designed their machine with off the shelf parts that anyone could buy. The only thing other manufactures lacked to create their own PCs was the operating system. But due to IBM’s non-exclusive operating system agreement. Microsoft was free to re-brand their own version of IBM’s PC-DOS and to sell it to potential companies who wanted to make copycat IBM PCs. That small mistake by IBM launched Microsoft, who became one of the world’s largest companies.

Why DOS?

Many popular franchises, genres and software companies were first launched or popularised on DOS. Warcraft, Grand Theft Auto, Sid Meier’s Civilization, Monkey Island, X-Com, Need for Speed. The first person shooter, the graphic adventure, simulators, real-time strategy are some examples. So DOS would be well worth investigating for anyone who has an interest in retro-gaming from this era.

Why not DOS?

It must be said that the DOS ecosystem has to be the most complicated and arcane mainstream gaming system of all time. The same openness that made it popular was also what made it so horrible to use.

Backwards compatibility was a major benefit and selling point of the DOS ecosystem. But at the same time it held it back and made it much more complicated than necessary. By the late 1980s, its text based 1970s lineage was horribly primitive and outdated.

Hardware support was complex. Often users had to manually change and configure their operating system for each item of software they ran. What unfortunately started off as a simple and clean system for IBM in 1981 evolved into an ad hoc open platform that it was never designed for.

Worse was hardware driver support. For the most part new hardware had to be directly supported by software. This meant that a user’s existing game collection would often not support new hardware simply because it didn’t exist when the software was written.

Finally many game developers targeted the lowest common denominator hardware. This meant you could have had the latest and most expensive hardware yet it could have been ignored by new release software that was designed for a machine that was 5+ years old.

Fortunately much of this hardware complexity is now avoided when using software emulation such as rpix86 on the Raspberry Pi.

Snooper Troops supporting IBM’s Color Graphics Adapter in 1982.

1984′s Troll’s Tale on IBM’s PCJr.

IBM’s Enhanced Graphics Adapter in use in 1986′s King’s Quest III.

IBM’s Multicolor Graphics Array in 1987 allowed photo like graphics in Moebius: The Orb of Celestial Harmony.

King’s Quest V from 1990 was one of the first titles to fully embrace IBM’s Video Graphics Array with upto 256 colours on-screen.

By 1992 some games such as Links 386 Pro had embraced hi-resolution Super VGA graphics.

DOS today?

DOS is still used on embedded systems that need a simple and small, text only operating system. Today most of the DOS community centres around the open source FreeDOS that offers modern tools as well as legacy compatibility.

DOS on the Pi.

At the time of writing in April 2013 the only acceptable DOS emulator for the Raspberry Pi is rpix86 by Patrick Aalto found at http://rpix86.patrickaalto.com.

What PC hardware does rpix86 emulate?

CPU: 80486 processor, including the protected mode features (for running DOS4GW games) but without virtual memory support. The emulation runs at a speed of around 20MHz 80486 (which equals a 40MHz 80386) machine.
Memory: 640KB of low memory, 4MB of EMS memory and 16MB of XMS memory.
Super-VGA graphics, with a maximum resolution of 640×480 with 256 colors.
SoundBlaster 2.0 (including AdLib-compatible FM sounds) sound card.

This combined hardware could have belonged to a very expensive high-end PC in the very late 1980s. Or a more affordable and common mid-range PC in the early 1990s. In that later era the audio would have been the machine’s weakest point. The graphics and memory would have been very good while the CPU would have been average.

My article DOSBox Sound Emulation talks about the huge variety of audio options on the x86-compatible platform while DOSBox Graphic and Machine Emulation covers the variety of graphic modes.

What operating systems does rpix86 support?

Unfortunately only the obscure 4DOS is supported by rpix86. 4DOS is a rather full featured and complex edition of DOS created by a small company out of Germany. While it was great for daily users of DOS based PCs back in the 1990s. It is an overkill for our purposes of PC gaming emulation on the Pi.

4DOS downloads http://www.4dos.info/v4dos.htm

Raspberry Pi set up.

This guide assumes that a clean install of Raspbian is in use on the Raspberry Pi.

Raspbian can be obtained from http://www.raspberrypi.org/downloads
A guide for installing Raspbian to an SD card for use with the Raspberry Pi is at http://elinux.org/RPi_Easy_SD_Card_Setup

raspi-config.

Start your Raspberry Pi with a new install of Raspbian and at the Raspbian Raspi-config configuration tool do the following.

First select update to make sure you’re using the latest version of the configuration tool.
Select expand_rootfs so the free space on the SD card is fully utilised.
Select change_timeszone to set your Pi’s time zone.
Select boot_behaviour and set Should we boot straight to desktop? as No.
Select change_pass to change the default pi user password.
Select <Finish> to exit.

Now in terminal if you ever need to return to Raspi-config.

sudo raspi-config

Now lets update Raspbian and any out-of-date software.

sudo apt-get upgrade -y

DOS set up.

We are going to use a directory called /dos located in our pi home directory /home/pi/ to store our rpix86 related files. For the commands shown below ~/ in Linux is treated a short-cut for the pi user home directory.

mkdir ~/dos cd ~/dos

mkdir ~/dos

This /dos directory will also double up for a mock hard drive that rpix86 will use for DOS.

Download the latest version of rpix86 the DOS emulator.

wget http://rpix86.patrickaalto.com/rpix86.tar.gz

wget rpix86.tar.gz

Now uncompress and run rpix86. Please note that rpix86 will not work over a remote terminal connection such as SSH or Telnet.

tar -xf rpix86.tar.gz ./rpix86

rpix86 running under a remote terminal connection.

When prompted press Y to download the required 4DOS.COM command interpreter.

rpix86 unable to find 4dos.com.

Once complete your Raspberry Pi should be running a DOS prompt.

rpix86 startup.

Basics of DOS shell.

At its core a DOS shell is essentially a self-contained program that allows commands to be run from a text prompt. It lets you interact with your computer’s file system and some hardware. By interacting with the file system you can launch programs; move, copy or delete files and modify the directories.

The rest of the DOS operating system comes in the form of extra programs, applications and hardware drivers that usually reside in a subdirectory. These additions can vary between different releases and iterations of DOS.

DOS command prompt explained.

C:\>

C: Is the active drive. In DOS each floppy, CD, DVD or hard drive is assigned its own drive letter. The C: drive is usually reserved for the first hard drive while A: and B: are reserved for floppy drives.
\ Is the active directory. When a backlash is displayed the active directory is at the root of the drive.
> Is the beginning of the text prompt.
_ Is a blinking prompt that shows where the user’s typed input will display.

Commands.

Please run the following commands in the DOS prompt. Note that unlike Linux, DOS is not case sensitive.

List the current directory.

DIR

Display some basic 4DOS information.

4DOS

Clear the screen.

CLS

Make a new directory named TEST.

MKDIR TEST

Copy the file 4DOS.COM to the TEST directory.

COPY 4DOS.COM TEST

List the content of the TEST directory.

DIR TEST

Change the active directory to TEST.

CD TEST

Delete the file 4DOS.COM

DEL 4DOS.COM

Change the active directory to root.

CD\

Remove the directory TEST.

RMDIR TEST

You can use the [Tab] key to auto-complete the names of files and directories that exist in the active directory.

4d [Tab]

You can cycle through previous commands typed at prompt by using the up and down arrow keys.

Quit rpix86.

EXIT

DOS commands.

Dumb DOS names.

DOS uses the FAT file system. The legacy FAT16 file system was first introduced into MS-DOS 4.0 and used through the late 1980s and early 1990s. It is horribly dated but can still be read by modern operating systems.

Because of its age FAT and DOS has a number of particular but important to know traits.

In DOS file and directory names can be at most 8 characters long. An additional identifier extension can be given but that can only be up to 4 characters and the first character must always be a period.
File usage is identified by extensions, many of which are still in use today. FILE.TXT is a plain text document. FILE.MP3 is an MP3 audio track. FILE.AVI is a video file. FILE.HTM is a HTML document. DOS script files use the .BAT (batch processing) file extension. Program files use .EXE (execute) or the .COM (command) file extensions.
Files and directories are case insensitive.
The backlash \ character is used to separate directory and file paths.

Fetch and use DOS software.

For the purposes of this article I will only use games sourced from the excellent DOS Game Archive. It only offers DOS software that is legal to distribute and use without payment.

Under the Copyright laws of many countries, copyright on software remains active until 50 or 70 years after an author’s death. This is designed to protect any potential long-term income and the resale value of a product.

The majority of dated, commercial computer software has no financial value and is usually long out of print. This is more apparent with software created for obsolete systems such as DOS. Copyright of this type is rarely defended by its owners and is unofficially known as abandonware.

While the act the downloading and use of abandonware is classed as piracy and is officially illegal. I personally do not see any serious legal or moral issues with downloading and using out of print, obsolete commercial software. But you should use your own judgement.

Where to get games?

You can find legal to download DOS shareware, freeware and public domain games at the DOS Game Archive http://www.dosgamesarchive.com.

To find abandoned but technically illegal to download DOS software, type DOS abandonware into Google and go from there.

DOS Demo 1 – LucasArts, Day of the Tentacle Demo

The surreal and funny Day of the Tentacle was a huge success for LucasArts in 1993. This non-interactive demonstration is easy to run and shows the old DOS low resolution VGA graphics and SoundBlaster audio hardware at its best.

At the Raspbian Linux terminal prompt create the directory DOTTDEMO.

mkdir ~/dos/DOTTDEMO

Change the active directory to DOTTDEMO.

cd ~/dos/DOTTDEMO

Use wget to download the Day of the Tentacle demo from DOS Game Archive.

wget ftp://download.dosgamesarchive.com/tentacle.zip

Uncompress the demo file using unzip.

unzip tentacle.zip

Return back to the /dos directory and run rpix86.

cd .. ./rpix86

Now in the rpix86 DOS prompt list the content of the DOS root to make sure the DOTTDEMO directory exists.

DIR

Change the active directory to DOTTDEMO.

CD DOTTDEMO

List the content of DOTTDEMO but only display items that use the extension .EXE which are execute programs.

DIR *.EXE

Run the program DOTTDEMO.EXE.

DOTTDEMO.EXE

Congratulations, hopefully you have a working Day of the Tentacle demo.

To exit the demo press [Alt] X and follow the prompt.

wget tentacle.zip

DOS for DOTT

DOTTDEMO.EXE

DOS Demo 2 – Sierra Online, Space Quest 3.

From early 1980s to the 1990s Sierra Online were all about pushing the envelopes of audio and visuals on the home computer. Space Quest 3 in early 1989 was certainly doing that on the PC. Unfortunately but common for the era, this demo requires a rather complex set-up procedure.

At the Raspbian Linux terminal prompt create the directory SQ3DEMO.

mkdir ~/dos/SQ3DEMO

Change the active directory to SQ3DEMO.

cd ~/dos/SQ3DEMO

Use wget to download the Space Quest 3 demo from DOS Game Archive.

wget ftp://download.dosgamesarchive.com/spaceq3.zip

Uncompress the demo file using unzip.

unzip spaceq3.zip

Return back to the /dos directory and run rpix86.

cd .. ./rpix86
Now in the rpix86 DOS prompt list the content of the DOS root to make sure the SQ3DEMO directory exists.

DIR

Change the active directory to SQ3DEMO.

CD SQ3DEMO

List the content of SQ3DEMO but only display items that use the extension .EXE which are execute programs.

DIR *.EXE

Run the program INSTALL.EXE to tell the demo of Space Quest 3 what hardware our DOS machine supports.

INSTALL.EXE

In the Sierra Game Setup/Installation Program.

Press [Enter] at the Sierra’s newest products .. notification.

Select EGA/VGA with RGB monitor.

Select AdLib Music Synthesizer Card.

Select IBM or IBM-compatible keyboard.

Press [Enter] at the NUMLOCK is OFF confirmation.

Select NO for the MicroSoft compatible mouse.

Press [Esc] to skip the install the game on your hard disk.

Press [Enter] at the Installation is complete prompt.

Back at the DOS prompt list the content of SQ3DEMO but only display items that use the extension .BAT which are script files. Many older Sierra games used batch script files to launch their programs.

DIR *.BAT

Run the batch script SQ3DEMO.BAT to launch the Space Quest 3 demo.

SQ3DEMO.BAT

Welcome to Scenes From Space Quest III.

To exit press [Esc] to bring up the in-game menu and then select File > Quit.

SQ3DEMO.BAT

Demo 3 - Windmill Software Digger

Next up is the rather simple Digger created by Windmill Software in 1983. The basic graphics and primitive audio was amazing compared to many early games created for the original IBM PC machines of this era. This game looks to be inspired by the 1982 Universal arcade game Mr. Do! Currently there is some audio emulation issues with this game when used on rpix86.

At the Raspbian Linux terminal prompt create the directory DIGGER.

mkdir ~/dos/DIGGER

Change the active directory to DIGGER.

cd ~/dos/DIGGER

Use wget to download Digger from DOS Game Archive.

wget ftp://download.dosgamesarchive.com/digger.zip

Uncompress the game file using unzip.

unzip digger.zip

Return back to the /dos directory and run rpix86.

cd .. ./rpix86

Now in the rpix86 DOS prompt list the content of the DOS root to make sure the DIGGER directory exists.

DIR

Change the active directory to DIGGER.

CD DIGGER

List the content of the directory DIGGER.

DIR

Run the program DIGGER.EXE.

DIGGER.EXE

Welcome to computer video gaming 1983 style. Use the arrow keys to control your character or to exit Digger press [F10] a couple of times.

DIGGER.COM

Demo 4 – Non-working Games

At the moment rpix86 is still an immature product and your mileage for running various software will vary. Some DOS software can be temperamental such as Epic MegaGame’s Zone 66 from 1993. It demands that we do not load any DOS XMS memory drivers but currently we can not do this in rpix86.

At the Raspbian Linux terminal prompt create the directory ZONE66.

mkdir ~/dos/ZONE66

Change the active directory to ZONE66.

cd ~/dos/ZONE66

Use wget to download Zone66 from DOS Game Archive.

wget ftp://download.dosgamesarchive.com/zone66.zip

Uncompress the game file using unzip.

unzip zone66.zip

Return back to the /dos directory and run rpix86.

cd .. ./rpix86

Now in the rpix86 DOS prompt list the content of the DOS root to make sure the ZONE66 directory exists.

CD ZONE66

List the content of ZONE66 but only display items that use the extension .EXE which are execute programs.

DIR *.EXE

Run the program ZONE66.EXE.

ZONE66.EXE

ZONE66.EXE

Disappointingly we receive an abort error message This program can not run with an XMS driver!!! because our DOS operating system is not configured correctly. As you can see gaming on PCs in this era often required more technical knowledge than today.

Fi

Well that is it for this tutorial. I will continue to update this entry as new revisions of rpix86 are released for the Raspberry Pi. Feel free to ask any questions in the comments below.

Digital Research, dos, IBM, Microsoft, Microsoft Windows, Raspberry Pi, Seattle Computer Products, X86

Leave a Comment

Use USB hard disk & flash drives with your Raspberry Pi

Posted by Ben in Linux, Raspberry Pi on March 21, 2013

This was meant to be a brief article on how to mount an external USB drive, but it quickly spiralled out when I starting writing about all the nuances and potential issues one might run into. So I have a created a quick summary of the commands on how to mount a drive below. But I highly recommend reading the rest of the article as there are a number of potential pitfalls with the Pi and external USB drives that are addressed.

In-brief

To mount a USB drive:

sudo mkdir /mnt/usbdrive sudo mount /dev/sda1 /mnt/usbdrive ls /mnt/usbdrive

To list your file systems:

sudo fdisk -l sudo mount -l df -h

Before disconnecting a USB drive:

sudo umount /dev/sda1

Format a drive to EXT4

sudo mkfs.ext4 /dev/sda1 -L untitled

Add Apple OS X HFS+ read/write support

sudo apt-get install hfsutils hfsprogs hfsutils

Format a drive to HFS+

sudo mkfs.hfsplus /dev/sda1 -v untitled

Add Windows NTFS read/write support

sudo apt-get install ntfs-3g

Format a drive to NTFS

sudo mkfs.ntfs /dev/sda1 -f -v -I -L untitled

Add Windows/DOS FAT32 read/write support

sudo apt-get install dosfstools

Format a drive to FAT32

sudo mkfs.vfat /dev/sda1 -n untitled

In-depth

The Raspberry Pi is a great and flexible little device. But one of its main limitations is the storage options if you wish to use the device as a file or multimedia server. SD memory cards are cheap and common with low-end specifications but reach an affordability and storage cap when their sizes increase. A cheap USB powered external drive with many times more space can be had for a similar price to a top capacity SD memory card.

USB Power Problems

A major limitation for running a USB drive on a Raspberry Pi are the power requirements. The Universal Serial Bus specification states that to adhere to the standard, up to 0.5A (amps) can be drawn from a single port.

My old Samsung G2 Portable 640 hard drive requires 0.85A to work which is 0.35A above the USB2 specification. This is not an isolated example, many modern desktop PCs and laptops supply a greater amperage than the standard 0.5A to their USB ports to support devices such as portable USB hard drives. Unfortunately the Pi cannot power many external USD devices such as hard drives as its USB ports are restricted to the standard amperage.

To get around this problem you need a powered USB hub. You attach the hub device to the Pi’s USB port, plug the USB hard drive into one of the hub’s USB ports and insert the hub’s power supply into a walled power socket. My tiny Logitech ‘Premium 4-port’ USB hub can share 2.5A between 4 devices which is more than enough to power my Samsung drive.

I would also recommend against powering the Raspberry Pi off the same USB Hub as a USB drive. For me this caused interference where the drive would momentary lose power.

Mounting A Drive

When a drive is mounted, it connects to your Pi and Linux recognises it. The drive is given a directory where you are able to access and modify its content. These directories are known as mount points and can be given any name that works for you but they should be placed in /mnt.

I will call my mount point ‘usbdrive‘. First we need to create a mount point.

sudo mkdir /mnt/usbdrive

Linux has the /dev directory that is in use to store special files that allow access to the computer’s hardware. The /dev/sd* collection of files represent drives. Each drive connected to your Raspberry Pi is given a letter.

/dev/sda Would be your first connected drive.
/dev/sdb Would be your second drive.

To mount the drive to your mount point ‘usbdrive‘.

sudo mount /dev/sda1 /mnt/usbdrive

The numeric 1 at the end of /dev/sda is a requirement and tells Linux to mount the first partition.

Partitions are beyond the scope of this article, but you can learn more about that at the Ubuntu Community Docs.

Disconnect / Unmount A Drive

It is always advisable that you unmount a USB drive before unplugging it from its power source. This forces all queued data to be written to the drive before it loses power.

sudo umount /dev/sda1

You may need to use the -f force option if the drive will not dismount.

sudo umount -f /dev/sda1

If you use the shutdown -P -h 0 command to power down your Pi you do not need to use unmount.

Disk File Systems

A disk file system is the method an operating system stores and reads data on a drive. There is an endless list of disk file systems out there as each operating system seems to have their own native but incompatible system.

Linux as a number of native file formats but generally today the most common is the EXT (Extended File System) series which include ext2, ext3 and ext4.

Apple OS X uses HFS+ (Hierarchical File System Plus) otherwise known as Mac OS Extended.

Modern Microsoft Windows systems mostly use NTFS (New Technology File System).

Legacy Microsoft Windows systems and ancient Microsoft DOS systems generally use a variation of the FAT (File Allocation Table) which includes FAT, VFAT, FAT32 and exFAT.

Excessive CPU usage with the ntfs-3 driver that slows Samba (smdb) transfers

With EXT4 the file transfer using Samba has an additional 250%+ CPU resource available for use

Disk File Systems Compatibility

EXT has native support in Linux and the Raspberry Pi. It has no official support in Windows. There are free third party drivers available for Windows offering limited read/write EXT support such as the open source EXT2FSD or EXT2Read. Apple OS X users need to use the commercial Paragon ExtFS to enable full EXT support.

To enable Linux EXT4 support:
It is turned on by default on the Raspberry Pi.

HFS+ has restricted support in Linux. It can read HFS+ formatted drives but can only write to them if journaling is disabled. Windows has no native HFS+ support but there are paid solutions such as Paragon HFS+ for Windows.

To enable Linux HFS+ support:
sudo apt-get install hfsutils hfsprogs hfsutils

FAT is probably the most supported file system but it is also the most limited. Linux, Windows and Apple OS X all support FAT, VFAT and FAT32. ExFAT otherwise known as FAT64 is native to modern Windows and Apple OS X but has no support in Linux due to patient incompatibilities.

To enable Linux FAT32 support:
sudo apt-get install dosfstools

NTFS has read only support in Linux and Apple OS X. Third party drivers are available to add write support including the commercial Paragon NTFS and the open source NTFS-3g.

To enable Linux NTFS support:
sudo apt-get install ntfs-3g

Performance Issues & Which Disk File System To Use?

As a non-scientific test I took a 4GB video file and copied it to various file systems using my Raspberry Pi and the USB hard drive.

The worst performer by far was the NTFS-3g driver for Linux NTFS read and write support.

The transfer that 4GB file from my Windows 7 PC to the NTFS formatted USB hard drive took around a minute or two. The same file from the Raspberry Pi’s SD memory card to the NTFS formatted USB drive took 30 minutes to write and 23 minutes to read!

Performance for EXT3, EXT4 and FAT32 were about the same at 12-14 minutes to both read and write. This suggests that there is a bottleneck with either the SD memory card or USB drivers and not the file system.

If your drives are mostly used by the Pi my recommendation would be to use EXT4 on your USB drives. EXT4 is mostly the same as EXT3 with some extra minor features but it is widely supported in the Linux world, plus it is backwards compatible with EXT3 and EXT2.

FAT32 is the most compatible file system but has a restrictive 4GB file size limit.

EXT2, HFS+ on Linux and FAT32 lack journaling support that makes them prone to errors when used on portable drives. As these file systems can’t elegantly recover if they unexpectedly lose power.

FAT32, NTFS can not store Linux file or user permissions.

Format A Drive

To change the file system of a drive you need to format it. Linux allows you to format any supported disk format using the mkfs tool.

In the examples below you will notice an option followed by ‘untitled‘. These are optional volume labels to name your drive.

First you must unmount the drive you wish to format.

sudo umount /dev/sda1

To format a drive to EXT3 (Linux):
sudo mkfs.ext3 /dev/sda1 -L untitled

To format a drive to EXT4 (Linux):
sudo mkfs.ext4 /dev/sda1 -L untitled

To format a drive to HFS+ (Mac OS X):
sudo mkfs.hfsplus /dev/sda1 -v untitled

To format a drive to FAT32 (DOS and legacy Windows):
sudo mkfs.vfat /dev/sda1 -n untitled

To format a drive to NTFS (Windows):
sudo mkfs.ntfs /dev/sda1 -f -v -I -L untitled

I have applied a few options here that I will explain.
-f Fast Format. Due to the poor performance of 3g.ntfs on the Pi I highly recommend using the less CPU intensive fast format mode.
-v Verbose. By default the NTFS status output is limited so this lets you know what is happening.
-I Disable Windows Indexing. This improves the write performance of the drive but it will mean Windows Search queries used on this drive will take longer.

Format NTFS

Format HFS+

Format FAT32 (vfat)

Format EXT4

Automatically Mount A Drive

To simplify the process of mounting a drive you can add the drive’s information to the fstab settings file located in /etc/. I would recommend taking a look at the Ubuntu FSTAB community page for a deeper understanding of this file.

First run nano to edit fstab. The -Bw options tell nano to backup the file and not to use any line-wrap.

sudo nano -Bw /etc/fstab

You should already see some existing entries. Do NOT change these as the two /mnt/mmcblk0p entries are there to mount the SD card.

Add the following to the bottom of the file.

/dev/sda1 /mnt/usbdisk auto defaults,user 0 1

These are explained:

/dev/sda1 Is the location of the drive to mount.
/mnt/usbdisk Is the mount point, which is the folder to access the content of the drive.
auto Is the file system type, here you can set ‘auto‘ or force a file system type such as ext2, ext3, ext4, hfsplus, ntfs, vfat.
defaults,user Are mount options. You normally need to only supply ‘defaults‘. Though there are some others that maybe useful such as ‘ro‘ for read-only or ‘user‘ to enable write permission for all users. Use a non-spaced comma to separate multiple options.
0 A binary value used for debugging. It is best to keep this set at zero.
1 Pass number for a file system check at boot. ‘0‘ (zero) to disable or ‘2‘ to enable.

Save the changes to fstab.

[Ctrl] x
Y at the Save modified buffer prompt.
[Enter] for the File name to Write: /etc/fstab prompt.

The drive will mount at boot as long as it is attached to the Pi. If you want to mount the drive after you have plugged it in use mount with the automatic option.

sudo mount -a

Apple, Disk Management, EXT3, EXT4, FAT32, Hard disk drive, HFS+, NTFS, Raspberry Pi, Storage, USB, USB flash drive

Leave a Comment

Using a Point-to-Point Tunnelling Protocol, Virtual Private Network (PPTP VPN) client on a Raspberry Pi

Posted by Ben in Linux, Raspberry Pi on February 19, 2013

Why use a VPN?

VPN or a Virtual Private Network is a common way of securing an Internet connection using encryption.

Basically a VPN sets a trusted, designated server on the Internet to act as a man in the middle. As a VPN client, all your web traffic gets encrypted and directed to this designated machine. It decrypts and then reroutes the traffic to its intended destination. This gives you a both benefits and disadvantages.

Having an encrypted connection between yourself and a trusted VPN server means your Internet traffic is mostly safe from snoopers. That could be anyone on the same Wi-Fi network as you, your ISP or even big-brother governments.

If you are behind a government or corporate firewall, the VPN could be used to reach blacklisted websites. As the firewall will only see that your traffic is directed to the designated VPN server.

A VPN also allows you to hide the point of origin of your Internet request from your destination. As the designated server intercepts and decrypts your Internet traffic before it resents it. This fools the destination to believe the Internet request came from the designated server and not from you.

If a website or service has region restrictions placed on it to allow only users from particular countries access. A VPN server hosted in a permitted country allows you to bypass this form of restriction.

The most obvious disadvantage in using a VPN server is that adding an extra layer of Internet bureaucracy slows down your overall Internet experience. The encryption requires more processing time and it consumes more bandwidth. Plus adding a man the middle means your Internet traffic isn’t going to be traversing across the Internet via the quickest possible route.

About this project

For this project I am going to carry out a VPN client connection on a Raspberry Pi without using any GUI tools. This could be used with a headless or server Raspbian Pi set-up. It should also work fine on a Ubuntu based Linux system.

Also this tutorial will use the most common VPN protocol PPTP, known as the Point-to-Point Tunnelling Protocol. Its encryption is not as secure or as safe as some other VPN options but it is the most supported of the protocols out there.

Software installs and updates

First in a Pi shell we need to update our repositories, our Pi operating system and install PPTP for Linux.

sudo apt-get update -y sudo apt-get upgrade -y sudo apt-get install -y pptp-linux

Setup and configuration

The pptp-linux installation includes the very useful Perl script pptpsetup that is found at /usr/sbin/pptpsetup.

List the help options for pptpsetup.

pptpsetup --help

pptpsetup has 6 options to create a PPTP connection configuration.

pptpsetup
--create [tunnel name]
--server [vpn server]
--username [vpn server username]
--password [vpn server password]
--encrypt (optional)
--start (optional)

Example usage of pptpsetp.

sudo pptpsetup --create examplela --server pptp-la.example.com --username exampleuser --password examplepassword --start

The --create option is the ‘tunnel’ name you wish to give your connection. It can not have spaces or weird characters and should be something that is descriptive of the VPN server it will connect to such as a geographical location.

--server Is the address of the VPN server to connect to.

--username Is your assigned user name used to authorise access to the VPN server.

--password Is the password required by the user name.

--encrypt Toggles encryption for all communication between your Pi and the VPN server. While this setting is optional, most VPN servers will not accept a connection unless encryption is in use.

--start Will connect to the VPN after creating your new connection configuration.

For the rest of this tutorial I will use my real world example of pptpsetup. Replace any references to my proxpnuk configuration with your own VPN server configuration.

sudo pptpsetup --create proxpnuk --server pptp-uk1.proxpn.com --username xxxx --password xxxx --encrypt --start

If you have a Connection termination but you are sure your server address, user name and password are correct, try using the --encrypt option.

pptpsetup creates a configuration file that gets placed in /etc/ppp/peers/ which is a superuser protected directory so you will need to run sudo to view its content.

sudo ls -l /etc/ppp/peers/ sudo cat /etc/ppp/peers/proxpnuk

You can see your point-to-point protocol connection using ifconfig -s. It should be listed under ppp0.

pptpsetup can also be used to delete an existing configuration file.

sudo pptpsetup --delete [tunnel name]

Usage

PPTP for Linux also installed two Bash scripts that are used to connect and disconnect to your PPTP VPN server. You can use the -h option to see the scripts help text.

/usr/bin/pon and /usr/bin/poff are scripts that call the /usr/sbin/pppd application. pppd known as the Point-to-Point Protocol Daemon is a complicated program but if needed its manual with an endless list of options can be found at http://ppp.samba.org/pppd.html

pon -h poff -h

If connected let us disconnect our current ppp connection using the poff script.

sudo poff -a ifconfig -s

Ifconfig should only list eth0 and lo under the Iface column.

Let us reconnect again.

sudo pon proxpnuk updetach ifconfig -s

To see your connection settings for troubleshooting.

sudo pon proxpnuk debug dump

To save the debug output to a file and view it.

sudo pon proxpnuk debug dump > ~/proxpnuk-debug.txt cat ~/proxpnuk-debug.txt

PPTP for Linux has a log file that can also be useful for troubleshooting. It is found at /var/log/messages. The tail command normally displays the last 10 lines of text file but when it’s combined with the -f follow option it displays the most recent text appended to the file.

tail -f /var/log/messages

Press [Ctrl] C to exit tail.

Test for a VPN connection

I use the traceroute command to test for an active VPN connection. First make sure there is no ppp0 connection.

sudo poff -a ifconfig -s

Then run traceroute to a well-known website such as google.com.

traceroute google.com

Make a note of the number of hops. In my case it took 8 hops for a request from my Raspberry Pi to reach Google’s servers.

Now if we enable our ppp0 connection and try again, you will probably discover that connecting to Google takes exactly the same number of hops.

sudo pon proxpnuk updetach traceroute google.com

This probably means the internet traffic sent and received from the Pi is not using our VPN connection.

We need to create a route to tunnel our Internet traffic through our VPN. The route command at /bin/route is perfect to carry out this task.

sudo route add -net "0.0.0.0/0" dev "ppp0"

The -net 0.0.0.0/0 sets the target network of the route to be the default route. dev toggles a device to reroute our traffic to. While PPP connection number one ppp0 is that device.

To check if our route was successful use netstat with the -a all sockets option and feed the output into grep to filter and display only the lines that contain the string "/var/run/pptp/". These lines signify our PPTP routes.

netstat -a | grep "/var/run/pptp/"

Test your connection again and you should have a greater number of hops than last time.

traceroute google.com

That meant the connection to Google.com from your Pi had rerouted through your VPN server. Congratulations.

VPN at boot

To automatically connect your Pi to a VPN server at start-up I’d recommend placing the following script into /etc/rc.local. This file runs all commands within as a superuser, at the end of a Pi boot but before the user login prompt. I use the -B option with the nano text editor to backup the rc.local file before making any changes to it.

sudo nano -B /etc/rc.local

Make sure you insert any script before the exit 0 line and after the hashed # comments.

vpn="on" if [ $vpn = on ]; then printf "\nVPN connection to ProXPN UK\n" pon proxpnuk updetach printf "Add Internet traffic route through ppp0\n" sudo route add -net "0.0.0.0/0" dev "ppp0" printf "Netstat output of all PPTP sockets\n" netstat -a | grep "/var/run/pptp/" fi

The script is pretty self-explanatory and I have contained it within a conditional statement to disable if needed. To do this simply change the vpn="on" variable to something else such as vpn="off".

In nano press the [Ctrl] X keys to prompt for a Save modified buffer? query. Answer Y at the prompt to save and press enter to keep the current File Name to Write.

You don’t need to reboot to test your changes to rc.local.

sudo /etc/rc.local

Executes the script same way as it would with a reboot.

fi

I am happy to reply to any questions about this post but I am no expert on the subject of VPN and can not help in setting up a VPN server or using other client protocols such as OpenVPN. Extracting this information on PPTP for Linux was quite time-consuming.

Linux, poff, Point-to-Point Tunneling Protocol, pon, pppd, pptp, pptp-linux, pptpsetup, Raspberry Pi, raspbian, ubuntu, Virtual Private Network, vpn

10 Comments

Ubuntu Linux Shell Tips Part I

Posted by Ben in Linux, Server on December 13, 2012

I love the Linux shell. While cryptic and horribly daunting for a first time user the text only command line offers an amazing amount of flexibility and power without the need to actually develop or code. Yet unlike a graphical user interface of which I don’t tend to like in Linux. The text shell is fast and allows you to remotely connect to a headless device with the bare minimum of configuration.

Take the Raspberry Pi, it only needs a plugged-in Ethernet cable and a micro-USB power cable. And this credit-card size computer can be controlled with Linux shell from your favourite Windows PC, Mac, tablet or mobile phone.

This series of entries is going to be a run-down of most of the useful Linux shell commands and configuration options that I use daily on my Linux boxes and images.

Understanding the Prompt

ben@developers-tidbit:/$ _

ben Is the logged in user.
@ At symbol.
developers-tidbit Is the server name. On some systems the server’s IP address maybe listed.
: Divider.
/ Is the current path. ~ Means the current path is the home directory.
$ Dollar states that the user is a standard user. It is replaced by a hash # if the user is a root administrator.
_ Is the cursor to show the point of keyboard input, text placement.

Linux supports auto-completion at prompt, so cl [Tab] [Enter] will finish the rest of the command clear and launch it for you. Auto-complete also works with directories, files and apt-get repositories. Everything in Linux is case-sensitive so this means Clear, clear and CLEAR are all treated as different commands.

The prompt also keeps a history of the previously used commands. The up and down arrow keys cycle through these.

Directories

cd Changes the active directory.
cd / Goes to the root of the file system. Like the C:\ on a Windows system.
cd ~ Goes to your personal home directory.
cd /var Goes into the directory var.

ls Lists information about the files and is probably one of the more useful tools in the Linux toolbox.

ls Lists the active directory.
ls / List the root of the file system.
ls /var List the content of the directory /var.
ls -l / List the root of the file system using a long detail format.

ll Ubuntu has an alias (a short-cut) for the ls -l command.

ls -lh / Lists the root of the file system using a long detail format that humanises the file size.
Alternatively ll -h / will display the same results.

To only list the directories of the active directory a hack will be required. It lists all the items returned by the ls command that end with a forward slash / directory tag.

ls -d */ List the directories of root.
ls -d /var/*/ Lists the directories of /var.

Chaining Commands; Running Multiple Commands in Sequence

The semicolon character ; allows the running of multiple commands in a sequence. The following changes the active directory to home, lists the content using a detail list, then changes the active directory to root.

cd ~ ; ls -lh ; cd /

Viewing Text Files

Linux is highly dependant on plain text files for its configuration and settings so it would be natural that it includes many tools to handle these. The few common tools to display the content of a text file are cat, more, less.

cat Is short for concatenate and prints the file on-screen.
more Operates the same as cat except that it pauses the printing of the file at the end of each page.
less Operates the same as cat except you can use the up / down arrow keys or the Pg Up / Pg Down keys to scroll through the document.
To quit from both less and more press the Q key.

cat /etc/mime.type more /etc/mime.type less /etc/mime.type

To clean the screen use the clear command.

To display multiple files in a sequence more than one file can be supplied to cat.

cat file1 file2 file3

Another useful choice is the -n option that appends line numbers to the file in view.

cat -n /etc/mime.type

To remove the display of double-line spacing use the -s option.

The head tool allows the partial display of a file.

head -n [number of lines to display]

head -n 29 /etc/mime.type

The use of a negative number will display the complete file except the last X lines.

The tail tool is the reverse of head, it displays the footer of a file.

tail -n 3 /etc/mime.type

Tail has as a really useful option that allows you to follow the most recent changes that have been append to a text file. This is an excellent tool for reading only the most recent entries in a large log file. You may need to press Ctrl C to exit tail.

cat /var/log/syslog clear tail -f /var/log/syslog

Some configuration files such as the Linux user and group settings consist of comma-delimiter data. To extract data from these files you can use the cut tool. By default cut uses the tab character as a delimiter but this maybe changed with the -d "[delimiter]" option. The -f option with a list or a range of numbers provides the columns for display.

cut -d ":" -f "[columns]" [file]

Lists the first 3 columns in the group settings file.

cut -d ":" -f "1,2,3" /etc/group cut -d ":" -f "1-3" /etc/group cut -d ":" -f "1,2-3" /etc/group

To search within a file the grep tool is most useful.

grep "[search term]" [file]

A few grep options.
-i Will ignore letter cases in the [search term], so "Abc", "abc", "ABC" will be considered matches for grep "abc" [file]
-c Returns a count of the number of search term matches rather than a summary of the finds.
-n Includes line numbers in the summary of the finds.
-m [limit value] Only returns the first X number of results.

grep -i "system v" /var/log/boot.log grep -in "system v" /var/log/boot.log grep -inm1 "system v" /var/log/boot.log grep -ic "system v" /var/log/boot.log

In-built Help

Linux shell has a number of help tools while you are in terminal. The quickest way to discover what a command’s purpose is the use of the whatis tool.

whatis more

Some terminal commands include a basic usage help which appears when you run the command with no options.

more

Many commands have an extended help which is trigged with the --help option.

whatis --help

Finally most commands have usage manuals that are accessible at any time with the tool man [command]. Reading manuals through the terminal is pretty tedious so fortunately you can find these same texts online at http://manpages.ubuntu.com/manpages/. Use the website’s search box to find the tool’s manual.

man more

Or read it at http://manpages.ubuntu.com/manpages/precise/en/man1/more.1.html

Piping and Input/Output redirection

This may sound a little abstract but it is easier to grasp when explained. Linux shell gives you the very powerful ability to take the output of one command and feed it into another.

ls --help

With the ls --help there is more text then what the terminal can display. A pipe using the | character will feed the output text into a secondary tool such as less or more. To exit either tool press the Q key.

ls --help | more ls --help | less

It’s not just help text that you can pipe. Any text output could be fed into another tool if it is supported.

ls -lh / | more

Output text can also be saved to a file using the greater-than > character. In the example below the output of ls is saved to result.txt that will be created in the Linux temporary directory.

ls -lh / > /tmp/result.txt ls -l /tmp/result.txt

less /tmp/result.txt

The greater-than > character will always overwrite an existing file with the new text content. Double greater-than >> will append the text to the end of the file.

echo "Hello world." > /tmp/hi.txt cat /tmp/hi.txt echo "Test 1, 2, 3." >> /tmp/hi.txt cat /tmp/hi.txt

The less-than < character will redirect the file content into a command.

less < /tmp/result.txt

Easy Text editing

To edit text files from Linux shell I believe the easiest to use tool is nano.

nano /tmp/hi.txt

Like in less you can use the arrow keys or Pg Up / Pg Down to scroll through the text. The left and right arrow keys can reposition the cursor. Any text typed will be inserted at active position of the cursor.

To cut a line of text press F9.
To cut a paragraph of text use F9 multiple times.
To paste text press F10.
To go to row position press Ctrl _.
For a help that unfortunately is slightly cryptic press Ctrl G. In the help text the ^ symbol means the Ctrl key while the M- symbol means Alt key.

To exit nano press Ctrl X.

When you exit nano you will be asked to Save modified butter (ANSWERING “No” WILL DESTROY CHANGES) ? To save the changes to file type y otherwise type n and the file will remain unedited and any changes lost.

Piping output text into nano doesn’t work as expected.

ls -lh / | nano

Nano expects to read text sourced from a file. To force nano to read text from standard input (aka stdin which is the Linux term for piped or streamed text) a dash - is used for the input file name.

ls -ln / | nano -

To get some basic statistics of a text file the wc word count tool is useful. wc without any options displays the number of lines, words and the byte count for the file. The -w option counts the words, the -m counts the number of characters while the -l counts the number of lines.

wc /proc/cpuinfo wc -w /proc/cpuinfo wc -m /proc/cpuinfo wc -l /proc/cpuinfo

That is the end of Part I of my Linux shell tips. The 2nd part that will be online at a later date will cover administration, file, network system and user tools.

cat, cd, cut, grep, head, less, ll, ls, man, more, nano, prompt, shell, tail, ubuntu, wc, whatis

Leave a Comment

MAME Arcade Game Fun With A Raspberry Pi

Posted by Ben in MAME, Raspberry Pi on November 26, 2012

To avoid confusion please note that this entry has been updated in May of 2013. Some of the screen captures show older versions of Moebius and AdvMAME in use.

This tutorial is a step-by-step walk through on getting MAME running on a Raspberry Pi. This guide is from November 2012 and parts of it maybe out of date if you are reading it at a later time. As of writing, software, operating systems and drivers for the Pi is quite young and constantly evolving so some of the solutions used here I consider as hacks and are not best practise.

The goal of this tutorial is to run a cut down, bare bones operating system and to create an image thats sole purpose is to run MAME arcade ROMs. The tutorial is written with a Windows user in mind but would also be useful for new Raspberry Pi owners who are not familiar or comfortable with the Linux shell.

January 2013: About a month after publishing this article Shea Silverman released PiMAME an all-in-one MAME solution for the Raspberry Pi based on Raspbian. If you want a ready, out of the box solution I recommend it. If you prefer to hack and customise your own MAME setup or to learn about the Linux shell on your Pi then please read on.

Firstly some jargon used in this tutorial.

MAME Is a piece of software that emulates the hardware of legacy arcade video games. http://en.wikipedia.org/wiki/MAME
Image Is a copy of a computer’s file system that is kept in a single file for easy storage and duplication.
ROM Is like an image but it is a copy of the software used to power arcade video games.
SD card Is a memory card used as storage for portable devices such as cameras, phones and the Raspberry Pi.
ARM Is the type of CPU used by the Raspberry Pi, it is incompatible with software written for x86 processors such as those used in most desktops and laptops.
Shell Is a text only interface that allows you to run programs and interact with the computer.

The hardware I used.

SD to USB card reader.
Micro-USB to USB cable.
Flat Ethernet cable.
USB keyboard.
USB arcade joystick. I have an old Elecom JC-AS01BK Arcade Stick.
Raspberry Pi Model B with 256MB RAM.
SanDisk SDHC 4 MB/sec 8GB card. You will need at least a 1GB card SD card for this project.

Choosing a Minimal Raspberry Pi Operating System

As mentioned before I want a bare bones operating system. There is no need for a graphical user interface or unnecessary development bloat as our Pi image has a singular purpose.

On searching the internet I know of a few options. The official downloads http://www.raspberrypi.org offer Raspbian and Arch Linux. Raspbian has too much bloat for our purpose, while Arch Linux for the ARM processor lacks software support.

Fortunately there is the little known Moebius Linux, a minimal Raspbian based distribution targeted for the Raspberry Pi. The Moebius website is at http://moebiuslinux.sourceforge.net while the download page is http://sourceforge.net/projects/moebiuslinux/files/raspberry.stable.

If you’re familiar with your Raspberry Pi and know how to transfer images to SD cards. Download and transfer the Moebius ~~moebius.minimal.image.1.0.1~~ moebius-1.1.0 image to your SD card, use it to boot your Raspberry Pi and read on. If that makes little sense please follow my supplemental article.

First Boot And Hardware Configuration

Make sure your Pi has a keyboard attached, a connection to a LCD monitor or TV and an Ethernet link to the internet. Power the device using the micro-USB cable.

Once booted you should be taken to the Moebius Main Configuration program.

First thing you should always do is change the default password by selecting Password.

You will want to run the AutoResize tool as only 850 MB of your SD card is in use by Moebius, and you may run out of available space using MAME.

The MemorySetup should be changed to the lowest value of 16Mb for VideoCore. This will set 16 MB of system memory to your GPU (graphics processor unit) instead of the default 128 MB. MAME does not use 3D graphics or video hardware acceleration which are the main benefactors of GPU memory.

Also take note of the Overclock which will use hardware tweaks to speed up your Pi. The downside of overclocking is it could shorten the life the Pi’s chip components. This might be an issue if you have a first release Pi that could become a collectors’ item in the future.

Once done, select Finish. You maybe rebooted, if so re-login as root.

Remote Login

Using Secure Shell you can remote login to your Pi from your PC. You can operate the Pi without the need for a plugged in keyboard or display. In addition it allows you copy and paste text from your PC to the Pi. Remote login is already active in Moebius so you just need a Windows SSH client to connect to the Pi.

I would recommend Putty for a quick and easy solution. You can download it from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html. You only need Putty.exe from the Binaries download for Windows on Intel x86. It is a stand-alone portable application so you can copy and run it from to your Raspberry Pi directory on your PC.

We need the numeric ip or inet address of the Pi which is its unique identifier when connected to your home network.

hostname -I (note the capital i)

My Pi’s address is highlighted as the yellow text and is 192.168.1.8.

On your PC launch the Putty application. At the Startup configuration screen enter the Pi’s address as the Host Name (or IP address). Make sure the Port is a value of 22 and that the Connection Type is SSH. Under Saved Sessions give it a name such as Raspberry Pi. Press the Save button.

To connect to your Pi press the Open button. If prompted press Yes to automatically trust future connections to the Pi device and then login with your root account. You should be remotely connected to the Pi via your PC. If you want to copy clipboard text from the PC to the Pi just right-mouse click the Putty text window. To copy text from your Pi, left-mouse click and hold to select the text, it is then copied to your PC’s clipboard.

If you’re new to the Linux shell try out some of the commands below.

Some Useful Shell Tools

df -h --total Lists the Size, Used and Available disk space on your SD card.
free -th Lists the system memory (RAM) usage.
top Lists CPU processor usage and active memory usage. To quit press Q.
ping -c 3 http://www.google.com To check to see that the Pi has a working internet connection.
cd ~ Takes you to your user home directory.
cd / Takes you to the file system root.
ls Lists files and directories of the active directory.
ls -l Lists them in a detailed format.
ls -l -color Lists them using colour coding.
reboot Reboots your Pi.
shutdown -hP 0 Shuts down and turns off your Pi. Note the zero.
cat Displays the content for a file. Example cat /etc/passwd displays the user list of the Pi. If you need to exit cat press [Ctrl] C.
clear will clear the screen.

Don’t forget you can use auto-completion in Linux, so moe [Tab] [Enter] will finish the rest of the command moebius.config and launch it for you. Auto-complete also works with directories, files and apt-get repositories. Everything in Linux is case-sensitive so this means Moebius.config, moebius.config and moebius.Config are all treated as different applications.

Software Update

Assuming your Pi is connected to the Internet we should update the Moebius software repositories and then its applications. This could be thought of as the Linux equivalent of Windows Update.

apt-get update
apt-get upgrade -y

Shell Customisations

Moebius uses the Bourne compatible (/bin/sh) Bash shell (/bin/bash). I apply some customisations to help make its usage a little more useful with a Pi. Most of your system settings are in plain text files stored in /etc/. Whenever you make a change to a file within here I recommend making a copy of the original so you can restore it if needed. Enter the settings directory, backup and then edit the Bash system-wide file.

cd /etc/
nano -B bash.bashrc

Nano is my recommended text file editor, the -B argument automatically backups the source file. bash.bashrc is the file name of the file we are editing. In Nano use the [down-arrow] key or [PnDn] to scroll to the end of the file and add the following if you wish. At the bottom of the file add the following though it is probably best if you copy the code and paste using a Putty SSH connection.

# Alias definitions.
# You may want to put all your additions into a separate file like
# /etc/bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.
if [ -f /etc/bash_aliases ]; then . /etc/bash_aliases fi

Now create the file bash_aliases that will be used to hold our customisations.

nano /etc/bash_aliases

Copy and paste the following code into Nano.

#!/bin/sh


# Alias definitions.

# You may want to put all your additions in here.

# See /usr/share/doc/bash-doc/examples in the bash-doc package.
# Enable color support

if [ -x /usr/bin/dircolors ]; then

test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"

alias ls='ls --color=auto'

alias grep='grep --color=auto'

alias fgrep='fgrep --color=auto'

alias egrep='egrep --color=auto'

fi
# Extra ls aliases

alias ll='ls -FAl'

alias la='ls -FA'

alias l='ls -FAoh'

# Raspberry Pi Aliases alias raspi-audio="/usr/bin/alsamixer" alias raspi-cpu="/bin/cat /proc/cpuinfo" alias raspi-disk="/bin/df -h --total" alias raspi-firmware="/opt/vc/bin/vcgencmd version" alias raspi-mem="/usr/bin/free -th" alias raspi-ip="/bin/hostname -I" alias raspi-ping="/bin/ping -c 3 http://www.google.com" alias raspi-reboot="/sbin/reboot" alias raspi-shutdown="/sbin/shutdown -Ph 0" alias raspi-update="/usr/bin/apt-get update -y;/usr/bin/apt-get upgrade -y" alias raspi-version="/bin/cat /proc/version"

Once done press [Ctrl] X to exit, answer Y to the Save modified buffer question and press [Enter] to leave as is the File Name to Write [Backup] prompt.

If you try one of the new aliases it will return an error.

ll
-bash: ll: command not found error.

Run the source command to apply the changes and then retry the ll alias.

source /etc/bash.bashrc
ll

The ll alias should now work and be treated as just another command. Another quick tip, you can pipe the output of one command into another.

ll | less

less allows you to use the arrows or the [PgUp] [PgDn] keys to scroll through the output of ll. To exit press Q.

You can read more on customising profiles at http://www.hypexr.org/bash_tutorial.php.

MAME Time

As mentioned at the start of this article MAME is a complex piece of software that emulates antiquated video arcade machines to trick embedded game software into thinking they are running on original hardware.

As MAME is open-source and has been a working project since 1997 it comes in many different flavours for many operating systems. The official MAME project is for modern Microsoft Windows PCs with emulation accuracy a more prioritised goal than emulation speed. That means ports of the current versions of MAME are not suited for low and underpowered devices such as the Raspberry Pi.

AdvanceMAME is a MAME port that is for custom, embedded hardware and is perfect for our use. The most recent AdvanceMAME version is 1.2 which is compatible with ROMS packaged for MAME version 0.106 that came out in July 2006. This is important to note as the official MAME build as of May 2013 is version 0.148. So ROMS that work on your PC edition of MAME may not work on your Pi.

There is also an alternative Pi port of MAME based on the iMAME code-base which first came out for the HP Ipaq colour PDA back in 2001! The age of its code base and the ancient MAME 0.37 ROM support means it is best avoided.

Shea Silverman at http://blog.sheasilverman.com/tag/mame/ has compiled a Raspberry Pi Raspian port of AdvanceMAME. Download it to your root home directory. Install it using the Debian package management program.

cd ~
wget http://sheasilverman.com/rpi/raspbian/debs/advancemame_1.2-1_armhf.deb
dpkg -i advancemame_1.2-1_armhf.deb

Fetch the software dependencies needed by AdvanceMAME that Moebius is missing.

apt-get -y install libsdl1.2debian libfreetype6

Run AdvanceMAME for the first time.

advmame

A configuration file advmame.rc is placed in the /root/.advance/ directory. The file holds a huge list of default settings. Backup and then edit this file with Nano.

cd /root/.advance/
cp advmame.rc advmame.rc~
nano advmame.rc

Change the following settings.

device_mouse none
misc_freeplay yes
misc_quiet yes
record_sound no
record_video no

device_mouse turns off mouse support.
record_sound and record_video turns off both recording functionality which is not suited to the Pi.
misc_freeplay turns the free-play mode switch on if a game supports it.
misc_quiet turns off the annoying start-up copyright message.

If the Pi is connected to a HDMI TV or monitor add the following to your settings.
device_video_clock 5 - 50 / 15.62 / 50 ; 5 - 50 / 15.73 / 60

For Pi’s using a HDMI to VGA converter this setting worked well for me.
device_video_clock 5 - 50 / 15.75 / 50

If you’re using the yellow composite connection get your device_video_clock value from the AdvanceMAME install document http://advancemame.sourceforge.net/doc-install.html#4.5. If you are not sure what analogue TV system your country uses try here http://countrycode.org/tv-standards.

Finally you may also want to tweak the audio with the sound_samplerate setting. This sets the audio quality output and I imagine could improve the performance of a game if lowered. The default value is 44100 but poorer quality values of 22050 and 11025 are permitted.

A detailed list of all the settings used by advmame.rc is in the AdvanceMAME documentation http://advancemame.sourceforge.net/doc-advmame.html#8.

Save changes to advmame.rc and exit Nano.

MAME ROMs And Launch

MAME by itself is pretty useless. To get any fun or functionality you need ROM images of the game programs that are found in the arcade machines. A legal collection of free for non-commercial use ROMs is available at the MAMEDev site.

Your ROMs can go in either the /root/.advance/rom/ or the /usr/local/share/advance/rom/ directories. I would recommend the /usr/local/share/advance/rom/ directory for greater flexibily. I download and then test the copy of AdvanceMAME with the 1982 arcade game Victory by Exidy.

cd /usr/local/share/advance/rom/
wget http://mamedev.org/roms/victory/victory.zip -U Firefox
advmame victory

Make sure you do not run advmame victory in a SSH shell or a Putty connection as it will not work. At start-up you will experience a period of colour cycles, this is the legacy arcade software testing the hardware at power-on. At the Diagnostic Test Failure Screen press the 1 key. Congratulations you have a MAME game running on your Raspberry Pi!

To bring up the MAME menu press [Tab].
To insert coins press the 5 key.
To start as player 1 after inserting a coin press the 1 key.
To exit MAME and return to Linux press [Esc].

Transfer Files From Your PC To Your Pi (optional).

Now that you have a running MAME device you may want to transfer files to the Pi from your PC. There are a couple of options available but I use SSH’s secure file transfer or SFTP. Its set-up is painless, near automatic and gives full access to the Pi’s file system if it is wanted. On the Windows side of things I use FileZilla Client, a free and open-source file transfer application.

On the Pi install openssh-server and fetch its IP address.

apt-get install -y openssh-server
hostname -I

Your Pi should be ready to receive incoming SFTP connections.

Download, install, launch FileZilla Client on the PC and fill in its Quickconnect bar and press the Quickconnect button.

Host: Is the Pi’s IP address returned by hostname -I,
Username: root
Password: Is your root account password.
Port: 22

The left Local site: pane is the file system on your PC, the right Remote site: is your Pi. You can browse through directories on either. To transfer files to the Pi, simply select and right-click, Upload. If you want to upload ROMs it is best that you first navigate the Remote site directory to /root/.advance/rom/ or /usr/local/share/advance/rom/.

Start MAME At Boot (optional)

To have a true portable arcade device you need the Pi to login and launch MAME automatic without the need for a keyboard or user input.

To auto-login as root with no user login or password prompt, backup and edit the initialisation process settings.

cd /etc/
cp inittab inittab~
nano +54 inittab

Replace

1:2345:respawn:/sbin/getty --noclear 38400 tty1

with

1:2345:respawn:/sbin/getty -a root --noclear 38400 tty1
#1:2345:respawn:/sbin/getty --noclear 38400 tty1

Save changes to inittab and exit Nano.

To auto-launch AdvanceMAME we need to set-up the root account to use the file .profile to run a custom Bash launch script after login.

cp /usr/share/base-files/dot.profile /root/.profile
cd /root/
nano .bashrc

In .bashrc add the following to auto-start AdvanceMame and save your changes. The echo command displays text on-screen while the && characters creates a new line. Hashes # are used to ignore lines in a script which makes them useful for programming comments.

# /root/.bashrc: executed by bash(1) for root shells.

if [ -z "$SSH_CONNECTION" ]; then # Launch AdvanceMAME with a ROM echo && echo "Starting AdvanceMAME" && echo /usr/local/bin/advmame victory fi

Congratulations you now have a self contained Raspberry Pi arcade emulator, reboot your Pi and test the auto-login and auto-launch.

reboot

While booting if you ever need to abort the auto-launch of AdvanceMAME you can tap [Ctrl] C at the Welcome in your Moebius. Starting AdvMAME notification.

Install a Menu to List Games (optional)

AdvanceMenu is complementary program to AdvMAME that displays a complete list of all the MAME games stored on your Pi. You can additionally select and start any game from this list. To enable this menu feature first download and install AdvanceMenu for the Pi.

cd ~ wget http://sheasilverman.com/rpi/raspbian/debs/advancemenu_2.6-1_armhf.deb dpkg -i advancemenu_2.6-1_armhf.deb

Then run the program.

advmenu

You should see a white menu with a list of all your MAME ROMs. The big grey box is where any preview images get displayed.

If you ever receive an XML syntax error at launch, delete the advmame.xml file and try again.

rm ~/.advance/advmenu.xml advmenu

Use the keyboard arrow keys to navigate between games and press [Enter] to launch the selected title. [F1] displays the AdvanceMenu options while [ESC] exits you to the Linux shell.

Preview images (snapshots) are in ~/.advance/snap. You can either download a collection online http://www.progettosnaps.net/snaps_en.html and place them in there or create your own by pressing the [F12] key when in-game.

If you want to launch advmenu at boot first make sure you follow the previous Start MAME At Boot instructions. But instead of launching advmame [game] edit the root .bashrc file to launch advmenu.

nano +6 ~/.bashrc

Replace
/usr/local/bin/advmame [game]

with
#/usr/local/bin/advmame [game] /usr/local/bin/advmenu

Save the changes and reboot.

Troubleshooting MAME

Slow or bugged hardware

If you are suffering from slower than expected hardware performance issues you may wish to update the Raspberry Pi’s fireware. Follow the README.md instructions for rpi-update at https://github.com/Hexxeh/rpi-update so your Pi is refreshed to utilise the latest drivers for its hardware. This could improve performance and solve possible hardware bugs.

Another option to improve performance would be to run moebius.config and select Overclock to change the Pi’s clock preset to something greater than Warp1.

Audio problems

The older ALSA drivers on the Pi are known to have quality issues when software is not running at full speed. I originally had big problems using the 3.5mm headphone jack combined with a HDMI display output, yet audio via HDMI worked perfectly. I would recommend an update to your Pi’s firmware, follow the README.md instructions for the use of rpi-update at https://github.com/Hexxeh/rpi-update.

If you have no audio output run alsamixer (alias raspi-audio). Use the arrow-keys to adjust the PCM volume to make sure it is at least a value of 41.

Controllers

Potentially due to the power requirements you may find that your USB joypad or joystick will not work with your Pi. My Xbox 360 USB controller failed to work even when attached to the Pi via a powered USB hub. Using a 10 volt over-charge USB source to power my Pi didn’t help either. My imported Elecom joystick mentioned at the start of this article worked fine.

Using a non-root account

Though it is against best practice to use root for a general Linux activity I could not get AdvanceMAME with the USB drivers to correctly work without it. If you have better luck please let me know. Strangely one edit you need to make to launch AdvanceMAME as a non-root user is to as root give the floppy disk driver other group read write access.

chmod o+rw /dev/fd0

ERROR: required files are missing

There are several reasons why you may receive this error, see the MAME development FAQ.
http://mamedev.org/devwiki/index.php/FAQ:ROMs#Why_does_MAME_report_.22missing_files.22_even_if_I_have_the_ROMs.3F

AdvanceMAME, AdvanceMenu, ARM, auto-launch, auto-login, auto-start, iMAME, Moebius, ROM, sftp, ssh

21 Comments

MAME Arcade Game Fun With A Raspberry Pi Beginner Supplimental

Posted by Ben in Raspberry Pi, Windows on November 24, 2012

To avoid confusion please note that this entry has been updated in May of 2013. Some of the screen captures show older versions of Moebius in use.

This mini-entry is a supplemental article for first time Raspberry Pi owners using Windows PCs who are reading my MAME Arcade Game Fun With A Raspberry Pi article. This article not intended to be read stand-alone.

Tools for Windows

On the PC software side of things I use a couple of programs.

7-Zip A free and open-source file archiver. http://www.7-zip.org/download.html
Image Writer for Windows A free and open-source USB stick and SD card image tool. https://launchpad.net/win32-image-writer/+download (only get the binaries download, not the source)

If installing 7-Zip for the first time you probably will want to change some of the options. For 64-bit Window users to successfully save setting changes they will need to execute Run as administrator, do this with a right-mouse click on the 7-Zip File Manager program.

7-Zip run as administrator

Launch 7-Zip and select Tools, Options. In the System dialogue make sure tar, gz and gzip are Associate 7-Zip with:. Unless you have a reason not to I’d recommend pressing Select All and using 7-Zip as your default file archiver.

7-zip system options

You may want to integrate 7-Zip into file explorer, that will allow you to use a right-click to interact with archives using a 7-Zip context menu. Select the 7-Zip tab and select both Integrate 7-Zip to shell context menu and Cascaded context menu, Press OK and close 7-Zip.

Setting Up Your PC

I keep a directory named Raspberry Pi that houses all my SD card images. This is so I can have any image readily available, to keep backups in-case I break something and need do a Raspberry Pi equivalent of a Windows System Restore. To speed things up I do not use a file archiver to compress the SD card images, rather I leave them in their raw .img state. Instead I use a little known feature and allow Windows to compress the directory.

To do this right-click the directory in question, select Properties, General tab, press Advanced and then check the Compress contents to save disk space under the Compress or Encrypt attributes. This shrank the 900 MB Moebius image to use just 322 MB on disk.

Windows folder properties advanced attributes

Download the Moebius minimal image which was at the time of writing is ~~moebius.minimal.image.1.0.1.tar.gz~~ moebius-1.1.0.tar.gz. Move it do your Raspberry Pi image directory and right-click on the file to use the 7-Zip context menu to Extract here. Once extracted you will have a larger ~~moebius.minimal.image.1.0.1.tar~~ moebius-1.1.0.tar archive. Right-click the file to again apply the 7-Zip Extract here context menu.

Mobeus extracted

The extracted image is missing a .img file extension and Windows does not recognise it. Right-click the file and Rename. Add .img to the end of the file name, press [Enter]. Press Yes at any Windows prompt.

Add .img extension to moebius

Delete the ~~moebius.minimal.image.1.0.1.tar.gz~~ moebius-1.1.0.tar.gz and ~~moebius.minimal.image.1.0.1.tar~~ moebius-1.1.0.tar archives if you wish.

Image Transfers

Copy the Image Writer for Windows file win32diskimager-binary.zip to your Raspberry Pi image directory and use the 7-Zip context menu Extract to win32diskimager-binary. Plug the SD card reader into your PC and run Win32DiskImager.exe which is in the win32diskimager-binary directory.

win32diskimager application

The SD card is given a drive letter by Windows and is under Devices. If you have more than 1 device listed, use My Computer in the Windows Start to find which letter to use.

Discover SD card

To backup the existing data on the SD card, use the Image File selector to create a new image name and then press Read. Otherwise to transfer Moebius to the SD card use the Image File selector to choose the ~~mobieus.minimal.image.1.0.1.img~~ moebius-1.1.0.img and press Write. This will overwrite the entire content of your SD card.

Write Moebius to SD card

Once the writing is complete, bring up My Computer from the Windows Start, right-click your SD card drive and Eject it. It is safe to remove the SD card from your PC and insert it into your Raspberry Pi.

Eject SD card

Continue to First Boot in MAME Arcade Game Fun With A Raspberry Pi.

Leave a Comment

Linux, Mac OS, Unix: File Permissions and User Classes

Posted by Ben in Linux, Server on July 1, 2012

Using the Linux terminal (command line) I am going to explain in thorough detail the mysteries that are the Linux system file permissions for the uninitiated. I will be using Ubuntu Linux 12.04 LTS but most of what is covered should be applicable for any POSIX compatible Linux, Mac OS or Unix environment.

If this walkthrough tutorial does not meet your needs I have provided some links to other articles on this subject at the end of the piece.

For this article I will interchange the term ‘file or directory’ for the word ‘item’.

List -long

If you have ready access to a Linux terminal enter the following commands.

cd /boot ls

Output of the ls command performed on the directory /boot

A list similar to the image above should be outputted. The ls command lists the content of the current directory. You can discover the list command’s many options and parameters in the Ubuntu manual pages. http://manpages.ubuntu.com/manpages/precise/man1/ls.1.html.

ls -l

Output of the ls -l command performed on the directory /boot

By using the -l (lowercase L) option with the list command we display the content in a long and more detail format.

ll

Hand coloured output of the ll command performed on the directory /boot

The ll (lowercase LL) command is available on many Linux distributions and is a shortcut that displays identical results to ls -l. For the output above I have colour coded the different columns of item information so I can describe their purpose.

Red: lists the item type
Green: lists the item’s permissions.
Purple: lists the total number of hard links pointing to the item.
Pink: lists the item’s user (which is the owner) and its applied group.
Blue: lists the size of the item in bytes.
Orange/Brown: lists the date and time the item was last modified.
White: list the item name though some items in this column may have alternative colourings.

Item Type

The item type can be broken down into 6 values.

- (dash) is a standard file.
d is a standard directory.
l (lowercase L) is a symbolic link otherwise known as a symlink.
p is a named pipe otherwise known as a FIFO (file-in, file-out).
b is a block special file.
c is a character special file.

Standard Files and Standard Directories

Standard files and standard directories are the bread and butter of your Linux system. Files are documents of data. While a directory is a container which can hold multiple files as well as additional directories which are called subdirectories.

Symbolic Link

A symbolic or a soft link is a pointer to another file or directory. This enables you to have an item (called a target) in one location and one or more links pointing to to the target located elsewhere on the file system. These pointers can open, read and write to the target. Here is a little demo.

mkdir ~/blogdemo cd ~/blogdemo/ echo "Hello world" >> hi cat hi ln -s hi readme cat readme

Create a text file and then a symlink to it

Okay so what I just did was to create a new text file called ‘hi’ and then display it. I then created a symbolic link named ‘readme’ that pointed to the new text file. And finally I displayed onscreen the content of the ‘readme’ link.

The mkdir command creates a new directory.
echo “text” prints the text contained within the quotes.
>> pipes (moves) the printed text into the file named ‘hi’.
cat displays the content of the the file ‘hi’ onscreen.
ln -s <target> <link name> creates a symbolic link to the target.

You can see with the output of the ll (lowercase LL) command the yellow highlighted line is a symbolic link. The item type is listed with a ‘l’ (lowercase L) while the item name ‘readme’ has an arrow ‘->’ pointing to its target, which is the text file named ‘hi’.

Named Pipes

While rare in usage, named pipes or FIFO are objects that allow you to feed the data output of a source program to the data input of destination program. Below will create, list and delete a pipe but its proper use is beyond this article.

The IBM developerWorks Learn Linux, 101: Steams, Pipes, and Redirects article covers pipes in proper detail. http://www.ibm.com/developerworks/linux/library/l-lpic1-v3-103-4/

mkdir ~/blogdemo cd ~/blogdemo/ mkfifo my-pipe ll rm my-pipe

Creating a named pipe

Block and Character Special Files

In most situations you will not need to worry about either one of these file types. Block special files and character special files are unique in that they can receive and generate data. They are most commonly used by files that permit terminal and file system interactions with a piece of physical hardware attached to the computer. Block special files can handle multiple characters at once while character special files only handle single characters. Below is an example of some block special files that can be used to interact with the computer’s SATA hard disks.

ll /dev/sda*

List of SATA hard disks

Permissions

From here I will cover file-directory permissions and classes. The commands below will setup a couple of demonstration files in your home (~) directory.

cd ~/blogdemo/ echo "I am a filler file" > file1 echo "I am another filler file" > file2 chown ben file1 sudo chown :root file2 ll

Creating 2 text files for testing

Owner and Groups

The first thing you should look at in the previous screenshot is the orange/brown column which contains the owners of the items. The purple column that sits next to it lists the groups assigned to the items.

A group is a collection of different user accounts associated together under a common name. A user account can either have no group assigned, a single primary group assigned or a primary group with multiple secondary groups assigned.

A list of your Linux users are stored in the /etc/passwd file.
nano /etc/passwd

The groups are stored in the /etc/group file.
nano /etc/group

An extraction of the /etc/passwd file

An extraction of the /etc/group file

The Ubuntu manual pages has more on the content and format of these passwd and group files.
http://manpages.ubuntu.com/manpages/precise/man5/passwd.5.html
http://manpages.ubuntu.com/manpages/precise/man5/group.5.html

Classes and Permissions

Classes and permissions colour coded

The permissions comprise of 9 cryptic columns of data which are individually called bits and they can be separated into 3 classes.

The first set of 3 columns (coloured in red) belong to the ‘user’ class which is the owner of the item. Remember the owner is listed in the orange/brown coloured column.

The second set of 3 columns (coloured in blue) belong to the ‘group’ assigned class, that group is listed in the purple coloured column.

While the third set of 3 columns (coloured in green) belong to the ‘other’ class. These are users who do not fall into the previous two classes.

Each of these classes with their 3 individual columns of bits adhere to an identical format that is set to one of these six combinations.

wrx, wr-, w–, -rx,-r-,–x

w states the write permission is granted
r states the read permission is granted
x states the execute permission is granted
- states the permission is denied and acts as the negative bit character for w, r and x.

So for example wrx means write, read and execute has been granted. While -r- means only read has been granted and write and execute have been denied.

The function of these permission bits is different depending on whether the item is a directory or a file.

File Permission Bits.

w is the write bit, it allows the file to be modified, renamed and deleted.
r is the read bit, it allows the content of a file to be viewed.
x is the execute bit, it registers the file as a program and allows it to be run.

Directory Permission Bits.

w is the write bit, it allows the directory to be renamed or deleted plus it allows new files and subdirectories to be created within.
r is the read bit, it allows the content of the directory to be listed using commands such as ls or ll.
x is the execute bit, it allows access into the directory using commands such as cd. If the execute bit has been denied (-) access to all subdirectories and files within this directory will be barred. This denied bit overrides both the directory’s write bit and read bit permissions.

Let’s take a closer look at the file ‘file1’.

file1 in detail

The orange/brown text tells us that this file is owned by the user ‘root’.
The purple text shows that the file is assigned to the group named ‘ben’.

Now looking at the permissions.

The red set of bits are the permissions for the user which is another name for the owner. This owner ‘root’ can read and write to (rename, delete or modify) this file. These permissions nullify any of the lesser class permissions that might also match the owner.

The blue set of bits are for the group ‘ben’ which is assigned to this file. Users assigned to ‘ben’ can read and write to (rename, delete or modify) this file.

The green set of bits are for others who are users that do not fit the previous two classes. They can only read this file.

Let’s take a closer look at the directory blogdemo/.

blogdemo/ in detail

Firstly the orange/brown text tells us this directory is owned by the user ‘ben’.

The purple text states that the directory has been assigned to the group ‘ben’. When a new user account is created a default group under the same name is also generated. This group is then assigned to the user and is used as their primary group.

The red set of bits is for the user which is another name for the owner. The owner ‘ben’ has write (rename, delete, add new files or subdirectories with), read (list content) and execute (access into the directory) permissions.

The blue set of bits is for the group assigned to this directory which is named ‘ben’. Users who are assigned to the group ‘ben’ have write (rename, delete, add new files or subdirectories with), read (list content) and execute (access into the directory) permissions.

Please note that ‘ben’ is the owner of this directory, he is also assigned to the group ‘ben’. As there is a conflict the (lesser) group permissions are nullified for him as the user (owner) permission bits take precedence.

The green set of bits are for other and they are the users that do not fit in the previous two classes. They have permission to read (list content) and execute (access into the directory).

Change owner (chown) or change modification (chmod)

When you create a new directory or a new file that item’s ownership is automatically set to you. Your primary group is also assigned to the item. When you create an item while using the sudo (superuser do) command ownership is given to ‘root’ and the group is assigned to ‘root’.

cd ~/blogdemo mkdir bens_stuff sudo mkdir roots_stuff ll

An example of the automatic ownership and group assignments to newly created directories

To change ownership or group assignment of an item you can use the chown command.

To change the owner.
chown <user name> <item>

To change the group assignment apply a semicolon before the group name.
chown :<group name> <item>

You can change both the owner and group assignment together and also apply the changes to multiple items at once.
chown <user name>:<group name> <item1> <item2> <item3>

In Ubuntu you can obtain a list of users and the list of groups by running the following.
nano /etc/passwd nano /etc/group

To change the item permissions you can use the chmod command.

chmod <option> <mode> <item>

chmod -c will verbosely display all changes to each item you make which is useful in checking for mistakes.

The <mode> is broken down into three sets of requirements that we will isolate with squared brackets.

chmod -c [class][sign][permission] <item>

The [class] set are the user classes of the item you wish to modify and it permits the following options.
[ugoa] u user (owner), g group, o other, a all.

You can apply more than one class at a time, for example the following would make changes to both the item’s user and the group classes.

chmod -c ug[sign][permission] <item>

While the all (a) class is a shortcut that combines the user (u), group (g) and other (o) classes. The two commands below have identical functionality.

chmod -c a[sign][permission] <item> chmod -c ugo[sign][permission] <item>

The second set of input [sign] involves mathematical signs which represent the method we want to apply.
[-+=] + add, - remove, = set (replace)

+ add will insert [permission] to the item class.
- remove will delete [permission] from the item class.
= set will delete all existing permissions from the class and replace them with [permission].

The following would add [permission] to item’s user (owner) and group classes.

chmod -c ug+[permission] <item>

The following would remove [permission] from the item’s user, group and other classes.

chmod -c ugo-[permission] <item>

The following would replace all permissions with [permission] for the item’s user, group and other classes.

chmod -c a=[permission] <item>

Finally the third set of input [permission] involves the types of access you wish to grant. These attributes are commonly referred to as bits. The word ‘bits’ is a contraction of the term ‘binary digit’ which describes a two state representation of data ie ‘enabled’ / ‘disabled’.
[rwxX] r read, w write, x execute, X safe execute.

Remember I mentioned earlier in ‘File Permission Bits’ and ‘Directory Permission Bits’ that the execute bit has different functions depending on if it is applied to a directory or a file.

The safe execute (X) bit is recommended for use over the execute (x) bit as it does not apply the potentially dangerous execute bit to files and will only set it to directories. This is convenient when you mass-apply permissions to the content of a directory.

There are a couple of additional permission bits that in general can be ignored unless specifically needed.

[st] s set user/group identity, t sticky

The set user/group identity (s) bit can only be used with the user (owner) and group classes but due to insecurities this functionality is ignored on many Linux distributions including Ubuntu.

When applied to a file the (s) bit grants the other class elevated execute permissions. If the (s) bit is applied to the user class, the other class will have identical execute permissions to those of the owner. The same is true when the (s) bit is applied to the group class except then the other class will temporarily be assigned to the group when executing the file.

When applied to a directory the (s) bit will force any newly created files or directories to inherit the group assignment of the parent directory rather than that of the user creating the item. This functionality is ignored on most Linux distributions and so the (s) bit will probably have no effect.

More detailed information on this bit can be read at http://en.wikipedia.org/wiki/Setuid

Finally the sticky (t) bit is an elevated setting to allow only the item’s owner or a superuser permission (such as root) to delete or rename a directory. This bit is only useful when the other or group classes have the write bit applied to the same directory. The write and sticky bit combination would then allow these classes to create new or delete existing items within the directory but it would block any changes to the directory itself.

So in summary you have the following classes, signs and bits that you can assign to an item.

chmod -c [ugoa][-+=][rwxXst] <item>

Below is a sample of chmod commands and a brief description of their function.

chmod -c go+r <item>Adds the read permission to the group and other classes.

chmod -c a-w <item>
chmod -c ugo-w <item>
chmod -c -w <item>
All 3 do the same thing and remove the write permission from all classes.

chmod -c go+wt <directory>
Adds the write permission to the group and other classes and assigns the sticky bit.

chmod -c a=rwX *
Removes all previous permissions for all classes and assigns read, write and safe execute to all items in the current directory. By using the safe execute bit (X) only directories will be set with an execute bit, the execute bit for files will remain unchanged.

chmod -cR a+rX *
Adds the read and safe execute permission to all classes in the current directory. The -R is a recursive option to mass apply changes to all items contained within the subdirectories. This is potentially dangerous as for example if you accidently ran chmod -R +x / you would grant execute permission to all users for nearly every file on your Linux file system!

chmod -c o= <item>
Will set the other class permission to deny.

Octal Numeric Modifications

While you now have all the information that you need to successfully change and modify directory-file user ownerships and permissions we are not yet finished. Often in Linux walkthroughs or instructions you will encounter a different set of numeric only permissions being used with the chmod command. For example..

chmod 755 <directory>

These numbers are known as octal notation permissions.

Octal numerals are a set of numbers with a base of 8 that use a numeric range of 0 – 7. This is slightly different to the more familiar decimal numerals that use a base of 10 with a numeric range of 0 – 9.

These octal based permissions have 4 specific digits that represent permission bits.

0 represents ‘-’ which is the denied bit.
1 represents ‘x’ which is the execute bit.
2 represents ‘w’ which is the write bit.
4 represents ‘r’ which is the read bit.

You can sum (add) different numeric values to represent combined bits.

1+2 = 3, represents ‘w-x’ which are the write and execute bits.
1+4 = 5, represents ‘-rx’ which are the read and execute bits.
2+4 = 6, represents ‘wr-’ which are the write and read bits.
1+2+4 = 7, represents ‘wrx’ which are the write, read and execute bits.

Now chmod requires a 3 digit numeric value between the range of 000 and 777. You cannot supply a single (0-7) or a double digit (00-77) value. As each digit represents a permission assignment for one of the three item classes. The first digit is for the user (owner) class, the middle digit is for the group assignment, while the final digit is for other.

So an octal permission value of 750 is..
7 = user (owner) class, write (2), read (4) and execute (1) bits.
5 = group class, read (4) and execute (1) bits.
0 = other class, the denied (0) bit.

Now hopefully these octal numbers start to make a bit more sense. But a word of warning the use of octal numbers will always overwrite any preexisting bit assignments for all classes. It is a use case shortcut and is not a replacement for adding or removing individual bits.

chmod -c 000 <item>
or in text
chmod -c a= <item>
This removes all permissions to the item.

chmod -c 755 <directory>
or in text
chmod -c u=wrx <directory>
chmod -c go=rx <directory>
This sets the user (owner) to have write, read and execute. While group and other classes have read and execute.

chmod -c 644 <file>
or in text
chmod -c u=wr <file>
chmod -c go=r <file>
This sets the user (owner) to have write and read permissions while the group and other classes have read only.

Congratulations that is the Linux and Unix file permissions and user classes in a nutshell. I hope it has not be too grueling and please feel free comment or ask any questions. Alternative articles on this subject can be found at the following links.

Ubuntu Community Pages: File Permissions
https://help.ubuntu.com/community/FilePermissions

Zzee Linux Permissions Help
http://www.zzee.com/solutions/linux-permissions.shtml

Mac-Terminal: Manage permission of files and folders in the Terminal for Mac OS X
http://www.mac-terminal.com/files-and-folders/permissions/

block special files, character special files, chmod, chown, classes, command line, fifo, groups, Linux, ll, ln, ls, Mac OS, named pipes, octal notion, owner, permission bits, permissions, symbolic link, terminal, unix