January 19

Linux: Ubuntu Mount/USB

Introduction

This page explains how to use USB drives, like external hard disks and USB flash drives (aka USB sticks, thumb drives, pen drives, etc). The material here also applies to flash cards (like in your digital camera).

USB storage devices have the enormous advantage that for the most part they use a standard set of protocols. Thus, instead of needing individual drivers, as does much computer hardware, a standard driver permits access to the devices, making them very portable and able to easily work on many platforms.

Automounting

Mounting

By default, storage devices that are plugged into the system mount automatically in the /media/<username> directory, open a file browser window for each volume and place an icon on your desktop. The rationale for this slight change of behavior can be found here. If you plug in a usb hard disk with many partitions, all of the partitions will automatically mount. This behaviour may not be what you want; you can configure it as shown below.

If the volumes have labels the icons will be named accordingly. Otherwise, they will be named “disk”, “disk-1” and so on.

Configuring Automounting

To enable or disable automount open a terminal and type:

dconf-editor

Browse to org.gnome.desktop.media-handling.

The automount key controls whether to automatically mount media. If set to true, Nautilus will automatically mount media such as user-visible hard disks and removable media on start-up and media insertion.

Another key, org.gnome.desktop.media-handling.automount-open, controls whether to automatically open a folder for automounted media.

If set to true, Nautilus will automatically open a folder when media is automounted. This only applies to media where no known x-content type was detected; for media where a known x-content type is detected, the user configurable action will be taken instead. This can be configured as shown below.

Configuring Program Autostart

To control which programs automatically start when you plug in a device, go to System-Settings – Details – Removable Media.

Unmounting/Ejecting

Before you disconnect the device, don’t forget to unmount it. This can be done in one of the following ways:

  • Right-click the desktop icon and select “Unmount” (or in some cases, “Eject”).
  • In the file manager window, click on the “eject” button next to the name of the mounted volume.
  • Right-click the icon in the launcher and select “Unmount”.

Auto-mounting (Ubuntu Server)

By default, disk drives do not auto-mount in Ubuntu Server Edition. If you are looking for a lightweight solution that does not depend on HAL/DBUS, you can install “usbmount”.

Manually Mounting

Using Disks

Disks (the GNOME disk utility) is an application for visually managing disk drives and media. When you run it, you will see a list of your drives, including USB drives. If you click a drive on the list, you can view its details, and you can click the triangle-shaped button (Play button) to mount the drive. (This method works even when the drive does not auto-mount.)

Using mount

Get the Information

Sometimes, devices don’t automount, in which case you should try to manually mount them. First, you must know what device you are dealing with and what filesystem it is formatted with. Most flash drives are FAT16 or FAT32 and most external hard disks are NTFS. Type the following:

sudo fdisk -l

Find your device in the list. It is probably something like /dev/sdb1.

Create the Mount Point

Now we need to create a mount point for the device. Let’s say we want to call it “external”. You can call it whatever you want, but if you use spaces in the name it gets a little more complicated. Instead, use an underscore to separate words (like “my_external”). Create the mount point:

sudo mkdir /media/external

Mount the Drive

We can now mount the drive. Let’s say the device is /dev/sdb1, the filesystem is FAT16 or FAT32 (like it is for most USB flash drives), and we want to mount it at /media/external (having already created the mount point):

sudo mount -t vfat /dev/sdb1 /media/external -o uid=1000,gid=1000,utf8,dmask=027,fmask=137

The options following the “-o” give you ownership of the drive, and the masks allow for extra security for file system permissions. If you don’t use those extra options you may not be able to read and write the drive with your regular username.

Otherwise, if the device is formatted with NTFS, run:

sudo mount -t ntfs-3g /dev/sdb1 /media/external

Note: You must have the ntfs-3g driver installed.

Unmounting the Drive

When you are finished with the device, don’t forget to unmount the drive before disconnecting it. Assuming /dev/sdb1 is mounted at /media/external, you can either unmount using the device or the mount point:

sudo umount /dev/sdb1

or:

sudo umount /media/external

You cannot unmount from the desktop by right-clicking the icon if the drive was manually mounted.

Using pmount

There is a program called pmount available in the repositories which allows unprivileged users to mount drives as if they were using sudo, even without an entry in /etc/fstab. This is perfect for computers that have users without RootSudo access, like public terminals or thin clients.

pmount can be used with the same syntax as mount (but without sudo), or quite simply as follows:

pmount <device> [ label ]

Example:

  • pmount /dev/sdb1 flash_driveThis will mount the device /dev/sdb1 at /media/flash_drive.

If you leave off the label option, it will mount by default at /media/device.

To unmount the device, use pumount, like so:

pumount <device>

Example:

  • pumount /dev/sdb1

For more help, see the man pages for pmount and pumount.

The Importance of Unmounting

Before disconnecting devices, you must unmount them first. This is similar to “Safely Remove” in Windows in that the device won’t unmount until data is finished being written to the device, or until other programs are finished using it. This applies to all types of storage devices, including flash drives, flash cards, external hard drives, ipods and other media players, and even remote storage like Samba or NFS shares.

Failure to unmount before disconnecting the device can result in loss of data and/or a corrupted file system. There are no exceptions to this rule. Be safe – unmount your drives before disconnecting them!

Other Useful Commands

To see a list of your USB devices (the vendor and device ID’s), run:

lsusb

To see all attached storage devices and their partitions, run:

sudo fdisk -l

To see information about currently mounted systems, simply run:

mount

Troubleshooting

Presented here are some common problems users encounter.

Interfering services

Two services/programs responsible for automounting might interfere and thereby prevent a successful automount and permission setting.

Example: Activating the Automount function of Nautilus while using pmount will result in read-only permissions for normal users. Either disable Nautilus’ Automount function or deinstall pmount.

Unclean LogFile

If you are mounting drives formatted with NTFS (like most external USB hard disks are), you must first have the ntfs-3g driver installed. This is done automatically in newer versions of Ubuntu. You should also install ntfs-config and enable mounting, which is not done automatically.

When a drive is not Safely Removed from a Windows machine (or a forced shutdown occurs from Windows), you may get an error like this when you plug in your drive:

$LogFile indicates unclean shutdown (0, 0)
Failed to mount '/dev/sda1': Operation not supported
Mount is denied because NTFS is marked to be in use. Choose one action:

Choice 1: If you have Windows then disconnect the external devices by
clicking on the 'Safely Remove Hardware' icon in the Windows
taskbar then shutdown Windows cleanly.

Choice 2: If you don't have Windows then you can use the 'force' option for
your own responsibility. For example type on the command line:

mount -t ntfs-3g /dev/sda1 /media/sda1/ -o force

The best option is Choice 1, but you can force the mount by running Choice 2 with sudo. You must then manually unmount it from the terminal (you can’t right click the desktop icon):

sudo umount <mount_point>

After that the drive should automount normally again.

User Privileges

If your usb device doesn’t appear on your desktop, you should check that your user has the correct privileges. Go to System->Administration->User and Groups, choose the user, click on “Properties”, then go to the “User Privileges” tab. You should have the “Access external storage devices automatically” option checked.

Preferences

If your usb device doesn’t appear on your desktop, you should check that the automount action is enabled in the preferences:

  • Navigate to System->Preferences->Removable Drives and Media

  • Verify that all “Mount removable drives when…” are checked.

NOTE: This does not seem to apply to Hardy Heron.

USB 2 Issues

old kernels workaround

If you encounter problems using your USB device with USB 2 (i.e. ‘high speed’ mode), you can revert to the ‘full speed’ mode (slower) by unloading ehci_hcd. To do that, type in a terminal:

sudo rmmod ehci_hcd

before plugging in your device.

recent kernels workaround, from Karmic

ehci_hcd is now built into the kernel and cannot be load/unloaded using modprobe. To revert a connected device from (failing) high-speed to full-speed:

  • Determine your device id using
     lsusb

     

  • Find which bus it is connected to. The bus id can be found as a folder in /sys/bus/pci/drivers/ehci_hcd. The following script explores buses and connected devices:
    pushd /sys/bus/pci/drivers/ehci_hcd > /dev/null
    for bus in 0000:??:??.? ; do
            echo "ehci_hcd bus $bus"
            pushd $bus/usb1 > /dev/null
            for dev in ?-?; do
                    idVendor=`cat $dev/idVendor`
                    idProduct=`cat $dev/idProduct`
                    echo "ehci_hcd bus $bus: device $dev = $idVendor:$idProduct"
            done
            popd > /dev/null
    done
    popd > /dev/null

    The information is also usually available in /var/log/kern.log

  • Unbind the bus (and all devices) from the ehci_hcd driver. Insert the bus id in the following command, using the format 0000:00:xx.x
     sudo sh -c 'echo -n "0000:00:xx.x" > unbind'

     

Buffer I/O Errors

If you see errors related to Buffer I/O when attaching a USB storage device, there are two ways to work around it. First, try using varying max_sectors settings, as such:

sudo sh -c "echo 120 > /sys/block/sda/queue/max_sectors_kb"

Try values of 120, 64 and 32.

If this does not resolve the issue, then you may need an unusual_dev entry for your device. It would look something like this:

UNUSUAL_DEV(0x03eb , 0x2002, 0x0100, 0x9999,
            "Generic",
            "MusicDrive",
            US_SC_DEVICE, US_PR_DEVICE, NULL,
            US_FL_IGNORE_RESIDUE),

The vendor and device IDs can be obtained from the output of “lsusb”. The entry would be placed in drivers/usb/storage/unusual_devs.h. If you cannot compile your own kernel, please file a bug report, and we’ll attempt to compile a test module for you.

Device suddenly becomes read-only

If your device changes suddenly to read-only mode, and you see this kind of error:

[17183798.908000] FAT: Filesystem panic (dev sda1)
[17183798.908000]     fat_get_cluster: invalid cluster chain (i_pos 0)
[17183798.908000]     File system has been set read-only

This might be the sign of an unclean device. You should check your device. Try TestingStorageMedia to do so. Or use “Disk Utility” (under System, Administration), find your device, unmount it, check the file system, then mount it again.

USB-Device is or becomes read-only without errors

If you see “Write Protect is off” and no errors in your logfiles, than you should set filesystem type specific mount options (FS_MOUNTOPTIONS) in /etc/usbmount/usbmount.conf. Wrong gid causes mounting read only.

General tip

When you encounter problems with USB devices, the first thing to do is to check the latest debug information generated from the kernel just after you plug in your device and/or just after you encounter the problem.
To do that, open a terminal and type :

dmesg

Check the latest messages; they should be related to your problem.

By: Ubuntu

Category: Linux | Comments Off on Linux: Ubuntu Mount/USB
January 11

Linux: Mount Vmware .vmdk file on linux using vmware-mount

Do you have a large .vmdk file on an external HDD that you need to access to pick up some files and don’t want to mount it with  a virtual machine player? 

This quick tip assumes that you are familiar with virtualization and have basic Linux knowledge, but even if you are not, go on and read it, it is basic level and easy to implement.

Vmware Converter, to those who are not familiar with it,  is a tool produced by Vmware that will help you convert a physical machine to a VM (virtual machine), it is a very handy tool if you are running a Vmware datacenter, converting a Physical machine to a VM provides you with mobility for the machine that once was very hard to move or transfer, preserve old software that is running on old risky hardware, Moreover a nice feature is that the Hard Disk real size would be the size of the data inside it, i.e a physical machine that has a 1 TB HDD but has only a 70 GB data would have a VMDK file of 70 GB in size, of course i will not go in to the general benefits of virtualization.

Technically the converter converts that whole physical machine in to a software Hard Disk of type .vmdk which can be run on Vmware player, workstation,fusion,  ESXI, virtualbox… etc, but sometimes you just need to keep a backup of certain machine without deploying it on any player, and when the time comes that you need to pickup some data files from the machine without taking it the whole 9 yards and deploying it, here is where the tool that I will use is called Vmware-mount which is a part of the VDDK (Virtual Disk Development Kit) comes in hand.

The Virtual Disk Development Kit ( VDDK) is a collection of C/C++ libraries, code samples, utilities, and documentation to help you create and access VMware virtual disk storage. The VDDK is useful in conjunction with the vSphere API for writing backup and recovery software, or similar applications.(Vmware definition)

I will be showing the steps of how to mount the .vmdk from the linux terminal : I am using Ubuntu Desktop 14.04 LTS

1. VDDK Download:

If you already have an vmware account you can download the VDDK from the following  URL  or you can use this link URL if you wish, I have uploaded it to google drive in case you don’t have permission to download it, you would need to use a browser to download it,  I have chosen the 5.1 version, because when I installed the 5.5 version the tool that will be used wasn’t available, and  don’t worry you don’t need to worry  you can open the 5.5 version  vmdk with it.

2. VDDK installation:

  • The  downloaded VDDK is a gz tarball that can be extract with the following

tar xzvf %path/to/tar/nameoftar

  •  go in to the directory just created in the same directory and you would find the following perl script “vmware-install.pl”, the file is already executable, but in case it is not, use “sudo chmod +x vmware-install.pl

run by “sudo ./vmware-install.pl”

  • keep the defaults, the tools would be installed in the /usr/bin so you don’t need to go to the directory, you can just open the terminal and run vmware-mount
  • vmware-mount is like a regular mount, so you need to create a mount point/directory for example:”/mnt/vmwaremount”, this directory will be the destination folder in the commands below.
  • The vmdk used in this example is a typical ntfs windows multi-partition HDD

3. The commands that  you need:

vmware-mount -p /path/to/vmdk
This will list the paritions inside the vmdk, so you would know what partition number to choose.

vmware-mount /path/to/vmdk %partition#% /path/to/mountdirectory
example:
vmware-mount /media/saeed/seagatehdd/mountme.vmdk 2 /mnt/vmwaremount
This will mount the second partition in the vmdk in the mounting point “/mnt/vmwaremount”

Now after mounting the vmdk, you can easily go to the mount/point use the regular Linux commands like:

cd /mount/point
for example
cd /mnt/vmwaremount
ls, cd , cp would be handy from here on

or you can use the GUI to go to the mount directory as an alternative

after you have finished and need to un-mount the vmdk in order to eject the external HDD, in order to un-mount cleanly the vmdk ( be careful the the vmdk would be damaged and would need repair in case you disconnect without cleanly un-mounting the mounted partitions), in order to do that you would need to run the below commands, you would use the -d command on all the mounted points, and you would keep checking the status of the mounted HDD using the -L, as long as the return of the -L command is not empty it means that the vmdk still has a mounted point.

vmware-mount -L
this will displays all virtual disks mounted, but not the partitions

vmware-mount -d /mount/point
This will un-mount the partition
for example to un-mount the partition that we mounted above
vmware-mount -d /mnt/vmwaremount

By: S Khodor

Category: Linux | Comments Off on Linux: Mount Vmware .vmdk file on linux using vmware-mount
December 8

Linux: Making a Ruby script run in cron

You need to setup your crontab with rvm e.g:

rvm cron setup

With that rvm sets your environment variables in your crontab file

then you have a crontab file having something similar to this at the top:

PATH="/usr/local/rvm/gems/ruby-1.9.3-p194/bin:/usr/local/rvm/gems/ruby-1.9.3-p194@global/bin:/usr/local/rvm/rubies/ruby-1.9.3-p194/bin:/usr/local/rvm/bin:/usr/local/rvm/gems/ruby-1.9.3-p194/bin:/usr/local/rvm/gems/ruby-1.9.3-p194@global/bin:/usr/local/rvm/rubies/ruby-1.9.3-p194/bin:/usr/local/rvm/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/rvm/gems/ruby-1.9.3-p194@global/"
rvm_env_string='ruby-1.9.3-p194'
rvm_path='/usr/local/rvm'
rvm_ruby_string='ruby-1.9.3-p194'
RUBY_VERSION='ruby-1.9.3-p194'
GEM_HOME='/usr/local/rvm/gems/ruby-1.9.3-p194'
GEM_PATH='/usr/local/rvm/gems/ruby-1.9.3-p194:/usr/local/rvm/gems/ruby-1.9.3-p194@global'
MY_RUBY_HOME='/usr/local/rvm/rubies/ruby-1.9.3-p194'
IRBRC='/usr/local/rvm/rubies/ruby-1.9.3-p194/.irbrc'

Then you can stick your crontask beneath it using crontab -e

By: bjhaid

Category: Linux | Comments Off on Linux: Making a Ruby script run in cron
December 7

Linux: How to reinstall newest Linux Kernel in Ubuntu

uname -r - shows running kernel version

dpkg -l | grep linux-image-.*-generic - shows installed kernel versions

Look for the kernel version you want to reinstall then run:

sudo apt-get install --reinstall linux-image-3.X.Y-ZZ-generic

By: E Carvalho
Category: Linux | Comments Off on Linux: How to reinstall newest Linux Kernel in Ubuntu