Cataloging SDHC Cards on Ubuntu using a bash script

I have a lot of SD memory cards. I use them for my Camera, my Raspberry Pi, and for my laptop. They are cheap enough, I get several spares. And it makes it easy to convert my Raspberry Pi into different types of systems, allowing me to switch between different projects by just swapping the cards.

But when I grab one, I have to figure out what I last used that card for, and if there is anything worth saving on the card, and how much room I have left on the card. Did I use it for a camera? For transferring files? For a RPi system? I could keep a piece of paper with each card, label them, or store then in different places, but like all Unix script writers, I am lazy. I’d rather spend hours debugging a shell script than spend an extra minute every time I remove an SD card.

So I wrote a Bash shell script. To use it, I simply put the SD card into my laptop, and execute

SDCatalog

I might prefer to add a note to indicate something about the card, such as the manufacturer, This is an additional argument that is added to my “catalog.”

SDCatalog SanDisk

And when I am done, I eject the card, put in  a new one, and repeat. The “data collection” is simple – My script just uses find to list the file names and store the results into a file – one file for each SD card. All of the files are stored in a single directory. I use the directory  ~/SD to store the files. So what do I collect besides the filenames? I collect (a) the card capacity, (b) the size of the individual partition, (c) how much of the partition is used,  and (d) the unique ID of the card. I store this information in the name of the file. And because I have the files stored on the card, I can use grep to figure out which card contains which file. For instance, if I am looking cards formatted for the Raspberry Pi, I can search for the  /home/pi in the “catalog” using grep:

grep '/home/pi$' ~/SD/*

and the output I get is:

SD04G_p2_4_45_b7b5ddff-ddb4-48dd-84d2-dd47bf00694a:./home/pi
SD08G_p2_3_80_3d81d9e2-7d1b-4015-8c2c-29ec0675f162:./home/pi
SDC_p2_30_20_af599925-1134-4b6e-8883-fb6a69ad57f2:./home/pi

I used the “_” character as a field separator in the filename.   Looking at the first line above, the identifier is “SD04G_p2_4_45_b7b5ddff-ddb4-48dd-84d2-dd47bf00694a”. There are 5 “fields” separated by “_” which in my script is decoded as follows:

SD04G - The string seen when the card is mounted - see dmesg(1)
p2 - which partition
4 - 4GB partition
45 - 45 % used of the 4GB partition
b7b5ddff-ddb4-48dd-84d2-dd47bf00694a - Device ID

The “:” and “./home/pi” are added by grep.

It would be trivial to use a script like AWK to parse this and pretty print the results. I’ll print my awk script at the end of this. post. But let me explain part of this script.

When I  mounded a SDHC card, and typed df, the following was the output:

Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sda6       47930932  22572060  22917420  50% /
none                   4         0         4   0% /sys/fs/cgroup
udev             3076304         4   3076300   1% /dev
tmpfs             617180      1420    615760   1% /run
none                5120         4      5116   1% /run/lock
none             3085884       240   3085644   1% /run/shm
none              102400        32    102368   1% /run/user
/dev/sda8      252922196 229762840  10304948  96% /home
/dev/mmcblk0p1  31689728   9316640  22373088  30% /media/bruce/9016-4EF8

To catalog this information, we have to parse the last line. The two critical parts are the strings “mmcblk0” and “media” – these may have to be tweaked depending on your operating system. So I made them easy to change with the lines:

SD=mmcblk0
MEDIA=media

I also store the results in a directory called ~/SD – but someone may wish to store them in a different location. So my script defines the location of the log with

LOG=${SDLOG-"$HOME/SD"}

If the environment variable SDLOG is defined, use that value, otherwise use the default.

Sometimes it’s easier to describe a card using an optional comment, such as the color, manufacturer, or device where you got the card from. So I define the value of EXTRA with an optional command line argument.

EXTRA=$1

The script then checks if the directory for the log exists, and if not, it creates it.

The script tries to get a manufacturer’s ID for the device using dmesg(1). It does some checking to make sure the card is mounted, that the output file is writable, etc. It also looks for all of the partitions on the card.

But before I show you the script, let me show you a simple awk script called SDCParse that pretty-prints the names of the files used to store the catalog. For instance, the command

ls ~/SD | SDCparse

prints the following on my system – one line for each partition of each card:

 
      Note   Man.  Partition  Size   Usage ID                            
            00000         p1     2       1 E0FD-1813                     
   SanDisk  SL08G         p1     8       1 6463-3162                     
            SD04G         p1     0      30 3312-932F                     
            SD04G         p1     4      67 957ef3e9-c7cc-4b32-9c77-9ab1cea45a34
            SD04G         p2     4      45 b7b5ddff-ddb4-48dd-84d2-dd47bf00564a
            SD08G         p1     0      18 boot                          
            SD08G         p1     8       1 ecaf3faa-ecb7-41e1-8433-9c7a5d7098cd
            SD08G         p2     3      80 3d81d9e2-7d1b-4015-8c2c-29ec0875f762
              SDC         p1     0      34 boot                          
              SDC         p1    32      30 9016-4EF8                     
              SDC         p2    30      20 af599925-1134-4b6e-8883-fb6a99cd58f1
            SL08G         p1     8       1 6463-3162 

So this lets you see at a glance a lot of information about each SD card. I can easily see that I have 3 8GB cards whose p1 partition only uses 1% of the space. The awk script SDCParse is

#!/bin/sh 
awk -F_ '
BEGIN {
    ST="%10s %6s %10s %5s %7s %-30s\n"
    printf(ST,"Note", "Man.", "Partition", "Size", "Usage", "ID")
}
{ 
    if (NF == 5) {
        printf(ST,"", $1, $2, $3, $4, $5)
    } else if ( NF == 6 ) {
        printf(ST,$1, $2, $3, $4, $5, $6)
    } else {
#        print ("Strange - line has ", NF, "fields: ", $0)
    }
}'

I should mention something I use often. The formatting of both the table headers, and the data, uses the same formatting string “ST”. If I want to change the spacing of the table, I only need to change it in the one line.

The SDCatalog script, which generates the catalog, is not quite so simple, but it seems to be robust.

#!/bin/bash

# Bruce Barnett Tue Oct  7 09:19:41 EDT 2014

# This program examples an SD card, and 
# created a "log" of the files on the card
# The log file is created in the directory ~/SD 
#    (unless the envirnment valiable SDLOG is defined
# Usage
#    SDCatalog [text] 
#          where text is an optional field
# Example:
#    SDCatalog
#    SDCatalog SanDisk

# Configuration options
# These may have to be tweaked. They work for Ubuntu 14
# The output of df on Ubuntu looks like this
#/dev/mmcblk0p1   7525000 4179840   2956248  59% /media/barnett/cd3fe458-fc22-48e4-8
#     ^^^^^^^                                     ^^^^^      
#     SD                                          MEDIA
# therefore the parameters I search for are
SD=mmcblk0
MEDIA=media


# Command line arguments
EXTRA=$1 # is there any extra field/comment to be added as part of the filename

# Environment variables

# Note that '~/SD' will NOT work because the shell won't expand '~'. 
#  "~/SD" will work, but "$HOME/SD" will always work
LOG=${SDLOG-"$HOME/SD"} # use the $SDLOG envinrment variable, if defined, else use ~/SD


# Make sure the initial conditions are correct - we have a directory?
if [ -d "$LOG" ] # If I have a directory 
then
    : okay we are all set
else
    if [ -e "$LOG"  ]
    then
        echo "$0: I want to use the directory $LOG, 
but another file exists with that name - ABORT!" 
        exit 1
    fi
    echo "mkdir $LOG"
    mkdir "$LOG" # the directory does not exist
fi


# Now I will execute df and parse the results
# sample results may look like this
#/dev/mmcblk0p1   752000 417940   295248  59% /media/barnett/cd3fe458-fc22-48e4-8

# For efficiency, let's just execute df once and save it
DFOUT=/tmp/${0##*/}.$$.tmp 
# create a temporary filename based on script name, i.e. 
#       Example temporary name
#            SDCatalog..tmp
trap "/bin/rm $DFOUT" 0 1 15 # delete this temp file on exit

df>$DFOUT

# is there anything mounted?
grep -q "$SD"  <$DFOUT || ( echo no SD card mounted ; exit 1)
# get the manufacturer of the card
# dmesg will report something like
#      [ 8762.029937] mmcblk0: mmc0:b368 SDC   30.2 GiB 
# and we want to get this part           ^^^^
# New version that uses the $SD variable
MANID=$(dmesg | sed -n 's/^.*'"$SD"':.mmc0:\(....\) \([a-zA-Z0-9]*\) .*$/\2/p' | tail -1)
# Get the current working directory
CWD=$(pwd) # Remember the current location so we can return to it



for p in p1 p2 p3 p4 p5 p6 p7 p8
do


    # get the mount point(s) of the card
    MOUNT=$(awk "/$SD$p"'/ {print $6}' <$DFOUT )
    # get the ID of the card
    if [ -n "$MOUNT" ]
    then
        # Get the size of the disk
        SIZE=$(awk "/$SD$p"'/ {printf("%1.0f\n",$2/1000000)}' < $DFOUT)
        # Get the usage of the partition
        USE=$(awk "/$SD$p"'/ {print $5}' < $DFOUT | tr -d '%') #        ID=$(echo $MOUNT| sed 's:^.*/::')         ID=$(echo $MOUNT| sed 's:/'"$MEDIA"'/[a-z0-9]*/::')         # I am going to store the results in this file         if [ -z "$EXTRA" ]         then             X=         else             X="${EXTRA}_"         fi         OFILE="$LOG/$X${MANID}_${p}_${SIZE}_${USE}_${ID}"         echo Log file name is $OFILE         cd "$MOUNT"         touch $OFILE  || (echo cannot write to $OFILE - ABORT)         echo "sudo find .  >$OFILE"
        sudo find .  >$OFILE
        cd "$CWD"
    fi
done


I hope you find this useful.

Advertisements
This entry was posted in Linux, Shell Scripting and tagged , , , , , , , . Bookmark the permalink.

One Response to Cataloging SDHC Cards on Ubuntu using a bash script

  1. Faz says:

    s/mounded/mounted/ 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s