Tutorial 2: Editing files and simple shell commands

Prerequisites

If you have not already done so, login to Linux and familiarise yourself with the windowing environment by starting 2 or 3 simple applications. Try using these, resizing, minimising, putting selected applications to the front or in full-screen mode and closing them. Many modern windowing management systems (e.g. KDE, MS-Windows) have a start button, by default in the lower-left hand corner of the screen which you can use to select applications from a menu. Some window managers (e.g. OpenWindows) will use the right mouse button on the display background to present this menu or other useful options. The Gnome window manager uses 3 menus, by default on the top of the desktop:

Experiment to get the feel of the various window management options, remembering to logout when your session ends.

Learning outcomes

At the end of this session you should be able to use a text file editor to create, merge and modify simple text files. You should be able to use simple shell commands to list directories, change directories, see where your working directory is within the filesystem tree, and to copy and display files, and count the numbers of words, lines and characters in files. You should be able to use the man command or other sources of reference information to find out more about all of the shell commands you have used.

While shell commands will be much the same across many different Unix and Linux systems, various simple and complex text editors are likely to be available - to get the best use of these you will need to do some of your own experimentation and learning. Also while the console or windowing systems likely to be found on all Unix - like systems will give access to the same shell commands, the precise details of how to access an application providing a shell-command interface will vary, so unless you can work out these details yourself, these notes may need to be supplemented by your tutor or someone else giving you a quick guided tour on the system you are using.

Using a text editor

If you are used to using a GUI-based text-file editor such as Notepad, you are likely to want to try one of the various equivalents available on Linux for editing text files. Any modern Linux installation should have equivalent tools available, typically by selecting an option on the menu made available through the start button, the workspace menu or a button bar. Details of available editors will vary between Linux distributions and versions. Not all Linux installations are installed with the same text editors, but most will have a selection. If you are able to complete the exercise below using a GUI editor such as Gedit, that will cover what you need to know.

If you are feeling a bit more adventurous consider how much time in your computer- using career you are likely to be spending editing text files. If this is going to be more than a few hours, you are likely to be more concerned about how efficiently you can use a text editor than how intuitive it is on first use. In practice the text editing programs which are most efficient for skilled users are also more difficult to get started with. To be most efficient an editor should be driveable entirely from the keyboard without needing to take your hands from the position a touch typist would use, in order to do anything commonly related to using the editor (e.g. compiling and running a program, saving files, cutting and pasting between files etc.), using as few keystrokes as possible.

Text file editors which fit into the efficient but requiring driver-training category, and which are generally available on many or all Unix and Linux systems include Emacs and Vi (often known as Vim). Teaching detailed use of either is beyond the scope of this tutorial, but some information on vi is provided at the end of these notes to help get you started with it if you decide you want to use this tool. While traditional use of vim and Emacs are as console programs (e.g. use the vim or emacs commands from a shell), GUI versions of both are available. You can also learn the short-cut keys on a standard GUI type editor instead of using the mouse for the most common operations.

Another reason for learning a standard console-based editor is that you are increasingly likely to encounter Linux or Unix on a small memmory embedded system providing some form of console access but no GUI. Detailed notes helping you get started with vi are provided below.

To continue with this part of the tutorial, using an editor of your choice, create 2 simple text files e.g. the first in a file called mary containing:

 Mary had a little lamb,
 its fleece as white as snow,
 

and the second in a file called go containing:

 and everywhere that Mary went,
 the lamb was sure to go.
 

Next open mary to edit this file, and insert the contents of the go file after mary to complete the nursery rhyme and save it.

Using simple shell commands

File managers were designed to prevent those who don't want to know how computers work from having to learn how to use simple shell commands. But this approach isn't adequate for those who need to understand how computer systems work well enough to make things more secure. The problem is that walking comes before running, and that expressing your detailed intentions quickly and fluently to a machine isn't possible until you have learned how to tell it what to do in simple sentences.

Learning how to program using modern and powerful scripting languages (e.g. shell scripts, Visual Basic, Perl, Python, Ruby, Tcl) is much easier if you already know how to communicate with a system using single command sentences. Many routine systems- administration operations have also become possible in recent years through the use of GUI programs. Again, the problem is that GUI type tools restrict you to the user facilities of exising programs, while command lines allow greater flexibility and more power and expressiveness in manipulating multiple file objects in single actions.

Command lines are also a short step away from fully automated operations. For example, you may want to be able to make a machine carry out a pre-programmed operation in receipt of a short text message or external sensor condition, or at a particular time every night or every week, when you don't have the luxury of being able to sit in front of the machine to push a GUI button because you are asleep or doing something else. A GUI interface can do nothing without a human being sitting in front of it, while a command line can be executed by a schuduling program such as cron every day or week, or 5 minutes, or on even at 3.20AM on the third Sunday of all the months with the letter 'r' in them. For cron to be able to schedule and run this kind of job for you it will need shell command lines for it to run.

Command lines enable a machine to be controlled, administrated and upgraded by a system administrator from anywhere else on the Internet using a secure shell connection. Many if not most web servers or embedded networked systems have no other means of administration.

In order to run command lines your system must be able to give you access to a command console or shell tool from the start menu or button bar etc.

Experiment by running the following commands:

command purpose
ls List current directory.
mkdir newdir Make new directory called newdir
cd newdir change down to directory newdir which must exist
touch f1 f2 f3 create the 3 files called f1, f2 and f3 in the current directory
ls list the contents of the current (working) directory
pwd list the pathname of the working directory
ls -l give a long (l) listing of the working diretory with more information about each file
ls -al give a long listing of all files in the working directory including hidden files
ls -la same as ls -al. The order of the a (all) and l (long) flags doesn't matter.
cd .. change directory up one level to the parent of the working directory
pwd list the pathname of the working directory (should be where you started)
ls -l newdir give a long listing of non-hidden objects in newdir (newdir must exist).
mkdir dir2 dir3 create directories dir2 and dir3
cp newdir/f1 dir2 copy file f1 in newdir to dir2
ls dir2 check that f1 was copied
cp newdir/* dir2 copy all non-hidden files from newdir to dir2
ls dir2 check this is what just happened
cd dir2 change working directory to dir2
ls see same files as before
rm -f f1 remove (delete) file f1 forcibly ( with -f you won't be asked to confirm)
ls see same files as before minus f1
rm -f f? remove all files starting with an f followed by any character
ls see what happened
cp /etc/passwd . copy a file with an absolute path (starts with / ) to the working directory, you need a space between passwd and the (.) full stop. )
less passwd display paged listing of the passwd file as copied. The < space > key shows next page. Shell commands don't work within less. Subcommands which do work are particular to less. Make yourself familiar with the effect of h, b, q, enter and space keys within less.
wc passwd reports the numbers of lines, words and characters in the passwd file
wc -l passwd reports just the number of lines in the passwd file. What do the -w and -c flags (options or adverbs of wc) do ? And how are words and lines delimited ?
mv passwd password renames passwd file copy as password.
echo hello world output text: "hello world" to screen
echo hello world > hellofile output text as before but to create a file called hellofile containing the text: "hello world"
cat hellofile output text from hellofile to screen. Try this with a longer text file. cat doesn't page in the manner allowed by the more or less commands if hellofile has more than one page.
man ls displays system manual information about the ls command
man less displays system manual information about the less command
man -k copy show index of manual pages containing the word: copy in the short description

In your logbook write down the names of commands which perform the following operations:

Introduction to vi

This is optional for those choosing to learn use of this flexible and powerful text editor in preference to other text editors on Linux which use modern GUI conventions. Emacs is like vim in being high powered, and having a longer learning curve. There are also many less advanced but easier to learn alternatives which work in a similar manner to Windows Notepad. A very useful feature of vim (an enhanced version of vi) is that on consoles capable of colour display, text in many modern programming languages is syntax highlighted, helping you to identify, inspect and debug code blocks, expressions, parameters, identifiers and keywords etc. by the fact that these and the delimiting punctuation characters (e.g. {} [] () braces, "" quotes and punctuation) are coloured distinctively.

There are 3 separate modes in vim :

INPUT MODE - Whatever you type is inserted into the file at the cursor position. You use the <esc> key to exit from input mode to command mode.

COMMAND MODE - Is used to move the cursor within the file and carry out many operations such as undo, repeat last command, delete characters, words or lines, cut, copy and paste, join adjacent lines, save and exit. Use of i (insert) or a (append) commands will change mode to insert mode. Use of the : (colon) command will change mode to ex mode.

EX MODE - this can be used to carry out any command available to the older ex line editor (see man ex for details). These are typically used to control settings (e.g. display line numbers or hidden characters), and carry out search and replace functions. You may also use ex mode to abort the edit, save edits and retrieve text from another file. The ex mode only lasts for a single ex command, afterwards you are automatically returned to command mode; by convention ex mode commands are documented in vim by preceding them with the : (colon) used to switch modes.

Many vim commands may be preceded with a number. This has the effect of making the command operate on the number of objects rather than just one, e.g. the dd command deletes one line while 20dd will delete 20 lines. When in ex mode, commands may be made applicable to a range of lines, expressed as the 2 line numbers separated by a comma. The current line is expressed as '.' (period) while the last line can be referred to as $ so to delete all lines from the cursor to the end of file use the command
:.,$d
and To delete all lines from the 1st to the 20th you would say
:1,20d

Here is a summary of some of the most frequently used vim commands:

  create/insert text:
  i - insert before cursor       }
  I - insert at start of line    }-  these all put you
  a - insert after cursor        }-  into input mode
  A - insert at end of line      }
  <esc> - leave input mode
  o - add (open) line below cursor
  O - add (open) line above cursor

cursor movement:
  ^  - to line start      $  - to line end
  W  - forwards 1 word    B  - backwards 1 word
  nG - go to line n       G  - go to end of file
  h,j,k,l left,down,up,right one character
          (you are now better off using the arrow keys but
          these don't work on all ancient terminal emulations.)

deletion:
  x   - 1 character       dw  - 1 word
  dd  - 1 line            4dd - 4 lines
  u   - undo last change
  :n1,n2d - delete from lines numbered from n1 through to n2

cut and paste:
  y  - yank line (to paste buffer)
  5y - yank 5 lines
  p  - paste above cursor     P  - paste below cursor
  :r filename
     - insert text from file below cursor

save and quit:
  :w  - save without quitting   ZZ  - save and quit
  :q! - quit without saving

miscellaneous:
  J - join next line to current line
  u - undo last change
  R - type over a line
  :1,$p      display entire file
  :set nu    display line numbers
  :set all   show all settings
  :set list  display invisible characters
  :help      enter the vi help system

search and replace:
  /text  - find next occurrence of text
  ?text  - find previous occurrence of text
  n  - repeat last search     N  - reverse last search
  :1,$s?old?new?g
      change all occurrences of old to new throughout
      the file. You may use characters other than ? to
      delimit the search and replace strings.

To run vim from a command tool or shell simply enter vim filename. If filename does not exist it will be created, otherwise you can edit it. vi is the earlier basis of more modernised editors vim and gvim. If you boot a damaged or misconfigured Linux system from a rescue disk you are likely to be able to repair the system without needing a full reinstall by editing relevant configuration files. However, you may need to do so using the more limited editors e.g. vi, ex or ed present on the much smaller rescue system.