Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-)

Nikolai Bezroukov. Portraits of Open Source Pioneers

For readers with high sensitivity to grammar errors access to this page is explicitly prohibited :-)

Shells History Webliography

Sven Mascheck's pages provide background information about the Bourne shell.

See manual pages of two main predecessors of the Bourne shell, the "Thompson" and the "Mashey" shell:

• Version 3 Thompson shell manual (1/15/73). Pipes are denoted by < and >.
  (comments from Normal Wilson on TUHS list.)
• Version 4 Thompson shell manual (4/18/73). Here, pipes are denoted by ^ and |.
  (DMR about this on his pages and at TUHS.)
• Version 6 Thompson shell manual (5/15/74).
• Programmers Work Bench Mashey shell manual (5/31/77).
• (for comparison here also the Version 7 Bourne shell manual)

Jeffrey A. Neitzel maintains a source package of a reimplementation of the Thompson shell, which compiles on current systems.
You'll also find some example scripts there.

See also an extensive V6 manual collection and a simh tape from Wolfgang Helbig

source port of a current Bourne shell by Gunnar Ritter. This is the first officially available Bourne shell source code.

[Mar 4, 2000] Slashdot Articles AT&T's Korn Shell Source Code Released KSH93 source was released.

Henk Langeveld writes, "This announcement can be found at kornshell.com: March 1, 2000: I am happy to annouce the the 'i' point release of ksh93 is now available for download. For the first time, source is available as well as binaries for several architectures. If you build binaries for new architectures, and send them to us, we can add them to the download site. The download page has been completely revised in a manner that hopefully will be easier to use. ksh93 is part of the ast-open package. tksh (ksh with tk support) is also part of this package. -- David Korn As a long-term fan I'm glad to see the korn shell now being released under a new license. The license is quite non-standard, and does include some restrictions (changes can only be distributed as patches), but as far as I can see it does allow anyone to bundle binaries with their products. The distribution format is quite non-standard. The research group at AT&T has their own packaging system, built around nmake. "

 Usenet posting by John Mashey that summarize shell history till mid 80th very well:

From: mash@mips.UUCP (John Mashey)
Newsgroups: net.unix-wizards
Subject: Re: Shell history, true facts, but long
Date: 18 Mar 86 09:05:12 GMT

At Todd Williams' request:

1) In the Beginning, there was the shell, /bin/sh, by Ken Thompson. I can't remember what version this appeared in, but it was well before my time, i.e., it sure looked and acted like a shell before late 73. For more info on the early days, see DMR's paper in the 2nd UNIX BLTJ. Almost nothing was builtin (on purpose), but there were separate if/goto commands. Goto acted by seeking the file pointer it shared with its parent shell. For a single-stream command-line interpreter, most, if not all of the fundamental (in my opinion) ideas were already present. See reference [1].

2) During the period 1973-1976, lots of people hacked on that shell in various and sundry ways. USG (UNIX Support Group at Murray Hill) added some things, and so did we:

3) The "PWB Shell" first appeared in mid-1975. It derived from a set of requirements and suggestions from me in early 1975 in trying to do serious shell programming. In mid-1975, the shell acquired variables, including 3 that were derived from per-process data. This is where the idea of more generalised path- searching came in. Some of the existing support commands were beefed up, including if/else/endif. Most of this work was done by Alan Glasser or Dick Haight. I ended up taking this over, doing performance analysis, moving most of the control structure commands into the shell, adding switch,, while, etc. etc. By this time there was serious shell programming in massive amounts. This had mostly stabilised by early/mid 1976, although a few more features were added in early 1977. For various reasons this came to be called the "Mashey shell", a term I universally abhorred, since it was fundamentally still a Thompson shell with selected additions and tunings by various of us, constrained to be mostly upward compatible, leading to some things that I found less than esthetic. Even less than esthetic was the title of a later internal Bell Labs course: "Bourne Shell Programming for Mashey Shell Programmers". See [2] and [3].

4) The Bourne shell work started either in early 1976, or maybe late 1975. The first version was VERY different; for example, it used $ as the escape character instead of \. During 1976 there was a long sequence of modification, dialog, discussion, with various players in shifting alliances [like the game Diplomacy]. Besides the Computing Research crew, SRB himself, and me, this is when (as I recall) Dave Korn started getting involved (as heavy earlier user and extender of Bourne sh). To summarise, some of the Bourne shell's fundamentals came from the Thompson shell, but some were new. Some of the added semantics, and a few pieces of syntax came either from the PWB shell, or by agreement on what it ought to be, given that everything was changing anyway. [For example, the environment idea was a much-needed generalisation of the restricted PWB variables; this came from DMR, SRB, and I.] This did become the standard shell, although it took PWBers a while to convert, since they had huge investments in existing shell procedures. [Not that many had been written in the standard V6 or earlier shells.] See [4]. As a minor tidbit, how many 68K programmers out there have cursed the shell's trapping of memory fault to grow it's storage? I admit to causing this, having goaded SRB into it by moaning at the speed of the then-current Bourne shell vs the faster PWB shell.

5) I haven't tracked the exact influences on the C-shell. However, I believe that certain pieces were at least derived from the PWB shell's documentation (implementation, if not always exact syntax, of control cmds): if-the-else- endif, break, breakaw, onintr, while-end. Ask Bill where he got everything. Finally, note that there have been a number of other interesting and important shells around, but I'm tired, and I think more people understand the later work.

[1] K. Thompson, "The UNIX Command Language", in Structured Programming, Infotech State of the Art Report, Nicholson House, Maidenhead, Berkshire, England: Infotech International Ltd (March 1975), 375-384.

Classic Thompson-style paper of maximal insights per word, incl: "Many familiar computing 'concepts' are missing from UNIX. Files have no records. There are no access methods. User programs contain no system buffers. There are no file types. These concepts fill a much needed gap...

[2] J. R. Mashey, "Using a Command Language as a High-Level Programming Language", Proc. 2nd Int. Conf on Software Engineering, IEEE (Oct 13-15, 1976), 169-176.

Includes some usage data; state of the world in PWBland in mid-1976.

[3] T. A. Dolotta, J. R. Mashey, "Using a Command Language as the Primary Programming Tool", in Command Language Directions: Proc. IFIP Working Conference on Command Languauges, Sept 10-14, 1979, D. Beech, Ed, North- Holland, AMsterdam, 1980.

3 years later; more data; Bourne shell(called here nsh) included.

[4] S. R. Bourne, "An Introduction to the UNIX Shell", BSTJ 57, No 6, Part 2 (Jul-Aug 1978), 2797-2822.

-john mashey UUCP: {decvax,ucbvax,ihnp4}!decwrl!mips!mash
DDD: 408-720-1700
USPS: MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086

Other classic historical papers:

• An Introduction to the UNIX Shell (sh) by S. R. Bourne -- the author of Bourne Shell -- the most interesting Unix shell language that was further developed in ksh, bash, zsh. HTMLized by Era Eriksson

• An Introduction to the C shell by William Joy.

• The new Korn Shell -- ksh93 by by David G. Korn, Charles J. Northrup, and Jeffery Korn  The paper provides a concise feature list of ksh93 and authored by the authors of ksh93. Ksh93 was influenced by Perl and Tcl. Opening ksh93 on March 2000 reminds me of the article posted a while ago on the original Unix source code being available. Something like "Open source in history"?  It's kind of sad that it took this long for one of the best shells to be released.

  Arrays

  ksh93 provides one-dimensional indexed and associative arrays. An array element is referenced as varName[subscript]. Indexed arrays use arithmetic expressions for subscripts. This permits computation within the subscript expression. The statement varName[3+8] for example, references the 11th element of the indexed array. (Arithmetic expressions are described more fully below).

  The elements of an indexed array can be initialized from a list using the varName=(....) command. This provides a convenient notation for initializing an array to contain the names of files in a given directory. The number of entries in the array describes the number of files found. As an example, consider the following statement to initialize the entries indexed array with the names of files found in the current directory: entries=(*)

  An associative array uses arbitrary strings for subscripts. We could, for example, create a state tax associative array and reference elements by the state name. This works even for space separated tokens within the string, such as New Jersey.

  typeset -A StateTax
  StateTax[New Jersey]=0.06
  print ${StateTax[New Jersey]}
  

  Several special positional parameter expansions are provided for array processing. Using ${varName[@]} refers to all elements of the array. The subscripts of an array can be referenced with ${!varName[@]}. The notation ${#varName[@]} provides the number of elements within the array. Elements within a numeric subscript range can be referenced using ${varName[@]:offset:length}. This special notation works with both indexed and associative arrays.

  Arrays are used throughout the example lsc script. We define video as an associative array with capability names from the terminfo database as subscripts. The definition of video is provided as a compound assignment for an associative array.

  video=(
     [bold]=$(tput bold)
     [reset]=$(tput reset)
     [reverse]=$(tput reverse)
  )
  

  Each element is assigned a value from the standard output of a tput execution for the capability name. For example, video[bold] is the terminfo sequence for bold lettering. Similarly, video[reverse] will provide reverse video output.

  Using the notation $(command) will cause command to execute in a subshell of the current ksh. In many instances, ksh will not actually fork/exec a subshell when command is a built-in or a shell function. (Built-in functions are described below).

  Expanded Name Space

  In ksh93 a variable is defined by a name=value pair. The variable name space is hierarchical with . (dot) delimiters. The expanded name space permits an aggregate definition for a variable.

  ... .... .... 
  

  Functions:

  ksh93 provides two methods for function definitions. The formats are given as:

  function name
  {
      body
  }
  name()
  {
      body
  }

  The second function format is provided for compatibility with POSIX standards. The primary distinction is that of variable name scope. In a POSIX function, a variable definition has global scope. In the following POSIX function bar, variable foo is redefined to a value of 6.

  typeset foo=5
  bar()
  {
      typeset foo=6
      echo $foo
  }
  bar
  6
  echo $foo
  6
  

  Variable definitions in ksh93 functions have local scope. In the following ksh93 function bar, a local variable foo is defined and has precedence over the global variable foo.

  typeset foo=5
  function bar
  {
      typeset foo=6
      echo $foo
  }
  bar
  6
  echo $foo
  5

  Discipline Functions

  ksh93 provides active variables through a series of discipline functions. From the shell level, you can write get, set, and unset disciplines. Through the KornShell Development Kit, you can also add disciplines unique to your environment.

  When a variable is referenced, as in $foo, ksh will invoke the get discipline associated with foo. The default discipline is to simply return the current value associated with foo. From the shell level, you can define a foo.get discipline function.

  The set discipline is called when a value is assigned to a variable. Within the set discipline, the special variable .sh.name is the name of the variable whose value is being set.

  On line 31 of lsc, we define a max_stringSize.get discipline function. Every reference to ${max_stringSize} will result in this function being executed. The value of the special .sh.value variable is the value returned from the discipline.

  printf Statement

  In ksh93, a printf statement is available following the ANSI C printf definition. This permits formatting specifications to be applied to each argument. To appreciate the differences between the standard print and printf statements, consider how you would output the contents of the entries array (from the lsc example), one per line. The standard print statement would display the file names as space-separated tokens on a single line. Using the printf statement with a "%s\n" format, however, would produce the desired results.

  Arithmetic Commands

  ksh93 statements of the form (( expression )) are called arithmetic commands. Arithmetic commands return True when the value of the enclosed expression is non-zero, and False when the expression evaluates to zero. The construct $((expression)) can be used as a word or part of a word. It is replaced by the value of expression.

  In the lsc example, line 38, we evaluate the value of the discipline function using:

  (( .sh.value = $(strlenList ${entries[@]}) + 3 ))
  

  ksh93 will evaluate the expression, which includes an assignment to the .sh.value variable. Note that the:

  $(strlenList ${entries[@]})
  

  will invoke the strlenList built-in function and return the maximum width of the strings (given as element values) in the entries[] array. We add 3 to this value for formatting purposes.

  ANSI C Strings

  An ANSI C string is defined by preceding the single-quoted string with a $. For example, $'*' is the literal asterisk, *. With ANSI C strings, all characters between the single quotes retain their literal meaning, except for escape sequences. An escape sequence is introduced by the escape character \.

  ANSI C string support provides an essential feature for shell programmers. Consider, for example, having to process variables with embedded tabs in their values. Without ANSI C string support, we would not be able to effectively test the value of the variable for embedded tabs. As an example, consider the following script:

  print "foo\tbar" > /tmp/foobar
  read aline < /tmp/foobar
  if [[ "${aline}" == "foo\tbar" ]]
  then print TRUE
  fi
  

  The comparison (see Conditional Commands, below) will fail. We can replace the conditional with ANSI C strings and ensure proper functionality. The example above should be rewritten as:

  print "foo\tbar" > /tmp/foobar
  read aline < /tmp/foobar
  if [[ "${aline}" == $'foo\tbar' ]]
  then 	print TRUE
  fi
  

  On line 45 of Listing 1, we must test to see if the directory is empty. The preceding entries=(*) in an empty directory will set the entries variable to the literal asterisk if no files are found.

  Conditional Commands

  A conditional command in ksh93 evaluates a test-expression and returns either True or False. Conditional commands can be used as part of an ``Or list'' (||), ``And list'' (&&), or as part of an if-elif-else command. Conditional commands have the format:

  [[ test-expression ]]
  

  When used in conjunction with an ``And list'', ksh93 evaluates the test-expression and will execute the ``And component'' only if the test-expression evaluates to True. We use a conditional command as part of an ``And list'' such that the return statement will be executed only if the test-expression is True.

  [[ ${entries[0]} == $'*' ]] && return 2
  

  Iteration Control

  The for command has two formats. The traditional format is provided to iterate on each word in a list. The format is:

  for variableName [ in word-list ]
  do   compound-list
  done

  An arithmetic for command has been provided that is very similar to the C programming language for statement. The format is:

  for (( initExpr ; condition ; loopExpr ))
  do   compound-list
  done
  

  The initExpression is evaluated by ksh prior to executing the for command. The condition is then evaluated prior to each iteration of compound-list. If the condition is non-zero, then ksh executes the compound-list. The loopExpression is evaluated at the end of each iteration.

  Name Referencing

  A new typeset option has been added for name referencing. Using typeset -n nameReference=variableName will associate nameReference with variableName. A special alias, nameref, is provided as the equivalent for typeset -n. A shell script may use the reference name to refer to the variable name. Name referencing provides a convenient mechanism to pass the name of compound variables, or arrays, to ksh functions. This is more efficient than passing the variable's content.

  In the lsc example, function setOutput must add the directory entries to the appropriate row and column. We could have defined separate functions named addToRow and addToColumn for this purpose. The main body of the functions, however, would be equivalent. Instead, we opted to write a single function addToCell that uses a nameref to the cell type passed as a parameter.

  The addToCell function accepts three arguments, of which the first two are required. The first argument is the cell type and must be either row or col. We create a nameref using the local variable cell to be equivalent to the cell type specified. A reference to ${cell.index} would therefore be equivalent to ${row.index} or ${col.index}.

  FPATH

  ksh functions are not inherited across invocations of ksh. A child shell process, for example, does not have access to the functions defined within the parent ksh invocation. This has historically limited the re-usability of ksh functions. As a solution, ksh93 will search the colon-separated list of directories given by the FPATH variable value, for an executable file with the same name as the function. In the lsc example, we can eliminate the last statement:

  lsc "${@}"
  

  The FPATH can then be set to the directory containing the lsc file. From the shell level, we can now call lsc. ksh93 will load the lsc script and will call the lsc function with the command line arguments specified. Note that the supporting functions defined in the lsc script are available to the lsc function.

  A function autoload feature is provided, in which an auto-loaded function definition is loaded and retained within the ksh93 environment upon the first reference to the function name. This provides better performance since the search and load steps are eliminated for subsequent references.

• Book Practical Reusable UNIX Software (contains links to ksh93 and a couple fragments, see below):
  • Foreword by Dennis Ritchie
  • Historical Overview
• CSH Programming Considered Harmful -- old and somewhat controversial article written by well-known Perl evangelist

Last modified: August 27, 2008