CISC361-010 Project 2

Assigned: March 16, 2009, Due: April 27, 2009
(set April 20, 2009 as your due date and it's like a week extension)

Adding some features to the shell

Objective

In this project you will be adding more functionality to your shell from project 1. So the first step is to get your shell from project 1 working, or at least enough to implement features for this project, if it doesn't already work. Features that you will implement include being able to background processes, file redirection and piping (IPC). Another objective of this assignment is to introduce programming with POSIX threads (pthreads). Experience with some more system calls and C library functions will also be done to hone your programming skills. Reading of man pages will also be a must to do well.

The Assignment

The first task is to implement the ability to background a job using '&'. This needs to only work for executing external commands. What you need to do is identify that the last thing given on the command line is a '&', the program is executed in parallel with the shell, rather than having the shell wait for it to complete. The shell should go and print another prompt to get the next command to execute. It is possible that several backgrounded jobs could be running at the same time.
As discussed in class when a parent does not do a wait call for its children the children will become zombie processes when they exit. To prevent this for your backgrounded jobs you need to do a nonblocking waitpid(2) call at some point. See the man page for options to use to reap an entry in a nonblocking fashion. I suggest doing this sometime before printing the next prompt. If the parent has a zombie child the call will reap the process entry, otherwise no harm is done either.

Add a watchmail builtin command. This command takes as the first argument a name of a file, which must already exist (give an error if it doesn't), to watch for new mail in. It takes an optional second argument of "off" to turn off of watching of mail for that file. It should start up a new thread using pthread_create(3thr) for each file to watch for mail in (the name of the file is passed as a parameter). This function, to watch for new mail, should run a while ( 1 ) loop, which will do a stat(2) to see if the file got bigger since the last check. A sleep for 1 second should be in the loop. The st_size member of a stat struct can be used to check the size of the file. When the file increases in size, the thread for that file should print out:

BEEP You have new mail in FILE at TIME

The BEEP should be done with a \a in printf(). FILE should be the name of the file that the thread is watching. And TIME should be the current time in ctime(3C) format. To get the current time use gettimeofday(3C) and pass tv_sec of the timeval struct to ctime().
When the "off" parameter is given the thread that is watching that file needs to be canceled with pthread_cancel(3thr). A linked list of active threads and files associated with each one should be maintained to support this. When an error condition happens in a thread pthread_exit(3thr) should be used. Your program should support watching multiple files. A separate thread will run to check each file.
Note that this command will print when a file it is watching gets bigger, which could be annoying when using the shell for normal functioning. However if you have multiple windows open this could be useful to run in one for notification.

Add support to your shell so the following file redirections will work:
- > - redirect standard output of command on left to file on right (i.e. command > file)
- >& - redirect standard output and standard error of command on left to file on right (i.e. command >& file).
- >> - redirect standard output of command on left to file on right (i.e. command >> file), appending to the file.
- >>& - redirect standard output and standard error of command on left to file on right (i.e. command >>& file), appending to the file.
- < - redirect standard input of command on left to come from file on the right (i.e. command < file).
To do the file redirection requires that you understand file descriptors well and how child processes inherit from their parent. You will need to close and reopen file descriptors 0, 1 and 2. For this you will need to use the open(), close() and dup() system calls. Please use dup(2) instead of dup2(3C). This doesn't require a lot of code just the right calls in the right places. To get the normal function of stdin, stdout and stderr back opening of /dev/tty will be required. Be sure to use the right options for open() in each situation (i.e. read/write, truncate/append). Read man pages!
To help out in testing both stdout and stderr there is a small program here. Compile as test-1+2 and use it in test runs.
Code example to redirect stdout to a file:
```
 
	fid = open(filename,O_WRONLY|O_CREAT|O_TRUNC);
	close(1);
	dup(fid);
	close(fid);
      
```
To redirect stdout back to the screen replace the open with:
```
 
	fid = open("/dev/tty",O_WRONLY);
      
```
and repeat the other 3 calls.
Additionally, add a noclobber command which will affect how these operators handle file creation. All it should do is change a variable, noclobber, from 0 to 1 or from 1 to 0 and print out the new value of it. This variable should default to 0 and cause your shell to act the same way as csh/tcsh does with respect to the noclobber variable. That is when it is 0 > and >& will overwrite existing files and >> and >>& will create the file if it doesn't exist. When noclobber is 1 the shell should refuse to overwrite existing files, refuse to create a file for appending and print the same error messages csh/tcsh do in those situations.

Add support for interprocess communication (IPC). This is adding support for | and |&. The | operator should pipe standard output of the command on the left to standard input of the command on the right. (i.e. command1 | command2). For |& standard error should be piped as well as standard output. It is only required that this works when the command on the right is an external command, the one on the left could be internal or external. You also need to avoid generating zombie processes. To implement this you will need to use pipe(2).
To implement pipes, file descriptor redirection will be required again by using close() and dup(). However you will be getting open file descriptors by using pipe(2) instead of open(). This task is a bit trickier than the previous, but again does not take much code. It will take some trials and thinking to get right. You will probably need to move your command running code to a new function which takes info about the style of the pipe (if any) and which side of the pipe this command will be on. The left side of the pipe needs to not cause the parent to do a blocking wait, while the right side does.
Once the parent process (your shell) has created a pipe and forked the two child processes that will read and write from the pipe, make sure the parent explicitly closes the file descriptors for its read and write access to the pipe. If you fail to do this, the child reading from the pipe will never terminate. This is because the child reading from the pipe will never get the END-OF-FILE (end-of-pipe) condition (and hence never terminate) as long as one process (in this case your shell program, by mistake) has an open write file descriptor for the pipe. The overall result will be a deadlock - your shell waiting for a child to terminate and the child waiting for the shell to close the pipe, which was inadvertently left open. (this paragraph is from http://www.cs.tamu.edu/faculty/pooch/course/CPSC613/Fall2000/graded_events/programs/prog01.htm, which seems to no longer exist.)
This part will also require that your command line processing puts the arguments into two separate argument vectors. And do not use popen(), you will get no credit.

As always document your code well and explain how things work. It is only necessary that one of &, the file redirections and piping work at a time for this project. Remember that man pages are your friends and you may find using truss(1) useful... Project 3 will be new and will not depend on project 1 or 2.

Test Runs

Test your shell by running the following commands in it (in order):

    pwd
    ls &
    ls -l &
    cd /
    sleep 20 &
    ls & 			; run before sleep is done
    pid
    tty
    /bin/ps -lfu USERNAME	; replace USERNAME with your own  
    cd
    cd [project test dir of your choosing]
    pwd
    ls -l
    rm -f mail1 mail2
    touch mail1			; create this file
    watchmail mail1
    echo hi >> mail1
    echo HiThere > mail2	; create another file
    watchmail mail2
    echo there >> mail1
    echo Subject: >> mail2
    cat mail1
    cat mail2
    watchmail mail1 off
    echo bye >> mail1
    echo bye >> mail2		; still watching this one

    rm -f test1 test2 test3 test4 test5 test6 test7 test8
    test-1+2 > test1
    test-1+2 >> test2
    test-1+2 >& test3
    test-1+2 >>& test4
    cat test1 test2 test3 test4
    test-1+2 > test1
    test-1+2 >> test2
    test-1+2 >& test3
    test-1+2 >>& test4
    cat test1 test2 test3 test4

    noclobber				; turn noclobber on
    test-1+2 > test5
    test-1+2 >> test6
    test-1+2 >& test7
    test-1+2 >>& test8
    cat test5 test6 test7 test8
    test-1+2 > test5
    test-1+2 >> test6
    test-1+2 >& test7
    test-1+2 >>& test8
    cat test5 test6 test7 test8

    grep hello < test7
    grep error < test7

    rm -f test9 test10 test11 test12
    noclobber				; turn noclobber off
    test-1+2 > test9
    test-1+2 >> test10
    test-1+2 >& test11
    test-1+2 >>& test12
    cat test9 test10 test11 test12

    ls | grep test                   ; show pipes works
    ./test-1+2 | grep  hello
    ./test-1+2 |& grep hello
    ./test-1+2 | grep output
    ./test-1+2 |& grep output
    ./test-1+2 |& grep error

    pid                                        ; zombie avoidance checking
    /bin/ps -lfu USERNAME | grep defunct       ; replace USERNAME with your username

If your file redirection doesn't work then use another shell to modify your mailfiles to test your watchmail command. The TA will also test each feature.

A sample of what the output should look like is here. Be sure that your code compiles and works on both Solaris and Linux.

Grading

10% for adding & support (including avoiding zombies)
20% for watchmail
25% file redirection support (including noclobber)
30% pipe support, and avoiding zombies
15% documentation and code structure (remember to check error situations and avoid too much duplicate code)

Turn In

Turn in all of your code, a script of the test runs (show doing a `make clean' followed by a make to build your shell) for your well documented program. Turn in one connected packet of paper with no blue sheets. Your name should be clearly visible on the first page you turn in and at the beginning of each source file.

Also, you need to tar up your source code and email it to the TA so that your shell can be tested some more. To do this do the following (assuming your code is in a directory named proj2):

cd ..../proj2
make clean
cd ..
tar cvf YourName_proj2.tar proj2

To verify that your files are in the tar file take a look at the table of contents of the tar file like:

tar tvf YourName_proj2.tar

Then upload the file to Sakai. Also turn in a copy of your test runs at the beginning of class on the due date for full credit. Remember that only code received/turned in can be graded. Late penalties will be given based on the timestamps of Sakai upload.