Mirroring/Copying the Unix Archive

Last update: May 28, 2001

  1. Introduction to the Unix Archive
  2. Mirroring the Unix Archive
  3. Distributing the Unix Archive on Media
  4. Making a CD Image of the Unix Archive
  5. Duplicating Unix Archive CDs
  6. Providing the Unix Archive On-line

Welcome to this document which describes how you can mirror the Unix Archive from minnie.tuhs.org. Once you have a copy of the archive, you might like to volunteer to distribute copies of the archive on CD-ROM or other media, or to set up an on-line mirror of the archive. If you do become such a Unix Archive Volunteer, Warren will add you to a volunteers mailing list so that we can discuss any problems or issues, suggest new approaches and keep each other informed of relevant events.

If you don't have the disk space to mirror the archive, but you do have a CD writer, then skip down to the section on duplicating Unix Archive CDs.


Introduction to the Unix Archive

The Unix Archive is a collection of files related to the UNIX system. Currently, the archive occupies around 1 Gigabyte of storage. The main areas of the Unix Archive are:


Mirroring the Unix Archive

There are two main ways of mirroring the 1 Gig of files in the Unix Archive: by rsync or by FTP. The rsync method is the preferred way, as it minimises data transfer.

The archive changes very slowly. If you are mirroring to provide an FTP mirror, you will only need to mirror the archive once a week or fortnight. If you are mirroring to distribute the archive by CD-ROM or other media, you will only need to mirror every few months.

Bootstrapping the Mirroring via Rsync

Mirroring the Unix Archive by rsync is pretty simple. First, obtain a recent copy of the rsync program from ftp://samba.anu.edu.au/pub/rsync. Any version from 2.4 up will be fine. Compile and install the program on your Unix system.

You can now use your existing Unix Archive username and password, plus rsync to mirror the Mirroring directory in the archive. Firstly, create a directory to hold the archive, on a disk with at least 1G of free disk space. Change into this directory and run the command:

rsync -avuz --exclude 'CD_IMAGE.*' minnie.tuhs.org::UA_Mirroring  Mirroring

Eventually you will have a synchronised copy of the Mirroring directory. Note: For now I am placing the two ISO CD images of the Unix Archive in this directory. If you can burn CDs, and are prepared to become a Unix Archive Volunteer, you might like to grab these images, and read the section on Distributing the Unix Archive on Media.

Completing the Mirroring via Rsync

Hint: if you have an existing copy of the Unix Archive on CD-ROMs, then you should copy it all into your Archive directory before you run rsync for the first time. This will greatly reduce the data transfer, saving time and money.

Once you have obtained a copy of the Mirroring directory, change into this directory. Copy the file .depend.sample to .depend, and:

From now on, you can mirror the full Unix Archive by going into Mirroring and doing make sync.

Mirroring via FTP

You can also mirror the Unix Archive by FTP, assuming you already have password-protected FTP access to the Unix Archive on minnie. Find a suitable FTP mirroring program, and use it to mirror the whole archive. Examples mirroring programs I know of are ftpmirror, jmirror, mirror, pavuk, and spegla.

There is one drawback with this approach: you'll have to copy whole files if they change; rsync only copies differences.


Distributing the Unix Archive on Media

If you are prepared to distribute the Unix Archive on media such as CD-ROM or tapes, and you have contacted Warren, then your geographic location (but not name) will be added to the web page describing how people can obtain archive copies.

If a person wishes to geta copy of the Archive, they can go to http://minnie.tuhs.org/cgi-bin/pups_req.cgi and order a copy of all or part of the Archive on some medium from some geographical area. The script will make sure that one of the volunteers can actually copy the requested media within that area.

When a person fills in an archive request on this web page for your geographic area, you will receive e-mail with the following information:

Once you are satisfied that the request is okay, you can then contact the person by e-mail. You should get further details on the request, tell them how much the request will cost them, and arrange a method of payment. Strictly speaking, you should work on a cost recovery basis, and not make a profit from the operation. However, you should charge for media, your own time, maintenance of equipment, postage and handling. The requester might get several offers, so don't do anything until they confirm that you are the chosen one.

Once you send off the request, then go to the web page named in the original e-mail, find the matching request, and tick it off, so it is removed from the list. Warren will scan this list periodically, and remind the mailing list if a request has been pending too long.

I have no idea how many requests you will receive. You might like to burn a number of CDs in one go, and then you can send them off as requests come in.


Making CD Images of the Unix Archive

The Unix Archive is much bigger than one CD-ROM, so the Makefile in the Mirroring directory can create two CD images, using mkisofs. If you make image1, you get the CD image with the PDP-11 area on it. If you make image2, you get the CD image with the VAX and 4BSD areas on it. Read through the Makefile to see what sections from the archive are excluded. Also read through and modify .depend, as this sets the location of the local Unix Archive and the output image.

To build the CD image, you will have to get a copy of the mkisofs program, compile it and install it on your system. This can be obtained as part of the cdrecord software. You'll also need a spare 650Megs of disk space to create each CD image. You can use any CD writing software & hardware to burn this Rock Ridge image onto a blank CD. It would be a good idea to verify the CD's contents after you do your first make image1.

As the Unix Archive changes and grows, I'll have to modify the Makefile in Mirroring. I'll let you know via the mailing list when this happens.


Mirroring the Two CD Images

As required, I will make the two CD images and place them in the Mirroring directory. If you want to mirror these images, please edit the .depend file and comment out the NOISO line.

Email Warren to get the images rebuilt, before you bother to mirror them.


Unix Archive CD Artwork

The files in Mirroring/Covers contains the artwork for the CD case, in both PostScript and Acrobat format. Each month, the date in the files is changed on minnie.tuhs.org, so this should reflect the date of your last archive synchronisation.


Duplicating Unix Archive CDs

If you don't have disk space to mirror the archive, but do have a CD writer, then you can still help out by buying the Unix Archive CDs from another volunteer. As requests for the CDs come in, you can duplicate your CDs and send them on to people, on a cost recovery basis as described above.

You should keep your Archive CDs up to date by obtaining new copies from the other volunteers when the archive contents change significantly. You may find that you can get the CDs for free, because you are helping us out!

If you decide to take this approach, e-mail Warren once you have obtained your CDs, and he will add you to the list of CD volunteers.


Providing the Unix Archive On-line

If you are prepared to make an on-line mirror of the archive, you'll be expected to have it on-line continuously. Archive users will be informed of your site and how to access it.

Make sure you mirror the archive weekly or fortnightly!

Finally...

If, after reading this, you still have questions, or have suggestions on how things can be improved, then please send them in to Warren or to the Archive Volunteers mailing list, so that we can help each other out and improve what we do by consensus.

And from me, many thanks for being a Unix Archive Volunteer.


Warren Toomey
2001-05-28