Users of the SDSS Data Archive Server (DAS) who wish to obtain more
than 10 files are required to use rsync, an open source
utility that provides fast incremental file transfer. Rsync is
available for download at the rsync website. Here we
provide detailed instructions for installation and use on a variety of
platforms.
The same instructions apply for mirroring all or parts of the data
archive server through the rsync URL rsync://dr3@rsync.sdss.org/DR3.
Unix and linux
Rsync is often installed by default on Unix and Linux systems. It
is included in Red Hat 7.2, for instance. If it is not installed on
your system, or you would like to upgrade to the latest version, you
can find links to the available
downloads. We recommend downloading and compiling the source,
rather than retrieving the binary.
The source can be downloaded directly from the rsync download area. You
will need to gunzip and untar the file if your browser does not do so
automatically. A directory called rsync-2.5.6 is created; just
cd into that directory, and execute
./configure
This will create a Makefile and config.h
appropriate for your system. Then type
make
to
compile the code. Finally, to put rsync in your path, either add it by
hand, or type
make install.
At this point, you are ready to use rsync. When you visit the DAS access page and
request data, you can choose rsync as your download method; if you
request more than 10 files this will be the default. When your data is
ready, you will be asked to download a file called
sdss-rsync.lis to a local directory where you want to save your
data. This file contains a listing of all the data files to be pulled
over by rsync. Then, change the working directory to where you want
the downloaded directory tree to be placed and execute
rsync
-vtrLPR --copy-unsafe-links --include-from=sdss-rsync.lis
rsync://dr3@rsync.sdss.org/DR3/ ./
to retrieve your
files. (If you use rsync for mirroring the archive, you can create
your own include file, or point rsync directly to a specific directory
if only one directory is to be downloaded.) You will be prompted for a
password, which is simply "dr3". If you will be downloading often, you
can set an environment variable of this password by adding
setenv RSYNC_PASSWORD dr3
to your .cshrc file, or
export RSYNC_PASSWORD=dr3
to your .bashrc
file.
NOTE:If you are behind a firewall, you will need to
setenv RSYNC_PROXY host:port
(or the equivalent
export in non-csh-like shells) to point to your
institution's proxy server and port number.
Windows
Windows users cannot download and run rsync directly. You will
first have to install cygwin, a linux emulator. It works with
all non-beta, non "release candidate", ix86 versions of Windows since
Windows 95, with the exception of Windows CE. Cygwin is available for
download here, free of charge.
Click on the "Install Now" link, and a file called setup.exe
will be downloaded. Running this will allow you to install
Cygwin.
Because rsync is not included with Cygwin, you
will also need to download and install rsync. Use your web browser to
download
the rsync source code. Open a Cygwin session, cd to the
location of the source, and follow the instructions in
the Unix section above to install rsync.
