Version Control with Subversion

All version control systems have to solve the same fundamental problem: how will the system allow users to share information, but prevent them from accidentally stepping on each other's feet? It's all too easy for users to accidentally overwrite each other's changes in the repository.

Consider the scenario shown in Figure 2.2, “The problem to avoid”. Suppose we have two co-workers, Harry and Sally. They each decide to edit the same repository file at the same time. If Harry saves his changes to the repository first, then it's possible that (a few moments later) Sally could accidentally overwrite them with her own new version of the file. While Harry's version of the file won't be lost forever (because the system remembers every change), any changes Harry made won't be present in Sally's newer version of the file, because she never saw Harry's changes to begin with. Harry's work is still effectively lost—or at least missing from the latest version of the file—and probably by accident. This is definitely a situation we want to avoid!

Figure 2.2. The problem to avoid

Many version control systems use a lock-modify-unlock model to address the problem of many authors clobbering each other's work. In this model, the repository allows only one person to change a file at a time. This exclusivity policy is managed using locks. Harry must “lock” a file before he can begin making changes to it. If Harry has locked a file, then Sally cannot also lock it, and therefore cannot make any changes to that file. All she can do is read the file, and wait for Harry to finish his changes and release his lock. After Harry unlocks the file, Sally can take her turn by locking and editing the file. Figure 2.3, “The lock-modify-unlock solution” demonstrates this simple solution.

Figure 2.3. The lock-modify-unlock solution

The problem with the lock-modify-unlock model is that it's a bit restrictive, and often becomes a roadblock for users:

Subversion, CVS, and other version control systems use a copy-modify-merge model as an alternative to locking. In this model, each user's client contacts the project repository and creates a personal working copy—a local reflection of the repository's files and directories. Users then work in parallel, modifying their private copies. Finally, the private copies are merged together into a new, final version. The version control system often assists with the merging, but ultimately a human being is responsible for making it happen correctly.

Here's an example. Say that Harry and Sally each create working copies of the same project, copied from the repository. They work concurrently, and make changes to the same file A within their copies. Sally saves her changes to the repository first. When Harry attempts to save his changes later, the repository informs him that his file A is out-of-date. In other words, that file A in the repository has somehow changed since he last copied it. So Harry asks his client to merge any new changes from the repository into his working copy of file A. Chances are that Sally's changes don't overlap with his own; so once he has both sets of changes integrated, he saves his working copy back to the repository. Figure 2.4, “The copy-modify-merge solution” and Figure 2.5, “The copy-modify-merge solution (continued)” show this process.

Figure 2.4. The copy-modify-merge solution

Figure 2.5. The copy-modify-merge solution (continued)

But what if Sally's changes do overlap with Harry's changes? What then? This situation is called a conflict, and it's usually not much of a problem. When Harry asks his client to merge the latest repository changes into his working copy, his copy of file A is somehow flagged as being in a state of conflict: he'll be able to see both sets of conflicting changes, and manually choose between them. Note that software can't automatically resolve conflicts; only humans are capable of understanding and making the necessary intelligent choices. Once Harry has manually resolved the overlapping changes—perhaps after a discussion with Sally—he can safely save the merged file back to the repository.

The copy-modify-merge model may sound a bit chaotic, but in practice, it runs extremely smoothly. Users can work in parallel, never waiting for one another. When they work on the same files, it turns out that most of their concurrent changes don't overlap at all; conflicts are infrequent. And the amount of time it takes to resolve conflicts is far less than the time lost by a locking system.

In the end, it all comes down to one critical factor: user communication. When users communicate poorly, both syntactic and semantic conflicts increase. No system can force users to communicate perfectly, and no system can detect semantic conflicts. So there's no point in being lulled into a false promise that a locking system will somehow prevent conflicts; in practice, locking seems to inhibit productivity more than anything else.

You've already read about working copies; now we'll demonstrate how the Subversion client creates and uses them.

A Subversion working copy is an ordinary directory tree on your local system, containing a collection of files. You can edit these files however you wish, and if they're source code files, you can compile your program from them in the usual way. Your working copy is your own private work area: Subversion will never incorporate other people's changes, nor make your own changes available to others, until you explicitly tell it to do so. You can even have multiple working copies of the same project.

After you've made some changes to the files in your working copy and verified that they work properly, Subversion provides you with commands to “publish” your changes to the other people working with you on your project (by writing to the repository). If other people publish their own changes, Subversion provides you with commands to merge those changes into your working directory (by reading from the repository).

A working copy also contains some extra files, created and maintained by Subversion, to help it carry out these commands. In particular, each directory in your working copy contains a subdirectory named .svn, also known as the working copy administrative directory. The files in each administrative directory help Subversion recognize which files contain unpublished changes, and which files are out-of-date with respect to others' work.

A typical Subversion repository often holds the files (or source code) for several projects; usually, each project is a subdirectory in the repository's filesystem tree. In this arrangement, a user's working copy will usually correspond to a particular subtree of the repository.

For example, suppose you have a repository that contains two software projects, paint and calc. Each project lives in its own top-level subdirectory, as shown in Figure 2.6, “The repository's filesystem”.

Figure 2.6. The repository's filesystem

To get a working copy, you must check out some subtree of the repository. (The term “check out” may sound like it has something to do with locking or reserving resources, but it doesn't; it simply creates a private copy of the project for you.) For example, if you check out /calc, you will get a working copy like this:

$ svn checkout http://svn.example.com/repos/calc
A    calc/Makefile
A    calc/integer.c
A    calc/button.c
Checked out revision 56.

$ ls -A calc
Makefile  integer.c  button.c  .svn/

The list of letter A's indicates that Subversion is adding a number of items to your working copy. You now have a personal copy of the repository's /calc directory, with one additional entry—.svn—which holds the extra information needed by Subversion, as mentioned earlier.

Suppose you make changes to button.c. Since the .svn directory remembers the file's modification date and original contents, Subversion can tell that you've changed the file. However, Subversion does not make your changes public until you explicitly tell it to. The act of publishing your changes is more commonly known as committing (or checking in) changes to the repository.

To publish your changes to others, you can use Subversion's commit command:

$ svn commit button.c
Sending        button.c
Transmitting file data .
Committed revision 57.

Now your changes to button.c have been committed to the repository; if another user checks out a working copy of /calc, they will see your changes in the latest version of the file.

Suppose you have a collaborator, Sally, who checked out a working copy of /calc at the same time you did. When you commit your change to button.c, Sally's working copy is left unchanged; Subversion only modifies working copies at the user's request.

To bring her project up to date, Sally can ask Subversion to update her working copy, by using the Subversion update command. This will incorporate your changes into her working copy, as well as any others that have been committed since she checked it out.

$ pwd
/home/sally/calc

$ ls -A 
.svn/ Makefile integer.c button.c

$ svn update
U    button.c
Updated to revision 57.

The output from the svn update command indicates that Subversion updated the contents of button.c. Note that Sally didn't need to specify which files to update; Subversion uses the information in the .svn directory, and further information in the repository, to decide which files need to be brought up to date.

An svn commit operation can publish changes to any number of files and directories as a single atomic transaction. In your working copy, you can change files' contents, create, delete, rename and copy files and directories, and then commit the complete set of changes as a unit.

In the repository, each commit is treated as an atomic transaction: either all the commit's changes take place, or none of them take place. Subversion tries to retain this atomicity in the face of program crashes, system crashes, network problems, and other users' actions.

Each time the repository accepts a commit, this creates a new state of the filesystem tree, called a revision. Each revision is assigned a unique natural number, one greater than the number of the previous revision. The initial revision of a freshly created repository is numbered zero, and consists of nothing but an empty root directory.

Figure 2.7, “The repository” illustrates a nice way to visualize the repository. Imagine an array of revision numbers, starting at 0, stretching from left to right. Each revision number has a filesystem tree hanging below it, and each tree is a “snapshot” of the way the repository looked after a commit.

Figure 2.7. The repository

It's important to note that working copies do not always correspond to any single revision in the repository; they may contain files from several different revisions. For example, suppose you check out a working copy from a repository whose most recent revision is 4:

calc/Makefile:4
     integer.c:4
     button.c:4

At the moment, this working directory corresponds exactly to revision 4 in the repository. However, suppose you make a change to button.c, and commit that change. Assuming no other commits have taken place, your commit will create revision 5 of the repository, and your working copy will now look like this:

calc/Makefile:4
     integer.c:4
     button.c:5

Suppose that, at this point, Sally commits a change to integer.c, creating revision 6. If you use svn update to bring your working copy up to date, then it will look like this:

calc/Makefile:6
     integer.c:6
     button.c:6

Sally's change to integer.c will appear in your working copy, and your change will still be present in button.c. In this example, the text of Makefile is identical in revisions 4, 5, and 6, but Subversion will mark your working copy of Makefile with revision 6 to indicate that it is still current. So, after you do a clean update at the top of your working copy, it will generally correspond to exactly one revision in the repository.

For each file in a working directory, Subversion records two essential pieces of information in the .svn/ administrative area:

Given this information, by talking to the repository, Subversion can tell which of the following four states a working file is in:

This may sound like a lot to keep track of, but the svn status command will show you the state of any item in your working copy. For more information on that command, see the section called “svn status”.

As a general principle, Subversion tries to be as flexible as possible. One special kind of flexibility is the ability to have a working copy containing files and directories with a mix of different working revision numbers. Unfortunately, this flexibility tends to confuse a number of new users. If the earlier example showing mixed revisions perplexed you, here's a primer on both why the feature exists and how to make use of it.

When you create a new Subversion repository, it begins its life at revision zero and each successive commit increases the revision number by one. After your commit completes, the Subversion client informs you of the new revision number:

$ svn commit --message "Corrected number of cheese slices."
Sending        sandwich.txt
Transmitting file data .
Committed revision 3.

If at any point in the future you want to refer to that revision (we'll see how and why we might want to do that later in this chapter), you can refer to it as “3”.

The Subversion client understands a number of revision keywords. These keywords can be used instead of integer arguments to the --revision switch, and are resolved into specific revision numbers by Subversion:

Here are some examples of revision keywords in action. Don't worry if the commands don't make sense yet; we'll be explaining these commands as we go through the chapter:

$ svn diff --revision PREV:COMMITTED foo.c
# shows the last change committed to foo.c

$ svn log --revision HEAD
# shows log message for the latest repository commit

$ svn diff --revision HEAD
# compares your working file (with local changes) to the latest version
# in the repository

$ svn diff --revision BASE:HEAD foo.c
# compares your “pristine” foo.c (no local changes) with the 
# latest version in the repository

$ svn log --revision BASE:HEAD
# shows all commit logs since you last updated

$ svn update --revision PREV foo.c
# rewinds the last change on foo.c
# (foo.c's working revision is decreased)

These keywords allow you to perform many common (and helpful) operations without having to look up specific revision numbers or remember the exact revision of your working copy.

Anywhere that you specify a revision number or revision keyword, you can also specify a date inside curly braces “{}”. You can even access a range of changes in the repository using both dates and revisions together!

Here are examples of the date formats that Subversion accepts. Remember to use quotes around any date that contains spaces.

$ svn checkout --revision {2002-02-17}
$ svn checkout --revision {15:30}
$ svn checkout --revision {15:30:00.200000}
$ svn checkout --revision {"2002-02-17 15:30"}
$ svn checkout --revision {"2002-02-17 15:30 +0230"}
$ svn checkout --revision {2002-02-17T15:30}
$ svn checkout --revision {2002-02-17T15:30Z}
$ svn checkout --revision {2002-02-17T15:30-04:00}
$ svn checkout --revision {20020217T1530}
$ svn checkout --revision {20020217T1530Z}
$ svn checkout --revision {20020217T1530-0500}
…

When you specify a date as a revision, Subversion finds the most recent revision of the repository as of that date:

$ svn log --revision {2002-11-28}
------------------------------------------------------------------------
r12 | ira | 2002-11-27 12:31:51 -0600 (Wed, 27 Nov 2002) | 6 lines
…

You can also use a range of dates. Subversion will find all revisions between both dates, inclusive:

$ svn log --revision {2002-11-20}:{2002-11-29}
…

As we pointed out, you can also mix dates and revisions:

$ svn log --revision {2002-11-20}:4040

Users should be aware of a subtlety that can become quite a stumbling-block when dealing with dates in Subversion. Since the timestamp of a revision is stored as a property of the revision—an unversioned, modifiable property—revision timestamps can be changed to represent complete falsifications of true chronology, or even removed altogether. This will wreak havoc on the internal date-to-revision conversion that Subversion performs.

When working on a project with a team, you'll want to update your working copy to receive any changes made since your last update by other developers on the project. Use svn update to bring your working copy into sync with the latest revision in the repository.

$ svn update
U  foo.c
U  bar.c
Updated to revision 2.

In this case, someone else checked in modifications to both foo.c and bar.c since the last time you updated, and Subversion has updated your working copy to include those changes.

Let's examine the output of svn update a bit more. When the server sends changes to your working copy, a letter code is displayed next to each item to let you know what actions Subversion performed to bring your working copy up-to-date:

Now you can get to work and make changes in your working copy. It's usually most convenient to decide on a particular change (or set of changes) to make, such as writing a new feature, fixing a bug, etc. The Subversion commands that you will use here are svn add, svn delete, svn copy, and svn move. However, if you are merely editing files that are already in Subversion, you may not need to use any of these commands until you commit. Changes you can make to your working copy:

To make file changes, use your text editor, word processor, graphics program, or whatever tool you would normally use. Subversion handles binary files just as easily as it handles text files—and just as efficiently too.

Here is an overview of the four Subversion subcommands that you'll use most often to make tree changes (we'll cover svn import and svn mkdir later).

Once you've finished making changes, you need to commit them to the repository, but before you do so, it's usually a good idea to take a look at exactly what you've changed. By examining your changes before you commit, you can make a more accurate log message. You may also discover that you've inadvertently changed a file, and this gives you a chance to revert those changes before committing. Additionally, this is a good opportunity to review and scrutinize changes before publishing them. You can see exactly what changes you've made by using svn status, svn diff, and svn revert. You will usually use the first two commands to find out what files have changed in your working copy, and then perhaps the third to revert some (or all) of those changes.

Subversion has been optimized to help you with this task, and is able to do many things without communicating with the repository. In particular, your working copy contains a secret cached “pristine” copy of each version controlled file within the .svn area. Because of this, Subversion can quickly show you how your working files have changed, or even allow you to undo your changes without contacting the repository.

  L     some_dir            # svn left a lock in the .svn area of some_dir
M       bar.c               # the content in bar.c has local modifications
 M      baz.c               # baz.c has property but no content modifications
X       3rd_party           # dir is part of an externals definition
?       foo.o               # svn doesn't manage foo.o
!       some_dir            # svn manages this, but it's missing or incomplete
~       qux                 # versioned as file/dir/link, but type has changed
I       .screenrc           # svn doesn't manage this, and is set to ignore it
A  +    moved_dir           # added with history of where it came from
M  +    moved_dir/README    # added with history and has local modifications
D       stuff/fish.c        # file is scheduled for deletion
A       stuff/loot/bloo.h   # file is scheduled for addition
C       stuff/loot/lump.c   # file has textual conflicts from an update
 C      stuff/loot/glub.c   # file has property conflicts from an update
R       xyz.c               # file is scheduled for replacement
    S   stuff/squawk        # file or dir has been switched to a branch
     K  dog.jpg             # file is locked locally; lock-token present 
     O  cat.jpg             # file is locked in the repository by other user
     B  bird.jpg            # file is locked locally, but lock has been broken
     T  fish.jpg            # file is locked locally, but lock has been stolen

$ svn status stuff/fish.c
D      stuff/fish.c

$ svn status --verbose
M               44        23    sally     README
                44        30    sally     INSTALL
M               44        20    harry     bar.c
                44        18    ira       stuff
                44        35    harry     stuff/trout.c
D               44        19    ira       stuff/fish.c
                44        21    sally     stuff/things
A                0         ?     ?        stuff/things/bloo.h
                44        36    harry     stuff/things/gloo.c

$ svn status --show-updates --verbose
M      *        44        23    sally     README
M               44        20    harry     bar.c
       *        44        35    harry     stuff/trout.c
D               44        19    ira       stuff/fish.c
A                0         ?     ?        stuff/things/bloo.h
Status against revision:   46

$ svn diff
Index: bar.c
===================================================================
--- bar.c	(revision 3)
+++ bar.c	(working copy)
@@ -1,7 +1,12 @@
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include <stdio.h>

 int main(void) {
-  printf("Sixty-four slices of American Cheese...\n");
+  printf("Sixty-five slices of American Cheese...\n");
 return 0;
 }

Index: README
===================================================================
--- README	(revision 3)
+++ README	(working copy)
@@ -193,3 +193,4 @@ 
+Note to self:  pick up laundry.

Index: stuff/fish.c
===================================================================
--- stuff/fish.c	(revision 1)
+++ stuff/fish.c	(working copy)
-Welcome to the file known as 'fish'.
-Information on fish will be here soon.

Index: stuff/things/bloo.h
===================================================================
--- stuff/things/bloo.h	(revision 8)
+++ stuff/things/bloo.h	(working copy)
+Here is a new file to describe
+things about bloo.

$ svn diff > patchfile

$ svn revert README
Reverted 'README'

$ svn status foo
?      foo

$ svn add foo
A         foo

$ svn revert foo
Reverted 'foo'

$ svn status foo
?      foo

$ svn status README 
       README

$ svn delete README 
D         README

$ svn revert README
Reverted 'README'

$ svn status README
       README

We've already seen how svn status -u can predict conflicts. Suppose you run svn update and some interesting things occur:

$ svn update
U  INSTALL
G  README
C  bar.c
Updated to revision 46.

The U and G codes are no cause for concern; those files cleanly absorbed changes from the repository. The files marked with U contained no local changes but were Updated with changes from the repository. The G stands for merGed, which means that the file had local changes to begin with, but the changes coming from the repository didn't overlap with the local changes.

But the C stands for conflict. This means that the changes from the server overlapped with your own, and now you have to manually choose between them.

Whenever a conflict occurs, three things typically occur to assist you in noticing and resolving that conflict:

For example, Sally makes changes to the file sandwich.txt in the repository. Harry has just changed the file in his working copy and checked it in. Sally updates her working copy before checking in and she gets a conflict:

$ svn update
C  sandwich.txt
Updated to revision 2.
$ ls -1
sandwich.txt
sandwich.txt.mine
sandwich.txt.r1
sandwich.txt.r2

At this point, Subversion will not allow you to commit the file sandwich.txt until the three temporary files are removed.

$ svn commit --message "Add a few more things"
svn: Commit failed (details follow):
svn: Aborting commit: '/home/sally/svn-work/sandwich.txt' remains in conflict

If you get a conflict, you need to do one of three things:

Once you've resolved the conflict, you need to let Subversion know by running svn resolved. This removes the three temporary files and Subversion no longer considers the file to be in a state of conflict.^[5]

$ svn resolved sandwich.txt
Resolved conflicted state of 'sandwich.txt'

$ cat sandwich.txt
Top piece of bread
Mayonnaise
Lettuce
Tomato
Provolone
<<<<<<< .mine
Salami
Mortadella
Prosciutto
=======
Sauerkraut
Grilled Chicken
>>>>>>> .r2
Creole Mustard
Bottom piece of bread

<<<<<<< .mine
Salami
Mortadella
Prosciutto
=======

=======
Sauerkraut
Grilled Chicken
>>>>>>> .r2

Top piece of bread
Mayonnaise
Lettuce
Tomato
Provolone
Salami
Mortadella
Prosciutto
Creole Mustard
Bottom piece of bread

$ svn resolved sandwich.txt
$ svn commit -m "Go ahead and use my sandwich, discarding Sally's edits."

$ svn update
C  sandwich.txt
Updated to revision 2.
$ ls sandwich.*
sandwich.txt  sandwich.txt.mine  sandwich.txt.r2  sandwich.txt.r1
$ cp sandwich.txt.r2 sandwich.txt
$ svn resolved sandwich.txt

$ svn revert sandwich.txt
Reverted 'sandwich.txt'
$ ls sandwich.*
sandwich.txt

Now you're ready to check in your changes. Note that svn resolved, unlike most of the other commands we've dealt with in this chapter, requires an argument. In any case, you want to be careful and only run svn resolved when you're certain that you've fixed the conflict in your file—once the temporary files are removed, Subversion will let you commit the file even if it still contains conflict markers.

Finally! Your edits are finished, you've merged all changes from the server, and you're ready to commit your changes to the repository.

The svn commit command sends all of your changes to the repository. When you commit a change, you need to supply a log message, describing your change. Your log message will be attached to the new revision you create. If your log message is brief, you may wish to supply it on the command line using the --message (or -m) option:

$ svn commit --message "Corrected number of cheese slices."
Sending        sandwich.txt
Transmitting file data .
Committed revision 3.

However, if you've been composing your log message as you work, you may want to tell Subversion to get the message from a file by passing the filename with the --file switch:

$ svn commit --file logmsg 
Sending        sandwich.txt
Transmitting file data .
Committed revision 4.

If you fail to specify either the --message or --file switch, then Subversion will automatically launch your favorite editor (see the editor-cmd section in the section called “Config”) for composing a log message.

$ svn commit
Waiting for Emacs...Done

Log message unchanged or not specified
a)bort, c)ontinue, e)dit
a
$

The repository doesn't know or care if your changes make any sense as a whole; it only checks to make sure that nobody else has changed any of the same files that you did when you weren't looking. If somebody has done that, the entire commit will fail with a message informing you that one or more of your files is out-of-date:

$ svn commit --message "Add another rule"
Sending        rules.txt
svn: Commit failed (details follow):
svn: Out of date: 'rules.txt' in transaction 'g'

At this point, you need to run svn update, deal with any merges or conflicts that result, and attempt your commit again.

That covers the basic work cycle for using Subversion. There are many other features in Subversion that you can use to manage your repository and working copy, but you can get by quite easily using only the commands that we've discussed so far in this chapter.

To find information about the history of a file or directory, use the svn log command. svn log will provide you with a record of who made changes to a file or directory, at what revision it changed, the time and date of that revision, and, if it was provided, the log message that accompanied the commit.

$ svn log
------------------------------------------------------------------------
r3 | sally | Mon, 15 Jul 2002 18:03:46 -0500 | 1 line

Added include lines and corrected # of cheese slices.
------------------------------------------------------------------------
r2 | harry | Mon, 15 Jul 2002 17:47:57 -0500 | 1 line

Added main() methods.
------------------------------------------------------------------------
r1 | sally | Mon, 15 Jul 2002 17:40:08 -0500 | 1 line

Initial import
------------------------------------------------------------------------

Note that the log messages are printed in reverse chronological order by default. If you wish to see a different range of revisions in a particular order, or just a single revision, pass the --revision (-r) switch:

$ svn log --revision 5:19    # shows logs 5 through 19 in chronological order

$ svn log -r 19:5            # shows logs 5 through 19 in reverse order

$ svn log -r 8               # shows log for revision 8

You can also examine the log history of a single file or directory. For example:

$ svn log foo.c
…
$ svn log http://foo.com/svn/trunk/code/foo.c
…

These will display log messages only for those revisions in which the working file (or URL) changed.

If you want even more information about a file or directory, svn log also takes a --verbose (-v) switch. Because Subversion allows you to move and copy files and directories, it is important to be able to track path changes in the filesystem, so in verbose mode, svn log will include a list of changed paths in a revision in its output:

$ svn log -r 8 -v
------------------------------------------------------------------------
r8 | sally | 2002-07-14 08:15:29 -0500 | 1 line
Changed paths:
M /trunk/code/foo.c
M /trunk/code/bar.h
A /trunk/code/doc/README

Frozzled the sub-space winch.

------------------------------------------------------------------------

svn log also takes a --quiet (-q) switch, which suppresses the body of the log message. When combined with --verbose, it gives just the names of the changed files.

$ svn log -r 2
------------------------------------------------------------------------
$

We've already seen svn diff before—it displays file differences in unified diff format; it was used to show the local modifications made to our working copy before committing to the repository.

In fact, it turns out that there are three distinct uses of svn diff:

$ svn diff
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth open
+Chew with your mouth closed
+Listen when others are speaking
$

$ svn diff --revision 3 rules.txt 
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth open
+Chew with your mouth closed
+Listen when others are speaking
$

$ svn diff --revision 2:3 rules.txt 
Index: rules.txt
===================================================================
--- rules.txt	(revision 2)
+++ rules.txt	(revision 3)
@@ -1,4 +1,4 @@
 Be kind to others
-Freedom = Chocolate Ice Cream
+Freedom = Responsibility
 Everything in moderation
 Chew with your mouth open
$

$ svn diff --revision 4:5 http://svn.red-bean.com/repos/example/trunk/text/rules.txt
…
$

If you want to examine an earlier version of a file and not necessarily the differences between two files, you can use svn cat:

$ svn cat --revision 2 rules.txt 
Be kind to others
Freedom = Chocolate Ice Cream
Everything in moderation
Chew with your mouth open
$

You can also redirect the output directly into a file:

$ svn cat --revision 2 rules.txt > rules.txt.v2
$

You're probably wondering why we don't just use svn update --revision to update the file to the older revision. There are a few reasons why we might prefer to use svn cat.

First, you may want to see the differences between two revisions of a file using an external diff program (perhaps a graphical one, or perhaps your file is in such a format that the output of unified diff is nonsensical). In this case, you'll need to grab a copy of the old revision, redirect it to a file, and pass both that and the file in your working copy to your external diff program.

Sometimes it's easier to look at an older version of a file in its entirety as opposed to just the differences between it and another revision.

The svn list command shows you what files are in a repository directory without actually downloading the files to your local machine:

$ svn list http://svn.collab.net/repos/svn
README
branches/
clients/
tags/
trunk/

If you want a more detailed listing, pass the --verbose (-v) flag to get output like this:

$ svn list --verbose http://svn.collab.net/repos/svn
   2755 harry          1331 Jul 28 02:07 README
   2773 sally               Jul 29 15:07 branches/
   2769 sally               Jul 29 12:07 clients/
   2698 harry               Jul 24 18:07 tags/
   2785 sally               Jul 29 19:07 trunk/

The columns tell you the revision at which the file or directory was last modified, the user who modified it, the size if it is a file, the date it was last modified, and the item's name.

In addition to all of the above commands, you can use svn update and svn checkout with the --revision switch to take an entire working copy “back in time” ^[7]:

$ svn checkout --revision 1729 # Checks out a new working copy at r1729
…
$ svn update --revision 1729 # Updates an existing working copy to r1729
…

When Subversion modifies your working copy (or any information within .svn), it tries to do so as safely as possible. Before changing the working copy, Subversion writes its intentions to a log file. Next it executes the commands in the log file to apply the requested change, holding a lock on the relevant part of the working copy while it works — to prevent othe Subversion clients from accessing the working copy in mid-change. Finally, Subversion removes the log file. Architecturally, this is similar to a journaled filesystem. If a Subversion operation is interrupted (if the process is killed, or if the machine crashes, for example), the log files remain on disk. By re-executing the log files, Subversion can complete the previously started operation, and your working copy can get itself back into a consistent state.

And this is exactly what svn cleanup does: it searches your working copy and runs any leftover logs, removing working copy locks in the process. If Subversion ever tells you that some part of your working copy is “locked”, then this is the command that you should run. Also, svn status will display an L next to locked items:

$ svn status
  L    somedir
M      somedir/foo.c 

$ svn cleanup
$ svn status
M      somedir/foo.c

Don't confuse these working copy locks with the ordinary locks that Subversion users create when using the “lock-modify-unlock” model of concurrent version control; see Three meanings of “lock” for clarification.

The svn import command is a quick way to copy an unversioned tree of files into a repository, creating intermediate directories as necessary.

$ svnadmin create /usr/local/svn/newrepos
$ svn import mytree file:///usr/local/svn/newrepos/some/project \
             -m "Initial import"
Adding         mytree/foo.c
Adding         mytree/bar.c
Adding         mytree/subdir
Adding         mytree/subdir/quux.h

Committed revision 1.

The previous example copied the contents of directory mytree under the directory some/project in the repository:

$ svn list file:///usr/local/svn/newrepos/some/project
bar.c
foo.c
subdir/

Note that after the import is finished, the original tree is not converted into a working copy. To start working, you still need to svn checkout a fresh working copy of the tree.

Creating a branch is very simple—you make a copy of the project in the repository using the svn copy command. Subversion is not only able to copy single files, but whole directories as well. In this case, you want to make a copy of the /calc/trunk directory. Where should the new copy live? Wherever you wish—it's a matter of project policy. Let's say that your team has a policy of creating branches in the /calc/branches area of the repository, and you want to name your branch my-calc-branch. You'll want to create a new directory, /calc/branches/my-calc-branch, which begins its life as a copy of /calc/trunk.

There are two different ways to make a copy. We'll demonstrate the messy way first, just to make the concept clear. To begin, check out a working copy of the project's root directory, /calc:

$ svn checkout http://svn.example.com/repos/calc bigwc
A  bigwc/trunk/
A  bigwc/trunk/Makefile
A  bigwc/trunk/integer.c
A  bigwc/trunk/button.c
A  bigwc/branches/
Checked out revision 340.

Making a copy is now simply a matter of passing two working-copy paths to the svn copy command:

$ cd bigwc
$ svn copy trunk branches/my-calc-branch
$ svn status
A  +   branches/my-calc-branch

In this case, the svn copy command recursively copies the trunk working directory to a new working directory, branches/my-calc-branch. As you can see from the svn status command, the new directory is now scheduled for addition to the repository. But also notice the “+” sign next to the letter A. This indicates that the scheduled addition is a copy of something, not something new. When you commit your changes, Subversion will create /calc/branches/my-calc-branch in the repository by copying /calc/trunk, rather than resending all of the working copy data over the network:

$ svn commit -m "Creating a private branch of /calc/trunk."
Adding         branches/my-calc-branch
Committed revision 341.

And now the easier method of creating a branch, which we should have told you about in the first place: svn copy is able to operate directly on two URLs.

$ svn copy http://svn.example.com/repos/calc/trunk \
           http://svn.example.com/repos/calc/branches/my-calc-branch \
      -m "Creating a private branch of /calc/trunk."

Committed revision 341.

There's really no difference between these two methods. Both procedures create a new directory in revision 341, and the new directory is a copy of /calc/trunk. This is shown in Figure 4.3, “Repository with new copy”. Notice that the second method, however, performs an immediate commit. ^[8] It's an easier procedure, because it doesn't require you to check out a large mirror of the repository. In fact, this technique doesn't even require you to have a working copy at all.

Figure 4.3. Repository with new copy

Now that you've created a branch of the project, you can check out a new working copy to start using it:

$ svn checkout http://svn.example.com/repos/calc/branches/my-calc-branch
A  my-calc-branch/Makefile
A  my-calc-branch/integer.c
A  my-calc-branch/button.c
Checked out revision 341.

There's nothing special about this working copy; it simply mirrors a different directory in the repository. When you commit changes, however, Sally won't ever see them when she updates. Her working copy is of /calc/trunk. (Be sure to read the section called “Switching a Working Copy” later in this chapter: the svn switch command is an alternate way of creating a working copy of a branch.)

Let's pretend that a week goes by, and the following commits happen:

There are now two independent lines of development, shown in Figure 4.4, “The branching of one file's history”, happening on integer.c.

Figure 4.4. The branching of one file's history

Things get interesting when you look at the history of changes made to your copy of integer.c:

$ pwd
/home/user/my-calc-branch

$ svn log --verbose integer.c
------------------------------------------------------------------------
r343 | user | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines
Changed paths:
   M /calc/branches/my-calc-branch/integer.c

* integer.c:  frozzled the wazjub.

------------------------------------------------------------------------
r341 | user | 2002-11-03 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines
Changed paths:
   A /calc/branches/my-calc-branch (from /calc/trunk:340)

Creating a private branch of /calc/trunk.

------------------------------------------------------------------------
r303 | sally | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  changed a docstring.

------------------------------------------------------------------------
r98 | sally | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  adding this file to the project.

------------------------------------------------------------------------

Notice that Subversion is tracing the history of your branch's integer.c all the way back through time, even traversing the point where it was copied. It shows the creation of the branch as an event in the history, because integer.c was implicitly copied when all of /calc/trunk/ was copied. Now look what happens when Sally runs the same command on her copy of the file:

$ pwd
/home/sally/calc

$ svn log --verbose integer.c
------------------------------------------------------------------------
r344 | sally | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  fix a bunch of spelling errors.

------------------------------------------------------------------------
r303 | sally | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  changed a docstring.

------------------------------------------------------------------------
r98 | sally | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  adding this file to the project.

------------------------------------------------------------------------

Sally sees her own revision 344 change, but not the change you made in revision 343. As far as Subversion is concerned, these two commits affected different files in different repository locations. However, Subversion does show that the two files share a common history. Before the branch-copy was made in revision 341, they used to be the same file. That's why you and Sally both see the changes made in revisions 303 and 98.

There are two important lessons that you should remember from this section.

In the previous section, we mentioned that both you and Sally made changes to integer.c on different branches. If you look at Sally's log message for revision 344, you can see that she fixed some spelling errors. No doubt, your copy of the same file still has the same spelling errors. It's likely that your future changes to this file will be affecting the same areas that have the spelling errors, so you're in for some potential conflicts when you merge your branch someday. It's better, then, to receive Sally's change now, before you start working too heavily in the same places.

It's time to use the svn merge command. This command, it turns out, is a very close cousin to the svn diff command (which you read about in Chapter 3). Both commands are able to compare any two objects in the repository and describe the differences. For example, you can ask svn diff to show you the exact change made by Sally in revision 344:

$ svn diff -r 343:344 http://svn.example.com/repos/calc/trunk

Index: integer.c
===================================================================
--- integer.c	(revision 343)
+++ integer.c	(revision 344)
@@ -147,7 +147,7 @@
     case 6:  sprintf(info->operating_system, "HPFS (OS/2 or NT)"); break;
     case 7:  sprintf(info->operating_system, "Macintosh"); break;
     case 8:  sprintf(info->operating_system, "Z-System"); break;
-    case 9:  sprintf(info->operating_system, "CPM"); break;
+    case 9:  sprintf(info->operating_system, "CP/M"); break;
     case 10:  sprintf(info->operating_system, "TOPS-20"); break;
     case 11:  sprintf(info->operating_system, "NTFS (Windows NT)"); break;
     case 12:  sprintf(info->operating_system, "QDOS"); break;
@@ -164,7 +164,7 @@
     low = (unsigned short) read_byte(gzfile);  /* read LSB */
     high = (unsigned short) read_byte(gzfile); /* read MSB */
     high = high << 8;  /* interpret MSB correctly */
-    total = low + high; /* add them togethe for correct total */
+    total = low + high; /* add them together for correct total */
 
     info->extra_header = (unsigned char *) my_malloc(total);
     fread(info->extra_header, total, 1, gzfile);
@@ -241,7 +241,7 @@
      Store the offset with ftell() ! */
 
   if ((info->data_offset = ftell(gzfile))== -1) {
-    printf("error: ftell() retturned -1.\n");
+    printf("error: ftell() returned -1.\n");
     exit(1);
   }
 
@@ -249,7 +249,7 @@
   printf("I believe start of compressed data is %u\n", info->data_offset);
   #endif
   
-  /* Set postion eight bytes from the end of the file. */
+  /* Set position eight bytes from the end of the file. */
 
   if (fseek(gzfile, -8, SEEK_END)) {
     printf("error: fseek() returned non-zero\n");

The svn merge command is almost exactly the same. Instead of printing the differences to your terminal, however, it applies them directly to your working copy as local modifications:

$ svn merge -r 343:344 http://svn.example.com/repos/calc/trunk
U  integer.c

$ svn status
M  integer.c

The output of svn merge shows that your copy of integer.c was patched. It now contains Sally's change—the change has been “copied” from the trunk to your working copy of your private branch, and now exists as a local modification. At this point, it's up to you to review the local modification and make sure it works correctly.

In another scenario, it's possible that things may not have gone so well, and that integer.c may have entered a conflicted state. You might need to resolve the conflict using standard procedures (see Chapter 3), or if you decide that the merge was a bad idea altogether, simply give up and svn revert the local change.

But assuming that you've reviewed the merged change, you can svn commit the change as usual. At that point, the change has been merged into your repository branch. In version control terminology, this act of copying changes between branches is commonly called porting changes.

When you commit the local modification, make sure your log message mentions that you're porting a specific change from one branch to another. For example:

$ svn commit -m "integer.c: ported r344 (spelling fixes) from trunk."
Sending        integer.c
Transmitting file data .
Committed revision 360.

As you'll see in the next sections, this is a very important “best practice” to follow.

$ svn diff -r 343:344 http://svn.example.com/repos/calc/trunk > patchfile
$ patch -p0  < patchfile
Patching file integer.c using Plan A...
Hunk #1 succeeded at 147.
Hunk #2 succeeded at 164.
Hunk #3 succeeded at 241.
Hunk #4 succeeded at 249.
done

A word of warning: while svn diff and svn merge are very similar in concept, they do have different syntax in many cases. Be sure to read about them in Chapter 9 for details, or ask svn help. For example, svn merge requires a working-copy path as a target, i.e. a place where it should apply the tree-changes. If the target isn't specified, it assumes you are trying to perform one of the following common operations:

If you are merging a directory and haven't specified a target path, svn merge assumes the first case above and tries to apply the changes into your current directory. If you are merging a file, and that file (or a file by the same name) exists in your current working directory, svn merge assumes the second case and tries to apply the changes to a local file with the same name.

If you want changes applied somewhere else, you'll need to say so. For example, if you're sitting in the parent directory of your working copy, you'll have to specify the target directory to receive the changes:

$ svn merge -r 343:344 http://svn.example.com/repos/calc/trunk my-calc-branch
U   my-calc-branch/integer.c

You've now seen an example of the svn merge command, and you're about to see several more. If you're feeling confused about exactly how merging works, you're not alone. Many users (especially those new to version control) are initially perplexed about the proper syntax of the command, and about how and when the feature should be used. But fear not, this command is actually much simpler than you think! There's a very easy technique for understanding exactly how svn merge behaves.

The main source of confusion is the name of the command. The term “merge” somehow denotes that branches are combined together, or that there's some sort of mysterious blending of data going on. That's not the case. A better name for the command might have been svn diff-and-apply, because that's all that happens: two repository trees are compared, and the differences are applied to a working copy.

The command takes three arguments:

Once these three arguments are specified, the two trees are compared, and the resulting differences are applied to the target working copy as local modifications. When the command is done, the results are no different than if you had hand-edited the files, or run various svn add or svn delete commands yourself. If you like the results, you can commit them. If you don't like the results, you can simply svn revert all of the changes.

The syntax of svn merge allows you to specify the three necessary arguments rather flexibly. Here are some examples:

      
$ svn merge http://svn.example.com/repos/branch1@150 \
            http://svn.example.com/repos/branch2@212 \
            my-working-copy
            
$ svn merge -r 100:200 http://svn.example.com/repos/trunk my-working-copy

$ svn merge -r 100:200 http://svn.example.com/repos/trunk

The first syntax lays out all three arguments explicitly, naming each tree in the form URL@REV and naming the working copy target. The second syntax can be used as a shorthand for situations when you're comparing two different revisions of the same URL. The last syntax shows how the working-copy argument is optional; if omitted, it defaults to the current directory.

To complete our running example, we'll move forward in time. Suppose several days have passed, and many changes have happened on both the trunk and your private branch. Suppose that you've finished working on your private branch; the feature or bug fix is finally complete, and now you want to merge all of your branch changes back into the trunk for others to enjoy.

So how do we use svn merge in this scenario? Remember that this command compares two trees, and applies the differences to a working copy. So to receive the changes, you need to have a working copy of the trunk. We'll assume that either you still have your original one lying around (fully updated), or that you recently checked out a fresh working copy of /calc/trunk.

But which two trees should be compared? At first glance, the answer may seem obvious: just compare the latest trunk tree with your latest branch tree. But beware—this assumption is wrong, and has burned many a new user! Since svn merge operates like svn diff, comparing the latest trunk and branch trees will not merely describe the set of changes you made to your branch. Such a comparison shows too many changes: it would not only show the addition of your branch changes, but also the removal of trunk changes that never happened on your branch.

To express only the changes that happened on your branch, you need to compare the initial state of your branch to its final state. Using svn log on your branch, you can see that your branch was created in revision 341. And the final state of your branch is simply a matter of using the HEAD revision. That means you want to compare revisions 341 and HEAD of your branch directory, and apply those differences to a working copy of the trunk.

$ svn log --verbose --stop-on-copy \
          http://svn.example.com/repos/calc/branches/my-calc-branch
…
------------------------------------------------------------------------
r341 | user | 2002-11-03 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines
Changed paths:
   A /calc/branches/my-calc-branch (from /calc/trunk:340)

$

Here's the final merging procedure, then:

$ cd calc/trunk
$ svn update
At revision 405.

$ svn merge -r 341:405 http://svn.example.com/repos/calc/branches/my-calc-branch
U   integer.c
U   button.c
U   Makefile

$ svn status
M   integer.c
M   button.c
M   Makefile

# ...examine the diffs, compile, test, etc...

$ svn commit -m "Merged my-calc-branch changes r341:405 into the trunk."
Sending        integer.c
Sending        button.c
Sending        Makefile
Transmitting file data ...
Committed revision 406.

Again, notice that the commit log message very specifically mentions the range of changes that was merged into the trunk. Always remember to do this, because it's critical information you'll need later on.

For example, suppose you decide to keep working on your branch for another week, in order to complete an enhancement to your original feature or bug fix. The repository's HEAD revision is now 480, and you're ready to do another merge from your private branch to the trunk. But as discussed in the section called “Best Practices for Merging”, you don't want to merge the changes you've already merged before; you only want to merge everything “new” on your branch since the last time you merged. The trick is to figure out what's new.

The first step is to run svn log on the trunk, and look for a log message about the last time you merged from the branch:

$ cd calc/trunk
$ svn log
…
------------------------------------------------------------------------
r406 | user | 2004-02-08 11:17:26 -0600 (Sun, 08 Feb 2004) | 1 line

Merged my-calc-branch changes r341:405 into the trunk.
------------------------------------------------------------------------
…

Aha! Since all branch-changes that happened between revisions 341 and 405 were previously merged to the trunk as revision 406, you now know that you want to merge only the branch changes after that—by comparing revisions 406 and HEAD.

$ cd calc/trunk
$ svn update
At revision 480.

# We notice that HEAD is currently 480, so we use it to do the merge:

$ svn merge -r 406:480 http://svn.example.com/repos/calc/branches/my-calc-branch
U   integer.c
U   button.c
U   Makefile

$ svn commit -m "Merged my-calc-branch changes r406:480 into the trunk."
Sending        integer.c
Sending        button.c
Sending        Makefile
Transmitting file data ...
Committed revision 481.

Now the trunk contains the complete second wave of changes made to the branch. At this point, you can either delete your branch (we'll discuss this later on), or continue working on your branch and repeat this procedure for subsequent merges.

Another common use for svn merge is to roll back a change that has already been committed. Suppose you're working away happily on a working copy of /calc/trunk, and you discover that the change made way back in revision 303, which changed integer.c, is completely wrong. It never should have been committed. You can use svn merge to “undo” the change in your working copy, and then commit the local modification to the repository. All you need to do is to specify a reverse difference:

$ svn merge -r 303:302 http://svn.example.com/repos/calc/trunk
U  integer.c

$ svn status
M  integer.c

$ svn diff
…
# verify that the change is removed
…

$ svn commit -m "Undoing change committed in r303."
Sending        integer.c
Transmitting file data .
Committed revision 350.

One way to think about a repository revision is as a specific group of changes (some version control systems call these changesets). By using the -r switch, you can ask svn merge to apply a changeset, or whole range of changesets, to your working copy. In our case of undoing a change, we're asking svn merge to apply changeset #303 to our working copy backwards.

Keep in mind that rolling back a change like this is just like any other svn merge operation, so you should use svn status and svn diff to confirm that your work is in the state you want it to be in, and then use svn commit to send the final version to the repository. After committing, this particular changeset is no longer reflected in the HEAD revision.

Again, you may be thinking: well, that really didn't undo the commit, did it? The change still exists in revision 303. If somebody checks out a version of the calc project between revisions 303 and 349, they'll still see the bad change, right?

Yes, that's true. When we talk about “removing” a change, we're really talking about removing it from HEAD. The original change still exists in the repository's history. For most situations, this is good enough. Most people are only interested in tracking the HEAD of a project anyway. There are special cases, however, where you really might want to destroy all evidence of the commit. (Perhaps somebody accidentally committed a confidential document.) This isn't so easy, it turns out, because Subversion was deliberately designed to never lose information. Revisions are immutable trees which build upon one another. Removing a revision from history would cause a domino effect, creating chaos in all subsequent revisions and possibly invalidating all working copies. ^[10]

The great thing about version control systems is that information is never lost. Even when you delete a file or directory, it may be gone from the HEAD revision, but the object still exists in earlier revisions. One of the most common questions new users ask is, “How do I get my old file or directory back?”.

The first step is to define exactly which item you're trying to resurrect. Here's a useful metaphor: you can think of every object in the repository as existing in a sort of two-dimensional coordinate system. The first coordinate is a particular revision tree, and the second coordinate is a path within that tree. So every version of your file or directory can be defined by a specific coordinate pair.

Subversion has no Attic directory like CVS does, ^[11] so you need to use svn log to discover the exact coordinate pair you wish to resurrect. A good strategy is to run svn log --verbose in a directory which used to contain your deleted item. The --verbose option shows a list of all changed items in each revision; all you need to do is find the revision in which you deleted the file or directory. You can do this visually, or by using another tool to examine the log output (via grep, or perhaps via an incremental search in an editor).

$ cd parent-dir
$ svn log --verbose
…
------------------------------------------------------------------------
r808 | joe | 2003-12-26 14:29:40 -0600 (Fri, 26 Dec 2003) | 3 lines
Changed paths:
   D /calc/trunk/real.c
   M /calc/trunk/integer.c

Added fast fourier transform functions to integer.c.
Removed real.c because code now in double.c.
…

In the example, we're assuming that you're looking for a deleted file real.c. By looking through the logs of a parent directory, you've spotted that this file was deleted in revision 808. Therefore, the last version of the file to exist was in the revision right before that. Conclusion: you want to resurrect the path /calc/trunk/real.c from revision 807.

That was the hard part—the research. Now that you know what you want to restore, you have two different choices.

One option is to use svn merge to apply revision 808 “in reverse”. (We've already discussed how to undo changes, see the section called “Undoing Changes”.) This would have the effect of re-adding real.c as a local modification. The file would be scheduled for addition, and after a commit, the file would again exist in HEAD.

In this particular example, however, this is probably not the best strategy. Reverse-applying revision 808 would not only schedule real.c for addition, but the log message indicates that it would also undo certain changes to integer.c, which you don't want. Certainly, you could reverse-merge revision 808 and then svn revert the local modifications to integer.c, but this technique doesn't scale well. What if there were 90 files changed in revision 808?

A second, more targeted strategy is not to use svn merge at all, but rather the svn copy command. Simply copy the exact revision and path “coordinate pair” from the repository to your working copy:

$ svn copy --revision 807 \
           http://svn.example.com/repos/calc/trunk/real.c ./real.c

$ svn status
A  +   real.c

$ svn commit -m "Resurrected real.c from revision 807, /calc/trunk/real.c."
Adding         real.c
Transmitting file data .
Committed revision 1390.

The plus sign in the status output indicates that the item isn't merely scheduled for addition, but scheduled for addition “with history”. Subversion remembers where it was copied from. In the future, running svn log on this file will traverse back through the file's resurrection and through all the history it had prior to revision 807. In other words, this new real.c isn't really new; it's a direct descendant of the original, deleted file.

Although our example shows us resurrecting a file, note that these same techniques work just as well for resurrecting deleted directories.

Version control is most often used for software development, so here's a quick peek at two of the most common branching/merging patterns used by teams of programmers. If you're not using Subversion for software development, feel free to skip this section. If you're a software developer using version control for the first time, pay close attention, as these patterns are often considered best practices by experienced folk. These processes aren't specific to Subversion; they're applicable to any version control system. Still, it may help to see them described in Subversion terms.

$ cd trunk-working-copy

$ svn update
At revision 1910.

$ svn merge http://svn.example.com/repos/calc/trunk@1910 \
            http://svn.example.com/repos/calc/branches/mybranch@1910
U  real.c
U  integer.c
A  newdirectory
A  newdirectory/newfile
…

Once again, svn copy comes to the rescue. If you want to create a snapshot of /calc/trunk exactly as it looks in the HEAD revision, then make a copy of it:

$ svn copy http://svn.example.com/repos/calc/trunk \
           http://svn.example.com/repos/calc/tags/release-1.0 \
      -m "Tagging the 1.0 release of the 'calc' project."

Committed revision 351.

This example assumes that a /calc/tags directory already exists. (If it doesn't, see svn mkdir). After the copy completes, the new release-1.0 directory is forever a snapshot of how the project looked in the HEAD revision at the time you made the copy. Of course you might want to be more precise about exactly which revision you copy, in case somebody else may have committed changes to the project when you weren't looking. So if you know that revision 350 of /calc/trunk is exactly the snapshot you want, you can specify it by passing -r 350 to the svn copy command.

But wait a moment: isn't this tag-creation procedure the same procedure we used to create a branch? Yes, in fact, it is. In Subversion, there's no difference between a tag and a branch. Both are just ordinary directories that are created by copying. Just as with branches, the only reason a copied directory is a “tag” is because humans have decided to treat it that way: as long as nobody ever commits to the directory, it forever remains a snapshot. If people start committing to it, it becomes a branch.

If you are administering a repository, there are two approaches you can take to managing tags. The first approach is “hands off”: as a matter of project policy, decide where your tags will live, and make sure all users know how to treat the directories they copy in there. (That is, make sure they know not to commit to them.) The second approach is more paranoid: you can use one of the access-control scripts provided with Subversion to prevent anyone from doing anything but creating new copies in the tags-area (See Chapter 6, Server Configuration.) The paranoid approach, however, isn't usually necessary. If a user accidentally commits a change to a tag-directory, you can simply undo the change as discussed in the previous section. This is version control, after all.

Sometimes you may want your “snapshot” to be more complicated than a single directory at a single revision.

For example, pretend your project is much larger than our calc example: suppose it contains a number of subdirectories and many more files. In the course of your work, you may decide that you need to create a working copy that is designed to have specific features and bug fixes. You can accomplish this by selectively backdating files or directories to particular revisions (using svn update -r liberally), or by switching files and directories to particular branches (making use of svn switch). When you're done, your working copy is a hodgepodge of repository locations from different revisions. But after testing, you know it's the precise combination of data you need.

Time to make a snapshot. Copying one URL to another won't work here. In this case, you want to make a snapshot of your exact working copy arrangement and store it in the repository. Luckily, svn copy actually has four different uses (which you can read about in Chapter 9), including the ability to copy a working-copy tree to the repository:

$ ls
my-working-copy/

$ svn copy my-working-copy http://svn.example.com/repos/calc/tags/mytag

Committed revision 352.

Now there is a new directory in the repository, /calc/tags/mytag, which is an exact snapshot of your working copy—mixed revisions, URLs, and all.

Other users have found interesting uses for this feature. Sometimes there are situations where you have a bunch of local changes made to your working copy, and you'd like a collaborator to see them. Instead of running svn diff and sending a patch file (which won't capture tree changes, symlink changes or changes in properties), you can instead use svn copy to “upload” your working copy to a private area of the repository. Your collaborator can then either checkout a verbatim copy of your working copy, or use svn merge to receive your exact changes.

There are some standard, recommended ways to organize a repository. Most people create a trunk directory to hold the “main line” of development, a branches directory to contain branch copies, and a tags directory to contain tag copies. If a repository holds only one project, then often people create these top-level directories:

/trunk
/branches
/tags

If a repository contains multiple projects, admins typically index their layout by project (see the section called “Choosing a Repository Layout” to read more about “project roots”):

/paint/trunk
/paint/branches
/paint/tags
/calc/trunk
/calc/branches
/calc/tags

Of course, you're free to ignore these common layouts. You can create any sort of variation, whatever works best for you or your team. Remember that whatever you choose, it's not a permanent commitment. You can reorganize your repository at any time. Because branches and tags are ordinary directories, the svn move command can move or rename them however you wish. Switching from one layout to another is just a matter of issuing a series of server-side moves; if you don't like the way things are organized in the repository, just juggle the directories around.

Remember, though, that while moving directories may be easy to do, you need to be considerate of your users as well. Your juggling can be disorienting to users with existing working copies. If a user has a working copy of a particular repository directory, your svn move operation might remove the path from the latest revision. When the user next runs svn update, she will be told that her working copy represents a path that no longer exists, and the user will be forced to svn switch to the new location.

Another nice feature of Subversion's model is that branches and tags can have finite lifetimes, just like any other versioned item. For example, suppose you eventually finish all your work on your personal branch of the calc project. After merging all of your changes back into /calc/trunk, there's no need for your private branch directory to stick around anymore:

$ svn delete http://svn.example.com/repos/calc/branches/my-calc-branch \
             -m "Removing obsolete branch of calc project."

Committed revision 375.

And now your branch is gone. Of course it's not really gone: the directory is simply missing from the HEAD revision, no longer distracting anyone. If you use svn checkout, svn switch, or svn list to examine an earlier revision, you'll still be able to see your old branch.

If browsing your deleted directory isn't enough, you can always bring it back. Resurrecting data is very easy in Subversion. If there's a deleted directory (or file) that you'd like to bring back into HEAD, simply use svn copy -r to copy it from the old revision:

$ svn copy -r 374 http://svn.example.com/repos/calc/branches/my-calc-branch \
                  http://svn.example.com/repos/calc/branches/my-calc-branch

Committed revision 376.

In our example, your personal branch had a relatively short lifetime: you may have created it to fix a bug or implement a new feature. When your task is done, so is the branch. In software development, though, it's also common to have two “main” branches running side-by-side for very long periods. For example, suppose it's time to release a stable version of the calc project to the public, and you know it's going to take a couple of months to shake bugs out of the software. You don't want people to add new features to the project, but you don't want to tell all developers to stop programming either. So instead, you create a “stable” branch of the software that won't change much:

$ svn copy http://svn.example.com/repos/calc/trunk \
         http://svn.example.com/repos/calc/branches/stable-1.0 \
         -m "Creating stable branch of calc project."

Committed revision 377.

And now developers are free to continue adding cutting-edge (or experimental) features to /calc/trunk, and you can declare a project policy that only bug fixes are to be committed to /calc/branches/stable-1.0. That is, as people continue to work on the trunk, a human selectively ports bug fixes over to the stable branch. Even after the stable branch has shipped, you'll probably continue to maintain the branch for a long time—that is, as long as you continue to support that release for customers.

Conceptually speaking, a Subversion repository is a sequence of directory trees. Each tree is a snapshot of how the files and directories versioned in your repository looked at some point in time. These snapshots are created as a result of client operations, and are called revisions.

Every revision begins life as a transaction tree. When doing a commit, a client builds a Subversion transaction that mirrors their local changes (plus any additional changes that might have been made to the repository since the beginning of the client's commit process), and then instructs the repository to store that tree as the next snapshot in the sequence. If the commit succeeds, the transaction is effectively promoted into a new revision tree, and is assigned a new revision number. If the commit fails for some reason, the transaction is destroyed and the client is informed of the failure.

Updates work in a similar way. The client builds a temporary transaction tree that mirrors the state of the working copy. The repository then compares that transaction tree with the revision tree at the requested revision (usually the most recent, or “youngest” tree), and sends back information that informs the client about what changes are needed to transform their working copy into a replica of that revision tree. After the update completes, the temporary transaction is deleted.

The use of transaction trees is the only way to make permanent changes to a repository's versioned filesystem. However, it's important to understand that the lifetime of a transaction is completely flexible. In the case of updates, transactions are temporary trees that are immediately destroyed. In the case of commits, transactions are transformed into permanent revisions (or removed if the commit fails). In the case of an error or bug, it's possible that a transaction can be accidentally left lying around in the repository (not really affecting anything, but still taking up space).

In theory, someday whole workflow applications might revolve around more fine-grained control of transaction lifetime. It is feasible to imagine a system whereby each transaction slated to become a revision is left in stasis well after the client finishes describing its changes to repository. This would enable each new commit to be reviewed by someone else, perhaps a manager or engineering QA team, who can choose to promote the transaction into a revision, or abort it.

Transactions and revisions in the Subversion repository can have properties attached to them. These properties are generic key-to-value mappings, and are generally used to store information about the tree to which they are attached. The names and values of these properties are stored in the repository's filesystem, along with the rest of your tree data.

Revision and transaction properties are useful for associating information with a tree that is not strictly related to the files and directories in that tree—the kind of information that isn't managed by client working copies. For example, when a new commit transaction is created in the repository, Subversion adds a property to that transaction named svn:date—a datestamp representing the time that the transaction was created. By the time the commit process is finished, and the transaction is promoted to a permanent revision, the tree has also been given a property to store the username of the revision's author (svn:author) and a property to store the log message attached to that revision (svn:log).

Revision and transaction properties are unversioned properties—as they are modified, their previous values are permanently discarded. Also, while revision trees themselves are immutable, the properties attached to those trees are not. You can add, remove, and modify revision properties at any time in the future. If you commit a new revision and later realize that you had some misinformation or spelling error in your log message, you can simply replace the value of the svn:log property with a new, corrected log message.

As of Subversion 1.1, there are two options for storing data in a Subversion repository. One type of repository stores everything in a Berkeley DB database; the other kind stores data in ordinary flat files, using a custom format. Because Subversion developers often refer to a repository as “the (versioned) filesystem”, they have adopted the habit of referring to the latter type of repository as FSFS ^[14] —a versioned filesystem implementation that uses the native OS filesystem to store data.

When a repository is created, an administrator must decide whether it will use Berkeley DB or FSFS. There are advantages and disadvantages to each, which we'll describe in a bit. Neither back-end is more “official” than another, and programs which access the repository are insulated from this implementation detail. Programs have no idea how a repository is storing data; they only see revision and transaction trees through the repository API.

Table 5.1, “Repository Data Store Comparison” gives a comparative overview of Berkeley DB and FSFS repositories. The next sections go into detail.

A hook is a program triggered by some repository event, such as the creation of a new revision or the modification of an unversioned property. Each hook is handed enough information to tell what that event is, what target(s) it's operating on, and the username of the person who triggered the event. Depending on the hook's output or return status, the hook program may continue the action, stop it, or suspend it in some way.

The hooks subdirectory is, by default, filled with templates for various repository hooks.

$ ls repos/hooks/
post-commit.tmpl          post-unlock.tmpl          pre-revprop-change.tmpl
post-lock.tmpl            pre-commit.tmpl           pre-unlock.tmpl
post-revprop-change.tmpl  pre-lock.tmpl             start-commit.tmpl

There is one template for each hook that the Subversion repository implements, and by examining the contents of those template scripts, you can see what triggers each such script to run and what data is passed to that script. Also present in many of these templates are examples of how one might use that script, in conjunction with other Subversion-supplied programs, to perform common useful tasks. To actually install a working hook, you need only place some executable program or script into the repos/hooks directory which can be executed as the name (like start-commit or post-commit) of the hook.

On Unix platforms, this means supplying a script or program (which could be a shell script, a Python program, a compiled C binary, or any number of other things) named exactly like the name of the hook. Of course, the template files are present for more than just informational purposes—the easiest way to install a hook on Unix platforms is to simply copy the appropriate template file to a new file that lacks the .tmpl extension, customize the hook's contents, and ensure that the script is executable. Windows, however, uses file extensions to determine whether or not a program is executable, so you would need to supply a program whose basename is the name of the hook, and whose extension is one of the special extensions recognized by Windows for executable programs, such as .exe or .com for programs, and .bat for batch files.

There are nine hooks implemented by the Subversion repository:

Subversion will attempt to execute hooks as the same user who owns the process which is accessing the Subversion repository. In most cases, the repository is being accessed via Apache HTTP server and mod_dav_svn, so this user is the same user that Apache runs as. The hooks themselves will need to be configured with OS-level permissions that allow that user to execute them. Also, this means that any file or programs (including the Subversion repository itself) accessed directly or indirectly by the hook will be accessed as the same user. In other words, be alert to potential permission-related problems that could prevent the hook from performing the tasks you've written it to perform.

A Berkeley DB environment is an encapsulation of one or more databases, log files, region files and configuration files. The Berkeley DB environment has its own set of default configuration values for things like the number of database locks allowed to be taken out at any given time, or the maximum size of the journaling log files, etc. Subversion's filesystem code additionally chooses default values for some of the Berkeley DB configuration options. However, sometimes your particular repository, with its unique collection of data and access patterns, might require a different set of configuration option values.

The folks at Sleepycat (the producers of Berkeley DB) understand that different databases have different requirements, and so they have provided a mechanism for overriding at runtime many of the configuration values for the Berkeley DB environment. Berkeley checks for the presence of a file named DB_CONFIG in each environment directory, and parses the options found in that file for use with that particular Berkeley environment.

The Berkeley configuration file for your repository is located in the db environment directory, at repos/db/DB_CONFIG. Subversion itself creates this file when it creates the rest of the repository. The file initially contains some default options, as well as pointers to the Berkeley DB online documentation so you can read about what those options do. Of course, you are free to add any of the supported Berkeley DB options to your DB_CONFIG file. Just be aware that while Subversion never attempts to read or interpret the contents of the file, and makes no use of the option settings in it, you'll want to avoid any configuration changes that may cause Berkeley DB to behave in a fashion that is unexpected by the rest of the Subversion code. Also, changes made to DB_CONFIG won't take effect until you recover the database environment (using svnadmin recover).

Subversion provides a handful of utilities useful for creating, inspecting, modifying and repairing your repository. Let's look more closely at each of those tools. Afterward, we'll briefly examine some of the utilities included in the Berkeley DB distribution that provide functionality specific to your repository's database backend not otherwise provided by Subversion's own tools.

$ svnlook help
general usage: svnlook SUBCOMMAND REPOS_PATH [ARGS & OPTIONS ...]
Note: any subcommand which takes the '--revision' and '--transaction'
      options will, if invoked without one of those options, act on
      the repository's youngest revision.
Type "svnlook help <subcommand>" for help on a specific subcommand.
…

$ svnlook info /path/to/repos
$ svnlook info /path/to/repos --revision 19

$ svnlook youngest /path/to/repos
19

$ svnlook info /path/to/repos
sally
2002-11-04 09:29:13 -0600 (Mon, 04 Nov 2002)
27
Added the usual
Greek tree.

$ svnlook tree /path/to/repos --show-ids
/ <0.0.1>
 A/ <2.0.1>
  B/ <4.0.1>
   lambda <5.0.1>
   E/ <6.0.1>
    alpha <7.0.1>
    beta <8.0.1>
   F/ <9.0.1>
  mu <3.0.1>
  C/ <a.0.1>
  D/ <b.0.1>
   gamma <c.0.1>
   G/ <d.0.1>
    pi <e.0.1>
    rho <f.0.1>
    tau <g.0.1>
   H/ <h.0.1>
    chi <i.0.1>
    omega <k.0.1>
    psi <j.0.1>
 iota <1.0.1>

$ svnadmin help
general usage: svnadmin SUBCOMMAND REPOS_PATH  [ARGS & OPTIONS ...]
Type "svnadmin help <subcommand>" for help on a specific subcommand.

Available subcommands:
   create
   deltify
   dump
   help (?, h)
…

$ svndumpfilter help
general usage: svndumpfilter SUBCOMMAND [ARGS & OPTIONS ...]
Type "svndumpfilter help <subcommand>" for help on a specific subcommand.

Available subcommands:
   exclude
   include
   help (?, h)

/
   calc/
      trunk/
      branches/
      tags/
   calendar/
      trunk/
      branches/
      tags/
   spreadsheet/
      trunk/
      branches/
      tags/

$ svnadmin dump /path/to/repos > repos-dumpfile
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
* Dumped revision 3.
…
$

$ cat repos-dumpfile | svndumpfilter include calc > calc-dumpfile
…
$ cat repos-dumpfile | svndumpfilter include calendar > cal-dumpfile
…
$ cat repos-dumpfile | svndumpfilter include spreadsheet > ss-dumpfile
…
$

Node-path: calc
Node-action: add
Node-kind: dir
Content-length: 0

$ svnadmin create calc; svnadmin load calc < calc-dumpfile
<<< Started new transaction, based on original revision 1
     * adding path : Makefile ... done.
     * adding path : button.c ... done.
…
$ svnadmin create calendar; svnadmin load calendar < cal-dumpfile
<<< Started new transaction, based on original revision 1
     * adding path : Makefile ... done.
     * adding path : cal.c ... done.
…
$ svnadmin create spreadsheet; svnadmin load spreadsheet < ss-dumpfile
<<< Started new transaction, based on original revision 1
     * adding path : Makefile ... done.
     * adding path : ss.c ... done.
…
$

…
Node-path: spreadsheet/Makefile
…

Your Subversion repository will generally require very little attention once it is configured to your liking. However, there are times when some manual assistance from an administrator might be in order. The svnadmin utility provides some helpful functionality to assist you in performing such tasks as:

Perhaps the most commonly used of svnadmin's subcommands is setlog. When a transaction is committed to the repository and promoted to a revision, the descriptive log message associated with that new revision (and provided by the user) is stored as an unversioned property attached to the revision itself. In other words, the repository remembers only the latest value of the property, and discards previous ones.

Sometimes a user will have an error in her log message (a misspelling or some misinformation, perhaps). If the repository is configured (using the pre-revprop-change and post-revprop-change hooks; see the section called “Hook Scripts”) to accept changes to this log message after the commit is finished, then the user can “fix” her log message remotely using the svn program's propset command (see Chapter 9, Subversion Complete Reference). However, because of the potential to lose information forever, Subversion repositories are not, by default, configured to allow changes to unversioned properties—except by an administrator.

If a log message needs to be changed by an administrator, this can be done using svnadmin setlog. This command changes the log message (the svn:log property) on a given revision of a repository, reading the new value from a provided file.

$ echo "Here is the new, correct log message" > newlog.txt
$ svnadmin setlog myrepos newlog.txt -r 388

The svnadmin setlog command alone is still bound by the same protections against modifying unversioned properties as a remote client is—the pre- and post-revprop-change hooks are still triggered, and therefore must be setup to accept changes of this nature. But an administrator can get around these protections by passing the --bypass-hooks option to the svnadmin setlog command.

Another common use of svnadmin is to query the repository for outstanding—possibly dead—Subversion transactions. In the event that a commit should fail, the transaction is usually cleaned up. That is, the transaction itself is removed from the repository, and any data associated with (and only with) that transaction is removed as well. Occasionally, though, a failure occurs in such a way that the cleanup of the transaction never happens. This could happen for several reasons: perhaps the client operation was inelegantly terminated by the user, or a network failure might have occurred in the middle of an operation, etc. Regardless of the reason, dead transactions can happen. They don't do any real harm, other than consuming a small bit of disk space. A fastidious administrator may nonetheless want to remove them.

You can use svnadmin's lstxns command to list the names of the currently outstanding transactions.

$ svnadmin lstxns myrepos
19
3a1
a45
$

Each item in the resultant output can then be used with svnlook (and its --transaction option) to determine who created the transaction, when it was created, what types of changes were made in the transaction—in other words, whether or not the transaction is a safe candidate for removal! If so, the transaction's name can be passed to svnadmin rmtxns, which will perform the cleanup of the transaction. In fact, the rmtxns subcommand can take its input directly from the output of lstxns!

$ svnadmin rmtxns myrepos `svnadmin lstxns myrepos`
$

If you use these two subcommands like this, you should consider making your repository temporarily inaccessible to clients. That way, no one can begin a legitimate transaction before you start your cleanup. The following is a little bit of shell-scripting that can quickly generate information about each outstanding transaction in your repository:

#!/bin/sh

### Generate informational output for all outstanding transactions in
### a Subversion repository.

REPOS="${1}"
if [ "x$REPOS" = x ] ; then
  echo "usage: $0 REPOS_PATH"
  exit
fi

for TXN in `svnadmin lstxns ${REPOS}`; do 
  echo "---[ Transaction ${TXN} ]-------------------------------------------"
  svnlook info "${REPOS}" --transaction "${TXN}"
done

You can run the previous script using /path/to/txn-info.sh /path/to/repos. The output is basically a concatenation of several chunks of svnlook info output (see the section called “svnlook”), and will look something like:

$ txn-info.sh myrepos
---[ Transaction 19 ]-------------------------------------------
sally
2001-09-04 11:57:19 -0500 (Tue, 04 Sep 2001)
0
---[ Transaction 3a1 ]-------------------------------------------
harry
2001-09-10 16:50:30 -0500 (Mon, 10 Sep 2001)
39
Trying to commit over a faulty network.
---[ Transaction a45 ]-------------------------------------------
sally
2001-09-12 11:09:28 -0500 (Wed, 12 Sep 2001)
0
$

A long-abandoned transaction usually represents some sort of failed or interrupted commit. A transaction's datestamp can provide interesting information—for example, how likely is it that an operation begun nine months ago is still active?

In short, transaction cleanup decisions need not be made unwisely. Various sources of information—including Apache's error and access logs, the logs of successful Subversion commits, and so on—can be employed in the decision-making process. Finally, an administrator can often simply communicate with a seemingly dead transaction's owner (via email, for example) to verify that the transaction is, in fact, in a zombie state.

While the cost of storage has dropped incredibly in the past few years, disk usage is still a valid concern for administrators seeking to version large amounts of data. Every additional byte consumed by the live repository is a byte that needs to be backed up offsite, perhaps multiple times as part of rotating backup schedules. If using a Berkeley DB repository, the primary storage mechanism is a complex database system, it is useful to know what pieces of data need to remain on the live site, which need to be backed up, and which can be safely removed. This section is specific to Berkeley DB; FSFS repositories have no extra data to be cleaned up or reclaimed.

Until recently, the largest offender of disk space usage with respect to Subversion repositories was the log files to which Berkeley DB performs its pre-writes before modifying the actual database files. These files capture all the actions taken along the route of changing the database from one state to another—while the database files reflect at any given time some state, the log files contain all the many changes along the way between states. As such, they can start to accumulate quite rapidly.

Fortunately, beginning with the 4.2 release of Berkeley DB, the database environment has the ability to remove its own unused log files without any external procedures. Any repositories created using an svnadmin which is compiled against Berkeley DB version 4.2 or greater will be configured for this automatic log file removal. If you don't want this feature enabled, simply pass the --bdb-log-keep option to the svnadmin create command. If you forget to do this, or change your mind at a later time, simple edit the DB_CONFIG file found in your repository's db directory, comment out the line which contains the set_flags DB_LOG_AUTOREMOVE directive, and then run svnadmin recover on your repository to force the configuration changes to take effect. See the section called “Berkeley DB Configuration” for more information about database configuration.

Without some sort of automatic log file removal in place, log files will accumulate as you use your repository. This is actually somewhat of a feature of the database system—you should be able to recreate your entire database using nothing but the log files, so these files can be useful for catastrophic database recovery. But typically, you'll want to archive the log files that are no longer in use by Berkeley DB, and then remove them from disk to conserve space. Use the svnadmin list-unused-dblogs command to list the unused log files:

$ svnadmin list-unused-dblogs /path/to/repos
/path/to/repos/log.0000000031
/path/to/repos/log.0000000032
/path/to/repos/log.0000000033

$ svnadmin list-unused-dblogs /path/to/repos | xargs rm
## disk space reclaimed!

To keep the size of the repository as small as possible, Subversion uses deltification (or, “deltified storage”) within the repository itself. Deltification involves encoding the representation of a chunk of data as a collection of differences against some other chunk of data. If the two pieces of data are very similar, this deltification results in storage savings for the deltified chunk—rather than taking up space equal to the size of the original data, it only takes up enough space to say, “I look just like this other piece of data over here, except for the following couple of changes”. Specifically, each time a new version of a file is committed to the repository, Subversion encodes the previous version (actually, several previous versions) as a delta against the new version. The result is that most of the repository data that tends to be sizable—namely, the contents of versioned files—is stored at a much smaller size than the original “fulltext” representation of that data.

As mentioned in the section called “Berkeley DB”, a Berkeley DB repository can sometimes be left in frozen state if not closed properly. When this happens, an administrator needs to rewind the database back into a consistent state.

In order to protect the data in your repository, Berkeley DB uses a locking mechanism. This mechanism ensures that portions of the database are not simultaneously modified by multiple database accessors, and that each process sees the data in the correct state when that data is being read from the database. When a process needs to change something in the database, it first checks for the existence of a lock on the target data. If the data is not locked, the process locks the data, makes the change it wants to make, and then unlocks the data. Other processes are forced to wait until that lock is removed before they are permitted to continue accessing that section of the database. (This has nothing to do with the locks that you, as a user, can apply to versioned files within the repository; see Three meanings of “lock” for more information.)

In the course of using your Subversion repository, fatal errors (such as running out of disk space or available memory) or interruptions can prevent a process from having the chance to remove the locks it has placed in the database. The result is that the back-end database system gets “wedged”. When this happens, any attempts to access the repository hang indefinitely (since each new accessor is waiting for a lock to go away—which isn't going to happen).

First, if this happens to your repository, don't panic. The Berkeley DB filesystem takes advantage of database transactions and checkpoints and pre-write journaling to ensure that only the most catastrophic of events ^[17] can permanently destroy a database environment. A sufficiently paranoid repository administrator will be making off-site backups of the repository data in some fashion, but don't call your system administrator to restore a backup tape just yet.

Secondly, use the following recipe to attempt to “unwedge” your repository:

This procedure fixes almost every case of repository lock-up. Make sure that you run this command as the user that owns and manages the database, not just as root. Part of the recovery process might involve recreating from scratch various database files (shared memory regions, for example). Recovering as root will create those files such that they are owned by root, which means that even after you restore connectivity to your repository, regular users will be unable to access it.

If the previous procedure, for some reason, does not successfully unwedge your repository, you should do two things. First, move your broken repository out of the way and restore your latest backup of it. Then, send an email to the Subversion user list (at <users@subversion.tigris.org>) describing your problem in detail. Data integrity is an extremely high priority to the Subversion developers.

A Subversion filesystem has its data spread throughout various database tables in a fashion generally understood by (and of interest to) only the Subversion developers themselves. However, circumstances may arise that call for all, or some subset, of that data to be collected into a single, portable, flat file format. Subversion provides such a mechanism, implemented in a pair of svnadmin subcommands: dump and load.

The most common reason to dump and load a Subversion repository is due to changes in Subversion itself. As Subversion matures, there are times when certain changes made to the back-end database schema cause Subversion to be incompatible with previous versions of the repository. Other reasons for dumping and loading might be to migrate a Berkeley DB repository to a new OS or CPU architecture, or to switch between Berkeley DB and FSFS back-ends. The recommended course of action is relatively simple:

svnadmin dump will output a range of repository revisions that are formatted using Subversion's custom filesystem dump format. The dump format is printed to the standard output stream, while informative messages are printed to the standard error stream. This allows you to redirect the output stream to a file while watching the status output in your terminal window. For example:

$ svnlook youngest myrepos
26
$ svnadmin dump myrepos > dumpfile
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
…
* Dumped revision 25.
* Dumped revision 26.

At the end of the process, you will have a single file (dumpfile in the previous example) that contains all the data stored in your repository in the requested range of revisions. Note that svnadmin dump is reading revision trees from the repository just like any other “reader” process would (svn checkout, for example). So it's safe to run this command at any time.

The other subcommand in the pair, svnadmin load, parses the standard input stream as a Subversion repository dump file, and effectively replays those dumped revisions into the target repository for that operation. It also gives informative feedback, this time using the standard output stream:

$ svnadmin load newrepos < dumpfile
<<< Started new txn, based on original revision 1
     * adding path : A ... done.
     * adding path : A/B ... done.
     …
------- Committed new rev 1 (loaded from original rev 1) >>>

<<< Started new txn, based on original revision 2
     * editing path : A/mu ... done.
     * editing path : A/D/G/rho ... done.

------- Committed new rev 2 (loaded from original rev 2) >>>

…

<<< Started new txn, based on original revision 25
     * editing path : A/D/gamma ... done.

------- Committed new rev 25 (loaded from original rev 25) >>>

<<< Started new txn, based on original revision 26
     * adding path : A/Z/zeta ... done.
     * editing path : A/mu ... done.

------- Committed new rev 26 (loaded from original rev 26) >>>

The result of a load is new revisions added to a repository—the same thing you get by making commits against that repository from a regular Subversion client. And just as in a commit, you can use hook scripts to perform actions before and after each of the commits made during a load process. By passing the --use-pre-commit-hook and --use-post-commit-hook options to svnadmin load, you can instruct Subversion to execute the pre-commit and post-commit hook scripts, respectively, for each loaded revision. You might use these, for example, to ensure that loaded revisions pass through the same validation steps that regular commits pass through. Of course, you should use these options with care—if your post-commit hook sends emails to a mailing list for each new commit, you might not want to spew hundreds or thousands of commit emails in rapid succession at that list for each of the loaded revisions! You can read more about the use of hook scripts in the section called “Hook Scripts”.

Note that because svnadmin uses standard input and output streams for the repository dump and load process, people who are feeling especially saucy can try things like this (perhaps even using different versions of svnadmin on each side of the pipe):

$ svnadmin create newrepos
$ svnadmin dump myrepos | svnadmin load newrepos

By default, the dump file will be quite large—much larger than the repository itself. That's because every version of every file is expressed as a full text in the dump file. This is the fastest and simplest behavior, and nice if you're piping the dump data directly into some other process (such as a compression program, filtering program, or into a loading process). But if you're creating a dump file for longer-term storage, you'll likely want to save disk space by using the --deltas switch. With this option, successive revisions of files will be output as compressed, binary differences—just as file revisions are stored in a repository. This option is slower, but results in a dump file much closer in size to the original repository.

We mentioned previously that svnadmin dump outputs a range of revisions. Use the --revision option to specify a single revision to dump, or a range of revisions. If you omit this option, all the existing repository revisions will be dumped.

$ svnadmin dump myrepos --revision 23 > rev-23.dumpfile
$ svnadmin dump myrepos --revision 100:200 > revs-100-200.dumpfile

As Subversion dumps each new revision, it outputs only enough information to allow a future loader to re-create that revision based on the previous one. In other words, for any given revision in the dump file, only the items that were changed in that revision will appear in the dump. The only exception to this rule is the first revision that is dumped with the current svnadmin dump command.

By default, Subversion will not express the first dumped revision as merely differences to be applied to the previous revision. For one thing, there is no previous revision in the dump file! And secondly, Subversion cannot know the state of the repository into which the dump data will be loaded (if it ever, in fact, occurs). To ensure that the output of each execution of svnadmin dump is self-sufficient, the first dumped revision is by default a full representation of every directory, file, and property in that revision of the repository.

However, you can change this default behavior. If you add the --incremental option when you dump your repository, svnadmin will compare the first dumped revision against the previous revision in the repository, the same way it treats every other revision that gets dumped. It will then output the first revision exactly as it does the rest of the revisions in the dump range—mentioning only the changes that occurred in that revision. The benefit of this is that you can create several small dump files that can be loaded in succession, instead of one large one, like so:

$ svnadmin dump myrepos --revision 0:1000 > dumpfile1
$ svnadmin dump myrepos --revision 1001:2000 --incremental > dumpfile2
$ svnadmin dump myrepos --revision 2001:3000 --incremental > dumpfile3

These dump files could be loaded into a new repository with the following command sequence:

$ svnadmin load newrepos < dumpfile1
$ svnadmin load newrepos < dumpfile2
$ svnadmin load newrepos < dumpfile3

Another neat trick you can perform with this --incremental option involves appending to an existing dump file a new range of dumped revisions. For example, you might have a post-commit hook that simply appends the repository dump of the single revision that triggered the hook. Or you might have a script that runs nightly to append dump file data for all the revisions that were added to the repository since the last time the script ran. Used like this, svnadmin's dump and load commands can be a valuable means by which to backup changes to your repository over time in case of a system crash or some other catastrophic event.

The dump format can also be used to merge the contents of several different repositories into a single repository. By using the --parent-dir option of svnadmin load, you can specify a new virtual root directory for the load process. That means if you have dump files for three repositories, say calc-dumpfile, cal-dumpfile, and ss-dumpfile, you can first create a new repository to hold them all:

$ svnadmin create /path/to/projects
$

Then, make new directories in the repository which will encapsulate the contents of each of the three previous repositories:

$ svn mkdir -m "Initial project roots" \
      file:///path/to/projects/calc \
      file:///path/to/projects/calendar \
      file:///path/to/projects/spreadsheet
Committed revision 1.
$ 

Lastly, load the individual dump files into their respective locations in the new repository:

$ svnadmin load /path/to/projects --parent-dir calc < calc-dumpfile
…
$ svnadmin load /path/to/projects --parent-dir calendar < cal-dumpfile
…
$ svnadmin load /path/to/projects --parent-dir spreadsheet < ss-dumpfile
…
$

We'll mention one final way to use the Subversion repository dump format—conversion from a different storage mechanism or version control system altogether. Because the dump file format is, for the most part, human-readable, ^[18] it should be relatively easy to describe generic sets of changes—each of which should be treated as a new revision—using this file format. In fact, the cvs2svn utility (see the section called “Converting a Repository from CVS to Subversion”) uses the dump format to represent the contents of a CVS repository so that those contents can be copied into a Subversion repository.

Despite numerous advances in technology since the birth of the modern computer, one thing unfortunately rings true with crystalline clarity—sometimes, things go very, very awry. Power outages, network connectivity dropouts, corrupt RAM and crashed hard drives are but a taste of the evil that Fate is poised to unleash on even the most conscientious administrator. And so we arrive at a very important topic—how to make backup copies of your repository data.

There are generally two types of backup methods available for Subversion repository administrators—incremental and full. We discussed in an earlier section of this chapter how to use svnadmin dump --incremental to perform an incremental backup (see the section called “Migrating a Repository”). Essentially, the idea is to only backup at a given time the changes to the repository since the last time you made a backup.

A full backup of the repository is quite literally a duplication of the entire repository directory (which includes either Berkeley database or FSFS environment). Now, unless you temporarily disable all other access to your repository, simply doing a recursive directory copy runs the risk of generating a faulty backup, since someone might be currently writing to the database.

In the case of Berkeley DB, Sleepycat documents describe a certain order in which database files can be copied that will guarantee a valid backup copy. And a similar ordering exists for FSFS data. Better still, you don't have to implement these algorithms yourself, because the Subversion development team has already done so. The hot-backup.py script is found in the tools/backup/ directory of the Subversion source distribution. Given a repository path and a backup location, hot-backup.py—which is really just a more intelligent wrapper around the svnadmin hotcopy command—will perform the necessary steps for backing up your live repository—without requiring that you bar public repository access at all—and then will clean out the dead Berkeley log files from your live repository.

Even if you also have an incremental backup, you might want to run this program on a regular basis. For example, you might consider adding hot-backup.py to a program scheduler (such as cron on Unix systems). Or, if you prefer fine-grained backup solutions, you could have your post-commit hook script call hot-backup.py (see the section called “Hook Scripts”), which will then cause a new backup of your repository to occur with every new revision created. Simply add the following to the hooks/post-commit script in your live repository directory:

(cd /path/to/hook/scripts; ./hot-backup.py ${REPOS} /path/to/backups &)

The resulting backup is a fully functional Subversion repository, able to be dropped in as a replacement for your live repository should something go horribly wrong.

There are benefits to both types of backup methods. The easiest is by far the full backup, which will always result in a perfect working replica of your repository. This again means that should something bad happen to your live repository, you can restore from the backup with a simple recursive directory copy. Unfortunately, if you are maintaining multiple backups of your repository, these full copies will each eat up just as much disk space as your live repository.

Incremental backups using the repository dump format are excellent to have on hand if the database schema changes between successive versions of Subversion itself. Since a complete repository dump and load are generally required to upgrade your repository to the new schema, it's very convenient to already have half of that process (the dump part) finished. Unfortunately, the creation of—and restoration from—incremental backups takes longer, as each commit is effectively replayed into either the dump file or the repository.

In either backup scenario, repository administrators need to be aware of how modifications to unversioned revision properties affect their backups. Since these changes do not themselves generate new revisions, they will not trigger post-commit hooks, and may not even trigger the pre-revprop-change and post-revprop-change hooks. ^[19] And since you can change revision properties without respect to chronological order—you can change any revision's properties at any time—an incremental backup of the latest few revisions might not catch a property modification to a revision that was included as part of a previous backup.

Generally speaking, only the truly paranoid would need to backup their entire repository, say, every time a commit occurred. However, assuming that a given repository has some other redundancy mechanism in place with relatively fine granularity (like per-commit emails), a hot backup of the database might be something that a repository administrator would want to include as part of a system-wide nightly backup. For most repositories, archived commit emails alone provide sufficient redundancy as restoration sources, at least for the most recent few commits. But it's your data—protect it as much as you'd like.

Often, the best approach to repository backups is a diversified one. You can leverage combinations of full and incremental backups, plus archives of commit emails. The Subversion developers, for example, back up the Subversion source code repository after every new revision is created, and keep an archive of all the commit and property change notification emails. Your solution might be similar, but should be catered to your needs and that delicate balance of convenience with paranoia. And while all of this might not save your hardware from the iron fist of Fate, ^[20] it should certainly help you recover from those trying times.

While Subversion allows you to move around versioned files and directories without any loss of information, doing so can still disrupt the workflow of those who access the repository often and come to expect things to be at certain locations. Try to peer into the future a bit; plan ahead before placing your data under version control. By “laying out” the contents of your repositories in an effective manner the first time, you can prevent a load of future headaches.

There are a few things to consider when setting up Subversion repositories. Let's assume that as repository administrator, you will be responsible for supporting the version control system for several projects. The first decision is whether to use a single repository for multiple projects, or to give each project its own repository, or some compromise of these two.

There are benefits to using a single repository for multiple projects, most obviously the lack of duplicated maintenance. A single repository means that there is one set of hook scripts, one thing to routinely backup, one thing to dump and load if Subversion releases an incompatible new version, and so on. Also, you can move data between projects easily, and without losing any historical versioning information.

The downside of using a single repository is that different projects may have different commit mailing lists or different authentication and authorization requirements. Also, remember that Subversion uses repository-global revision numbers. Some folks don't like the fact that even though no changes have been made to their project lately, the youngest revision number for the repository keeps climbing because other projects are actively adding new revisions.

A middle-ground approach can be taken, too. For example, projects can be grouped by how well they relate to each other. You might have a few repositories with a handful of projects in each repository. That way, projects that are likely to want to share data can do so easily, and as new revisions are added to the repository, at least the developers know that those new revisions are at least remotely related to everyone who uses that repository.

After deciding how to organize your projects with respect to repositories, you'll probably want to think about directory hierarchies in the repositories themselves. Because Subversion uses regular directory copies for branching and tagging (see Chapter 4, Branching and Merging), the Subversion community recommends that you choose a repository location for each project root—the “top-most” directory which contains data related to that project—and then create three subdirectories beneath that root: trunk, meaning the directory under which the main project development occurs; branches, which is a directory in which to create various named branches of the main development line; tags, which is a directory of branches that are created, and perhaps destroyed, but never changed. ^[21]

For example, your repository might look like:

/
   calc/
      trunk/
      tags/
      branches/
   calendar/
      trunk/
      tags/
      branches/
   spreadsheet/
      trunk/
      tags/
      branches/
   …

Note that it doesn't matter where in your repository each project root is. If you have only one project per repository, the logical place to put each project root is at the root of that project's respective repository. If you have multiple projects, you might want to arrange them in groups inside the repository, perhaps putting projects with similar goals or shared code in the same subdirectory, or maybe just grouping them alphabetically. Such an arrangement might look like:

/
   utils/
      calc/
         trunk/
         tags/
         branches/
      calendar/
         trunk/
         tags/
         branches/
      …
   office/
      spreadsheet/
         trunk/
         tags/
         branches/
      …

Lay out your repository in whatever way you see fit. Subversion does not expect or enforce a layout schema—in its eyes, a directory is a directory is a directory. Ultimately, you should choose the repository arrangement that meets the needs of the people who work on the projects that live there.

After deciding how to arrange the projects in your repository, you'll probably want to actually populate the repository with that layout and with initial project data. There are a couple of ways to do this in Subversion. You could use the svn mkdir command (see Chapter 9, Subversion Complete Reference) to create each directory in your skeletal repository layout, one-by-one. A quicker way to accomplish the same task is to use the svn import command (see the section called “svn import”). By first creating the layout in a temporary location on your drive, you can import the whole layout tree into the repository in a single commit:

$ mkdir tmpdir
$ cd tmpdir
$ mkdir projectA
$ mkdir projectA/trunk
$ mkdir projectA/branches
$ mkdir projectA/tags
$ mkdir projectB
$ mkdir projectB/trunk
$ mkdir projectB/branches
$ mkdir projectB/tags
…
$ svn import . file:///path/to/repos --message 'Initial repository layout'
Adding         projectA
Adding         projectA/trunk
Adding         projectA/branches
Adding         projectA/tags
Adding         projectB
Adding         projectB/trunk
Adding         projectB/branches
Adding         projectB/tags
…
Committed revision 1.
$ cd ..
$ rm -rf tmpdir
$

You can verify the results of the import by running the svn list command:

$ svn list --verbose file:///path/to/repos
      1 harry               May 08 21:48 projectA/
      1 harry               May 08 21:48 projectB/
…
$

Once you have your skeletal layout in place, you can begin importing actual project data into your repository, if any such data exists yet. Once again, there are several ways to do this. You could use the svn import command. You could checkout a working copy from your new repository, move and arrange project data inside the working copy, and use the svn add and svn commit commands. But once we start talking about such things, we're no longer discussing repository administration. If you aren't already familiar with the svn client program, see Chapter 3, Guided Tour.

The Subversion client spends most of its time managing working copies. When it needs information from a repository, however, it makes a network request, and the server responds with an appropriate answer. The details of the network protocol are hidden from the user; the client attempts to access a URL, and depending on the URL schema, a particular protocol is used to contact the server (see Repository URLs). Users can run svn --version to see which URL schemas and protocols the client knows how to use.

When the server process receives a client request, it typically demands that the client identify itself. It issues an authentication challenge to the client, and the client responds by providing credentials back to the server. Once authentication is complete, the server responds with the original information the client asked for. Notice that this system is different from systems like CVS, where the client pre-emptively offers credentials (“logs in”) to the server before ever making a request. In Subversion, the server “pulls” credentials by challenging the client at the appropriate moment, rather than the client “pushing” them. This makes certain operations more elegant. For example, if a server is configured to allow anyone in the world to read a repository, then the server will never issue an authentication challenge when a client attempts to svn checkout.

If the client's network request writes new data to the repository (e.g. svn commit), then a new revision tree is created. If the client's request was authenticated, then the authenticated user's name is stored as the value of the svn:author property on the new revision (see the section called “Unversioned Properties”). If the client was not authenticated (in other words, the server never issued an authentication challenge), then the revision's svn:author property is empty. ^[22]

Many servers are configured to require authentication on every request. This can become a big annoyance to users, who are forced to type their passwords over and over again.

Happily, the Subversion client has a remedy for this: a built-in system for caching authentication credentials on disk. By default, whenever the command-line client successfully responds to a server's authentication challenge, it saves the credentials in the user's private runtime configuration area—in ~/.subversion/auth/ on Unix-like systems or %APPDATA%/Subversion/auth/ on Windows. (The runtime area is covered in more detail in the section called “Runtime Configuration Area”.) Successful credentials are cached on disk, keyed on a combination of hostname, port, and authentication realm.

When the client receives an authentication challenge, it first looks for the appropriate credentials in the user's disk cache; if not present, or if the cached credentials fail to authenticate, then the client simply prompts the user for the information.

Security-conscious people may be thinking to themselves, “Caching passwords on disk? That's terrible! You should never do that!” Please remain calm, it's not as dangerous as it sounds.

To disable caching for a single command, pass the --no-auth-cache option:

$ svn commit -F log_msg.txt --no-auth-cache
Authentication realm: <svn://host.example.com:3690> example realm
Username:  joe
Password for 'joe':

Adding         newfile
Transmitting file data .
Committed revision 2324.

# password was not cached, so a second commit still prompts us

$ svn delete newfile
$ svn commit -F new_msg.txt
Authentication realm: <svn://host.example.com:3690> example realm
Username:  joe
…

Or, if you want to disable credential caching permanently, you can edit your runtime config file (located next to the auth/ directory). Simply set store-auth-creds to no, and no credentials will be cached on disk, ever.

[auth]
store-auth-creds = no

Sometimes users will want to remove specific credentials from the disk cache. To do this, you need to navigate into the auth/ area and manually delete the appropriate cache file. Credentials are cached in individual files; if you look inside each file, you will see keys and values. The svn:realmstring key describes the particular server realm that the file is associated with:

$ ls ~/.subversion/auth/svn.simple/
5671adf2865e267db74f09ba6f872c28        
3893ed123b39500bca8a0b382839198e
5c3c22968347b390f349ff340196ed39

$ cat ~/.subversion/auth/svn.simple/5671adf2865e267db74f09ba6f872c28

K 8
username
V 3
joe
K 8
password
V 4
blah
K 15
svn:realmstring
V 45
<https://svn.domain.com:443> Joe's repository
END

Once you have located the proper cache file, just delete it.

One last word about client authentication behavior: a bit of explanation about the --username and --password options is needed. Many client subcommands accept these options; however it is important to understand using these options does not automatically send credentials to the server. As discussed earlier, the server “pulls” credentials from the client when it deems necessary; the client cannot “push” them at will. If a username and/or password are passed as options, they will only be presented to the server if the server requests them. ^[23] Typically, these options are used when:

Here is a final summary that describes how a Subversion client behaves when it receives an authentication challenge:

If the client successfully authenticates by any of the methods listed above, it will attempt to cache the credentials on disk (unless the user has disabled this behavior, as mentioned earlier).

There are a few different ways to invoke the svnserve program. If invoked with no options, you'll see nothing but a help message. However, if you're planning to have inetd launch the process, then you can pass the -i (--inetd) option:

$ svnserve -i
( success ( 1 2 ( ANONYMOUS ) ( edit-pipeline ) ) )

When invoked with the --inetd option, svnserve attempts to speak with a Subversion client via stdin and stdout using a custom protocol. This is the standard behavior for a program being run via inetd. The IANA has reserved port 3690 for the Subversion protocol, so on a Unix-like system you can add lines to /etc/services like these (if they don't already exist):

svn           3690/tcp   # Subversion
svn           3690/udp   # Subversion

And if your system is using a classic Unix-like inetd daemon, you can add this line to /etc/inetd.conf:

svn stream tcp nowait svnowner /usr/bin/svnserve svnserve -i

Make sure “svnowner” is a user which has appropriate permissions to access your repositories. Now, when a client connection comes into your server on port 3690, inetd will spawn an svnserve process to service it.

On a Windows system, third-party tools exist to run svnserve as a service. Look on Subversion's website for a list of these tools.

A second option is to run svnserve as a standalone “daemon” process. Use the -d option for this:

$ svnserve -d
$               # svnserve is now running, listening on port 3690

When running svnserve in daemon mode, you can use the --listen-port= and --listen-host= options to customize the exact port and hostname to “bind” to.

There's still a third way to invoke svnserve, and that's in “tunnel mode”, with the -t option. This mode assumes that a remote-service program such as RSH or SSH has successfully authenticated a user and is now invoking a private svnserve process as that user. The svnserve program behaves normally (communicating via stdin and stdout), and assumes that the traffic is being automatically redirected over some sort of tunnel back to the client. When svnserve is invoked by a tunnel agent like this, be sure that the authenticated user has full read and write access to the repository database files. (See Servers and Permissions: A Word of Warning.) It's essentially the same as a local user accessing the repository via file:/// URLs.

Once the svnserve program is running, it makes every repository on your system available to the network. A client needs to specify an absolute path in the repository URL. For example, if a repository is located at /usr/local/repositories/project1, then a client would reach it via svn://host.example.com/usr/local/repositories/project1 . To increase security, you can pass the -r option to svnserve, which restricts it to exporting only repositories below that path:

$ svnserve -d -r /usr/local/repositories
…

Using the -r option effectively modifies the location that the program treats as the root of the remote filesystem space. Clients then use URLs that have that path portion removed from them, leaving much shorter (and much less revealing) URLs:

$ svn checkout svn://host.example.com/project1
…

When a client connects to an svnserve process, the following things happen:

At the time of writing, the server only knows how to send a CRAM-MD5 ^[24] authentication challenge. In essence, the server sends a bit of data to the client. The client uses the MD5 hash algorithm to create a fingerprint of the data and password combined, then sends the fingerprint as a response. The server performs the same computation with the stored password to verify that the result is identical. At no point does the actual password travel over the network.

It's also possible, of course, for the client to be externally authenticated via a tunnel agent, such as SSH. In that case, the server simply examines the user it's running as, and uses it as the authenticated username. For more on this, see the section called “SSH authentication and authorization”.

As you've already guessed, a repository's svnserve.conf file is the central mechanism for controlling authentication and authorization policies. The file has the same format as other configuration files (see the section called “Runtime Configuration Area”): section names are marked by square brackets ([ and ]), comments begin with hashes (#), and each section contains specific variables that can be set (variable = value). Let's walk through this file and learn how to use them.

[general]
password-db = userfile
realm = example realm

[users]
harry = foopassword
sally = barpassword

[general]
password-db = userfile
realm = example realm

# anonymous users can only read the repository
anon-access = read

# authenticated users can both read and write
auth-access = write

[general]
password-db = userfile
realm = example realm

# anonymous users aren't allowed
anon-access = none

# authenticated users can both read and write
auth-access = write

svnserve's built-in authentication can be very handy, because it avoids the need to create real system accounts. On the other hand, some administrators already have well-established SSH authentication frameworks in place. In these situations, all of the project's users already have system accounts and the ability to “SSH into” the server machine.

It's easy to use SSH in conjunction with svnserve. The client simply uses the svn+ssh:// URL schema to connect:

$ whoami
harry

$ svn list svn+ssh://host.example.com/repos/project
harry@host.example.com's password:  *****

foo
bar
baz
…

In this example, the Subversion client is invoking a local ssh process, connecting to host.example.com, authenticating as the user harry, then spawning a private svnserve process on the remote machine running as the user harry. The svnserve command is being invoked in tunnel mode (-t) and its network protocol is being “tunneled” over the encrypted connection by ssh, the tunnel-agent. svnserve is aware that it's running as the user harry, and if the client performs a commit, the authenticated username will be attributed as the author of the new revision.

The important thing to understand here is that the Subversion client is not connecting to a running svnserve daemon. This method of access doesn't require a daemon, nor does it notice one if present. It relies wholly on the ability of ssh to spawn a temporary svnserve process, which then terminates when the network connection is closed.

When using svn+ssh:// URLs to access a repository, remember that it's the ssh program prompting for authentication, and not the svn client program. That means there's no automatic password caching going on (see the section called “Client Credentials Caching”). The Subversion client often makes multiple connections to the repository, though users don't normally notice this due to the password caching feature. When using svn+ssh:// URLs, however, users may be annoyed by ssh repeatedly asking for a password for every outbound connection. The solution is to use a separate SSH password-caching tool like ssh-agent on a Unix-like system, or pageant on Windows.

When running over a tunnel, authorization is primarily controlled by operating system permissions to the repository's database files; it's very much the same as if Harry were accessing the repository directly via a file:/// URL. If multiple system users are going to be accessing the repository directly, you may want to place them into a common group, and you'll need to be careful about umasks. (Be sure to read the section called “Supporting Multiple Repository Access Methods”.) But even in the case of tunneling, the svnserve.conf file can still be used to block access, by simply setting auth-access = read or auth-access = none.

You'd think that the story of SSH tunneling would end here, but it doesn't. Subversion allows you to create custom tunnel behaviors in your run-time config file (see the section called “Runtime Configuration Area”). For example, suppose you want to use RSH instead of SSH. In the [tunnels] section of your config file, simply define it like this:

[tunnels]
rsh = rsh

And now, you can use this new tunnel definition by using a URL schema that matches the name of your new variable: svn+rsh://host/path. When using the new URL schema, the Subversion client will actually be running the command rsh host svnserve -t behind the scenes. If you include a username in the URL (for example, svn+rsh://username@host/path) the client will also include that in its command (rsh username@host svnserve -t). But you can define new tunneling schemes to be much more clever than that:

[tunnels]
joessh = $JOESSH /opt/alternate/ssh -p 29934

This example demonstrates a couple of things. First, it shows how to make the Subversion client launch a very specific tunneling binary (the one located at /opt/alternate/ssh) with specific options. In this case, accessing a svn+joessh:// URL would invoke the particular SSH binary with -p 29934 as arguments—useful if you want the tunnel program to connect to a non-standard port.

Second, it shows how to define a custom environment variable that can override the name of the tunneling program. Setting the SVN_SSH environment variable is a convenient way to override the default SSH tunnel agent. But if you need to have several different overrides for different servers, each perhaps contacting a different port or passing a different set of options to SSH, you can use the mechanism demonstrated in this example. Now if we were to set the JOESSH environment variable, its value would override the entire value of the tunnel variable—$JOESSH would be executed instead of /opt/alternate/ssh -p 29934.

It's not only possible to control the way in which the client invokes ssh, but also to control the behavior of sshd on your server machine. In this section, we'll show how to control the exact svnserve command executed by sshd, as well as how to have multiple users share a single system account.

  ssh-dsa AAAABtce9euch.... user@example.com

  command="program" ssh-dsa AAAABtce9euch.... user@example.com

  command="program" TYPE KEY COMMENT

  command="/path/to/svnserve -t -r /virtual/root" TYPE KEY COMMENT

  command="svnserve -t --tunnel-user=harry" TYPE1 KEY1 harry@example.com
  command="svnserve -t --tunnel-user=sally" TYPE2 KEY2 sally@example.com

  command="svnserve -t --tunnel-user=harry",no-port-forwarding,\
           no-agent-forwarding,no-X11-forwarding,no-pty \
           TYPE1 KEY1 harry@example.com

To network your repository over HTTP, you basically need four components, available in two packages. You'll need Apache httpd 2.0, the mod_dav DAV module that comes with it, Subversion, and the mod_dav_svn filesystem provider module distributed with Subversion. Once you have all of those components, the process of networking your repository is as simple as:

You can accomplish the first two items either by compiling httpd and Subversion from source code, or by installing pre-built binary packages of them on your system. For the most up-to-date information on how to compile Subversion for use with the Apache HTTP Server, as well as how to compile and configure Apache itself for this purpose, see the INSTALL file in the top level of the Subversion source code tree.

Once you have all the necessary components installed on your system, all that remains is the configuration of Apache via its httpd.conf file. Instruct Apache to load the mod_dav_svn module using the LoadModule directive. This directive must precede any other Subversion-related configuration items. If your Apache was installed using the default layout, your mod_dav_svn module should have been installed in the modules subdirectory of the Apache install location (often /usr/local/apache2). The LoadModule directive has a simple syntax, mapping a named module to the location of a shared library on disk:

LoadModule dav_svn_module     modules/mod_dav_svn.so

Note that if mod_dav was compiled as a shared object (instead of statically linked directly to the httpd binary), you'll need a similar LoadModule statement for it, too. Be sure that it comes before the mod_dav_svn line:

LoadModule dav_module         modules/mod_dav.so
LoadModule dav_svn_module     modules/mod_dav_svn.so

At a later location in your configuration file, you now need to tell Apache where you keep your Subversion repository (or repositories). The Location directive has an XML-like notation, starting with an opening tag, and ending with a closing tag, with various other configuration directives in the middle. The purpose of the Location directive is to instruct Apache to do something special when handling requests that are directed at a given URL or one of its children. In the case of Subversion, you want Apache to simply hand off support for URLs that point at versioned resources to the DAV layer. You can instruct Apache to delegate the handling of all URLs whose path portions (the part of the URL that follows the server's name and the optional port number) begin with /repos/ to a DAV provider whose repository is located at /absolute/path/to/repository using the following httpd.conf syntax:

<Location /repos>
  DAV svn
  SVNPath /absolute/path/to/repository
</Location>

If you plan to support multiple Subversion repositories that will reside in the same parent directory on your local disk, you can use an alternative directive, the SVNParentPath directive, to indicate that common parent directory. For example, if you know you will be creating multiple Subversion repositories in a directory /usr/local/svn that would be accessed via URLs like http://my.server.com/svn/repos1, http://my.server.com/svn/repos2, and so on, you could use the httpd.conf configuration syntax in the following example:

<Location /svn>
  DAV svn

  # any "/svn/foo" URL will map to a repository /usr/local/svn/foo
  SVNParentPath /usr/local/svn
</Location>

Using the previous syntax, Apache will delegate the handling of all URLs whose path portions begin with /svn/ to the Subversion DAV provider, which will then assume that any items in the directory specified by the SVNParentPath directive are actually Subversion repositories. This is a particularly convenient syntax in that, unlike the use of the SVNPath directive, you don't have to restart Apache in order to create and network new repositories.

Be sure that when you define your new Location, it doesn't overlap with other exported Locations. For example, if your main DocumentRoot is /www, do not export a Subversion repository in <Location /www/repos>. If a request comes in for the URI /www/repos/foo.c, Apache won't know whether to look for a file repos/foo.c in the DocumentRoot, or whether to delegate mod_dav_svn to return foo.c from the Subversion repository.

ServerName svn.example.com

At this stage, you should strongly consider the question of permissions. If you've been running Apache for some time now as your regular web server, you probably already have a collection of content—web pages, scripts and such. These items have already been configured with a set of permissions that allows them to work with Apache, or more appropriately, that allows Apache to work with those files. Apache, when used as a Subversion server, will also need the correct permissions to read and write to your Subversion repository. (See Servers and Permissions: A Word of Warning.)

You will need to determine a permission system setup that satisfies Subversion's requirements without messing up any previously existing web page or script installations. This might mean changing the permissions on your Subversion repository to match those in use by other things that Apache serves for you, or it could mean using the User and Group directives in httpd.conf to specify that Apache should run as the user and group that owns your Subversion repository. There is no single correct way to set up your permissions, and each administrator will have different reasons for doing things a certain way. Just be aware that permission-related problems are perhaps the most common oversight when configuring a Subversion repository for use with Apache.

At this point, if you configured httpd.conf to contain something like

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
</Location>

...then your repository is “anonymously” accessible to the world. Until you configure some authentication and authorization policies, the Subversion repositories you make available via the Location directive will be generally accessible to everyone. In other words,

$ ### First time: use -c to create the file
$ ### Use -m to use MD5 encryption of the password, which is more secure
$ htpasswd -cm /etc/svn-auth-file harry
New password: ***** 
Re-type new password: *****
Adding password for user harry
$ htpasswd -m /etc/svn-auth-file sally
New password: *******
Re-type new password: *******
Adding password for user sally
$

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /etc/svn-auth-file
</Location>

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /etc/svn-auth-file
  Require valid-user
</Location>

$ svn list https://host.example.com/repos/project

Error validating server certificate for 'https://host.example.com:443':
 - The certificate is not issued by a trusted authority. Use the
   fingerprint to validate the certificate manually!
Certificate information:
 - Hostname: host.example.com
 - Valid: from Jan 30 19:23:56 2004 GMT until Jan 30 19:23:56 2006 GMT
 - Issuer: CA, example.com, Sometown, California, US
 - Fingerprint: 7d:e1:a9:34:33:39:ba:6a:e9:a5:c4:22:98:7b:76:5c:92:a0:9c:7b

(R)eject, accept (t)emporarily or accept (p)ermanently?

[global]
ssl-authority-files = /path/to/CAcert1.pem;/path/to/CAcert2.pem

$ svn list https://host.example.com/repos/project

Authentication realm: https://host.example.com:443
Client certificate filename: /path/to/my/cert.p12
Passphrase for '/path/to/my/cert.p12':  ********
…

[groups]
examplehost = host.example.com

[examplehost]
ssl-client-cert-file = /path/to/my/cert.p12
ssl-client-cert-password = somepassword

At this point, you've configured authentication, but not authorization. Apache is able to challenge clients and confirm identities, but it has not been told how to allow or restrict access to the clients bearing those identities. This section describes two strategies for controlling access to your repositories.

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn

  # how to authenticate a user
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file
  
  # only authenticated users may access the repository
  Require valid-user
</Location>

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn

  # how to authenticate a user
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file

  # For any operations other than these, require an authenticated user.
  <LimitExcept GET PROPFIND OPTIONS REPORT>
    Require valid-user
  </LimitExcept>
</Location>

LoadModule dav_module         modules/mod_dav.so
LoadModule dav_svn_module     modules/mod_dav_svn.so
LoadModule authz_svn_module   modules/mod_authz_svn.so

<Location /repos>
  DAV svn
  SVNParentPath /usr/local/svn

  # our access control policy
  AuthzSVNAccessFile /path/to/access/file                 
</Location>
          

<Location /repos>
  DAV svn
  SVNParentPath /usr/local/svn
            
  # our access control policy
  AuthzSVNAccessFile /path/to/access/file                 
            
  # only authenticated users may access the repository
  Require valid-user
            
  # how to authenticate a user
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file                  
</Location>
          

<Location /repos>
  DAV svn
  SVNParentPath /usr/local/svn
            
  # our access control policy
  AuthzSVNAccessFile /path/to/access/file                 
            
  # try anonymous access first, resort to real 
  # authentication if necessary.
  Satisfy Any
  Require valid-user
            
  # how to authenticate a user
  AuthType Basic
  AuthName "Subversion repository"
  AuthUserFile /path/to/users/file                  
</Location>
          

[calc:/branches/calc/bug-142]
harry = rw
sally = r

[calc:/branches/calc/bug-142]
harry = rw
sally = r

# give sally write access only to the 'testing' subdir
[calc:/branches/calc/bug-142/testing]
sally = rw

[calc:/branches/calc/bug-142]
harry = rw
sally = r

[calc:/branches/calc/bug-142/secret]
harry =

[/]
* = r

[groups]
calc-developers = harry, sally, joe
paint-developers = frank, sally, jane
everyone = harry, sally, joe, frank, sally, jane

[calc:/projects/calc]
@calc-developers = rw

[paint:/projects/paint]
@paint-developers = rw
jane = r 

[groups]
calc-developers = harry, sally, joe
paint-developers = frank, sally, jane
everyone = @calc-developers, @paint-developers

<Location /repos>
  DAV svn
  SVNParentPath /usr/local/svn
            
  SVNPathAuthz off
</Location>            
          

We've covered most of the authentication and authorization options for Apache and mod_dav_svn. But there are a few other nice features that Apache provides.

<Location /svn>
  DAV svn
  SVNParentPath /usr/local/svn
  SVNIndexXSLT "/svnindex.xsl"
  …
</Location>

The first time that the svn command-line client is executed, it creates a per-user configuration area. On Unix-like systems, this area appears as a directory named .subversion in the user's home directory. On Win32 systems, Subversion creates a folder named Subversion, typically inside the Application Data area of the user's profile directory (which, by the way, is usually a hidden directory). However, on this platform the exact location differs from system to system, and is dictated by the Windows registry. ^[29] We will refer to the per-user configuration area using its Unix name, .subversion.

In addition to the per-user configuration area, Subversion also recognizes the existence of a system-wide configuration area. This gives system administrators the ability to establish defaults for all users on a given machine. Note that the system-wide configuration area does not alone dictate mandatory policy—the settings in the per-user configuration area override those in the system-wide one, and command-line arguments supplied to the svn program have the final word on behavior. On Unix-like platforms, the system-wide configuration area is expected to be the /etc/subversion directory; on Windows machines, it looks for a Subversion directory inside the common Application Data location (again, as specified by the Windows Registry). Unlike the per-user case, the svn program does not attempt to create the system-wide configuration area.

The configuration area currently contains three files—two configuration files (config and servers), and a README.txt file which describes the INI format. At the time of their creation, the files contain default values for each of the supported Subversion options, mostly commented out and grouped with textual descriptions about how the values for the key affect Subversion's behavior. To change a certain behavior, you need only to load the appropriate configuration file into a text editor, and modify the desired option's value. If at any time you wish to have the default configuration settings restored, you can simply remove (or rename) your configuration directory and then run some innocuous svn command, such as svn --version. A new configuration directory with the default contents will be created.

The per-user configuration area also contains a cache of authentication data. The auth directory holds a set of subdirectories that contain pieces of cached information used by Subversion's various supported authentication methods. This directory is created in such a way that only the user herself has permission to read its contents.

In addition to the usual INI-based configuration area, Subversion clients running on Windows platforms may also use the Windows registry to hold the configuration data. The option names and their values are the same as in the INI files. The “file/section” hierarchy is preserved as well, though addressed in a slightly different fashion—in this schema, files and sections are just levels in the registry key tree.

Subversion looks for system-wide configuration values under the HKEY_LOCAL_MACHINE\Software\Tigris.org\Subversion key. For example, the global-ignores option, which is in the miscellany section of the config file, would be found at HKEY_LOCAL_MACHINE\Software\Tigris.org\Subversion\Config\Miscellany\global-ignores. Per-user configuration values should be stored under HKEY_CURRENT_USER\Software\Tigris.org\Subversion.

Registry-based configuration options are parsed before their file-based counterparts, so are overridden by values found in the configuration files. In other words, configuration priority is granted in the following order on a Windows system:

Also, the Windows Registry doesn't really support the notion of something being “commented out”. However, Subversion will ignore any option key whose name begins with a hash (#) character. This allows you to effectively comment out a Subversion option without deleting the entire key from the Registry, obviously simplifying the process of restoring that option.

The svn command-line client never attempts to write to the Windows Registry, and will not attempt to create a default configuration area there. You can create the keys you need using the REGEDIT program. Alternatively, you can create a .reg file, and then double-click on that file from the Explorer shell, which will cause the data to be merged into your registry.

REGEDIT4

[HKEY_LOCAL_MACHINE\Software\Tigris.org\Subversion\Servers\groups]

[HKEY_LOCAL_MACHINE\Software\Tigris.org\Subversion\Servers\global]
"#http-proxy-host"=""
"#http-proxy-port"=""
"#http-proxy-username"=""
"#http-proxy-password"=""
"#http-proxy-exceptions"=""
"#http-timeout"="0"
"#http-compression"="yes"
"#neon-debug-mask"=""
"#ssl-authority-files"=""
"#ssl-trust-default-ca"=""
"#ssl-client-cert-file"=""
"#ssl-client-cert-password"=""

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\auth]
"#store-auth-creds"="no"

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\helpers]
"#editor-cmd"="notepad"
"#diff-cmd"=""
"#diff3-cmd"=""
"#diff3-has-program-arg"=""

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\miscellany]
"#global-ignores"="*.o *.lo *.la #*# .*.rej *.rej .*~ *~ .#* .DS_Store"
"#log-encoding"=""
"#use-commit-times"=""
"#template-root"=""
"#enable-auto-props"=""

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\tunnels]

[HKEY_CURRENT_USER\Software\Tigris.org\Subversion\Config\auto-props]

The previous example shows the contents of a .reg file which contains some of the most commonly used configuration options and their default values. Note the presence of both system-wide (for network proxy-related options) and per-user settings (editor programs and password storage, among others). Also note that all the options are effectively commented out. You need only to remove the hash (#) character from the beginning of the option names, and set the values as you desire.

In this section, we will discuss the specific run-time configuration options that are currently supported by Subversion.

[groups]
beanie-babies = *.red-bean.com
collabnet = svn.collab.net

[beanie-babies]
…

[collabnet]
…