Version Control with Subversion

All version control systems have to solve the same fundamental problem: how will the system allow users to share information, but prevent them from accidentally stepping on each other's feet? It's all too easy for users to accidentally overwrite each other's changes in the repository.

Consider the scenario shown in Figure 1.2, “The problem to avoid”. Suppose we have two coworkers, Harry and Sally. They each decide to edit the same repository file at the same time. If Harry saves his changes to the repository first, it's possible that (a few moments later) Sally could accidentally overwrite them with her own new version of the file. While Harry's version of the file won't be lost forever (because the system remembers every change), any changes Harry made won't be present in Sally's newer version of the file, because she never saw Harry's changes to begin with. Harry's work is still effectively lost—or at least missing from the latest version of the file—and probably by accident. This is definitely a situation we want to avoid!

Figure 1.2. The problem to avoid

Many version control systems use a lock-modify-unlock model to address the problem of many authors clobbering each other's work. In this model, the repository allows only one person to change a file at a time. This exclusivity policy is managed using locks. Harry must “lock” a file before he can begin making changes to it. If Harry has locked a file, Sally cannot also lock it, and therefore cannot make any changes to that file. All she can do is read the file and wait for Harry to finish his changes and release his lock. After Harry unlocks the file, Sally can take her turn by locking and editing the file. Figure 1.3, “The lock-modify-unlock solution” demonstrates this simple solution.

Figure 1.3. The lock-modify-unlock solution

The problem with the lock-modify-unlock model is that it's a bit restrictive and often becomes a roadblock for users:

Subversion, CVS, and many other version control systems use a copy-modify-merge model as an alternative to locking. In this model, each user's client contacts the project repository and creates a personal working copy—a local reflection of the repository's files and directories. Users then work simultaneously and independently, modifying their private copies. Finally, the private copies are merged together into a new, final version. The version control system often assists with the merging, but ultimately, a human being is responsible for making it happen correctly.

Here's an example. Say that Harry and Sally each create working copies of the same project, copied from the repository. They work concurrently and make changes to the same file A within their copies. Sally saves her changes to the repository first. When Harry attempts to save his changes later, the repository informs him that his file A is out of date. In other words, file A in the repository has somehow changed since he last copied it. So Harry asks his client to merge any new changes from the repository into his working copy of file A. Chances are that Sally's changes don't overlap with his own; once he has both sets of changes integrated, he saves his working copy back to the repository. Figure 1.4, “The copy-modify-merge solution” and Figure 1.5, “The copy-modify-merge solution (continued)” show this process.

Figure 1.4. The copy-modify-merge solution

Figure 1.5. The copy-modify-merge solution (continued)

But what if Sally's changes do overlap with Harry's changes? What then? This situation is called a conflict, and it's usually not much of a problem. When Harry asks his client to merge the latest repository changes into his working copy, his copy of file A is somehow flagged as being in a state of conflict: he'll be able to see both sets of conflicting changes and manually choose between them. Note that software can't automatically resolve conflicts; only humans are capable of understanding and making the necessary intelligent choices. Once Harry has manually resolved the overlapping changes—perhaps after a discussion with Sally—he can safely save the merged file back to the repository.

The copy-modify-merge model may sound a bit chaotic, but in practice, it runs extremely smoothly. Users can work in parallel, never waiting for one another. When they work on the same files, it turns out that most of their concurrent changes don't overlap at all; conflicts are infrequent. And the amount of time it takes to resolve conflicts is usually far less than the time lost by a locking system.

In the end, it all comes down to one critical factor: user communication. When users communicate poorly, both syntactic and semantic conflicts increase. No system can force users to communicate perfectly, and no system can detect semantic conflicts. So there's no point in being lulled into a false sense of security that a locking system will somehow prevent conflicts; in practice, locking seems to inhibit productivity more than anything else.

Throughout this book, Subversion uses URLs to identify versioned files and directories in Subversion repositories. For the most part, these URLs use the standard syntax, allowing for server names and port numbers to be specified as part of the URL:

$ svn checkout http://svn.example.com:9834/repos
…

But there are some nuances in Subversion's handling of URLs that are notable. For example, URLs containing the file:// access method (used for local repositories) must, in accordance with convention, have either a server name of localhost or no server name at all:

$ svn checkout file:///var/svn/repos
…
$ svn checkout file://localhost/var/svn/repos
…

Also, users of the file:// scheme on Windows platforms will need to use an unofficially “standard” syntax for accessing repositories that are on the same machine, but on a different drive than the client's current working drive. Either of the two following URL path syntaxes will work, where X is the drive on which the repository resides:

C:\> svn checkout file:///X:/var/svn/repos
…
C:\> svn checkout "file:///X|/var/svn/repos"
…

In the second syntax, you need to quote the URL so that the vertical bar character is not interpreted as a pipe. Also, note that a URL uses forward slashes even though the native (non-URL) form of a path on Windows uses backslashes.

Finally, it should be noted that the Subversion client will automatically encode URLs as necessary, just like a web browser does. For example, if a URL contains a space or upper-ASCII character as in the following:

$ svn checkout "http://host/path with space/project/españa"

then Subversion will escape the unsafe characters and behave as though you had typed:

$ svn checkout http://host/path%20with%20space/project/espa%C3%B1a

If the URL contains spaces, be sure to place it within quotation marks so that your shell treats the whole thing as a single argument to the svn program.

You've already read about working copies; now we'll demonstrate how the Subversion client creates and uses them.

A Subversion working copy is an ordinary directory tree on your local system, containing a collection of files. You can edit these files however you wish, and if they're source code files, you can compile your program from them in the usual way. Your working copy is your own private work area: Subversion will never incorporate other people's changes, nor make your own changes available to others, until you explicitly tell it to do so. You can even have multiple working copies of the same project.

After you've made some changes to the files in your working copy and verified that they work properly, Subversion provides you with commands to “publish” your changes to the other people working with you on your project (by writing to the repository). If other people publish their own changes, Subversion provides you with commands to merge those changes into your working copy (by reading from the repository).

A working copy also contains some extra files, created and maintained by Subversion, to help it carry out these commands. In particular, each directory in your working copy contains a subdirectory named .svn, also known as the working copy's administrative directory. The files in each administrative directory help Subversion recognize which files contain unpublished changes, and which files are out of date with respect to others' work.

A typical Subversion repository often holds the files (or source code) for several projects; usually, each project is a subdirectory in the repository's filesystem tree. In this arrangement, a user's working copy will usually correspond to a particular subtree of the repository.

For example, suppose you have a repository that contains two software projects, paint and calc. Each project lives in its own top-level subdirectory, as shown in Figure 1.6, “The repository's filesystem”.

Figure 1.6. The repository's filesystem

To get a working copy, you must check out some subtree of the repository. (The term check out may sound like it has something to do with locking or reserving resources, but it doesn't; it simply creates a private copy of the project for you.) For example, if you check out /calc, you will get a working copy like this:

$ svn checkout http://svn.example.com/repos/calc
A    calc/Makefile
A    calc/integer.c
A    calc/button.c
Checked out revision 56.

$ ls -A calc
Makefile  button.c integer.c .svn/

The list of letter As in the left margin indicates that Subversion is adding a number of items to your working copy. You now have a personal copy of the repository's /calc directory, with one additional entry—.svn—which holds the extra information needed by Subversion, as mentioned earlier.

Suppose you make changes to button.c. Since the .svn directory remembers the file's original modification date and contents, Subversion can tell that you've changed the file. However, Subversion does not make your changes public until you explicitly tell it to. The act of publishing your changes is more commonly known as committing (or checking in) changes to the repository.

To publish your changes to others, you can use Subversion's svn commit command:

$ svn commit button.c -m "Fixed a typo in button.c."
Sending        button.c
Transmitting file data .
Committed revision 57.

Now your changes to button.c have been committed to the repository, with a note describing your change (namely, that you fixed a typo). If another user checks out a working copy of /calc, she will see your changes in the latest version of the file.

Suppose you have a collaborator, Sally, who checked out a working copy of /calc at the same time you did. When you commit your change to button.c, Sally's working copy is left unchanged; Subversion modifies working copies only at the user's request.

To bring her project up to date, Sally can ask Subversion to update her working copy, by using the svn update command. This will incorporate your changes into her working copy, as well as any others that have been committed since she checked it out.

$ pwd
/home/sally/calc

$ ls -A
Makefile button.c integer.c .svn/

$ svn update
U    button.c
Updated to revision 57.

The output from the svn update command indicates that Subversion updated the contents of button.c. Note that Sally didn't need to specify which files to update; Subversion uses the information in the .svn directory as well as further information in the repository, to decide which files need to be brought up to date.

An svn commit operation publishes changes to any number of files and directories as a single atomic transaction. In your working copy, you can change files' contents; create, delete, rename, and copy files and directories; and then commit a complete set of changes as an atomic transaction.

By atomic transaction, we mean simply this: either all of the changes happen in the repository, or none of them happens. Subversion tries to retain this atomicity in the face of program crashes, system crashes, network problems, and other users' actions.

Each time the repository accepts a commit, this creates a new state of the filesystem tree, called a revision. Each revision is assigned a unique natural number, one greater than the number of the previous revision. The initial revision of a freshly created repository is numbered 0 and consists of nothing but an empty root directory.

Figure 1.7, “The repository” illustrates a nice way to visualize the repository. Imagine an array of revision numbers, starting at 0, stretching from left to right. Each revision number has a filesystem tree hanging below it, and each tree is a “snapshot” of the way the repository looked after a commit.

Figure 1.7. The repository

It's important to note that working copies do not always correspond to any single revision in the repository; they may contain files from several different revisions. For example, suppose you check out a working copy from a repository whose most recent revision is 4:

calc/Makefile:4
     integer.c:4
     button.c:4

At the moment, this working directory corresponds exactly to revision 4 in the repository. However, suppose you make a change to button.c, and commit that change. Assuming no other commits have taken place, your commit will create revision 5 of the repository, and your working copy will now look like this:

calc/Makefile:4
     integer.c:4
     button.c:5

Suppose that, at this point, Sally commits a change to integer.c, creating revision 6. If you use svn update to bring your working copy up to date, it will look like this:

calc/Makefile:6
     integer.c:6
     button.c:6

Sally's change to integer.c will appear in your working copy, and your change will still be present in button.c. In this example, the text of Makefile is identical in revisions 4, 5, and 6, but Subversion will mark your working copy of Makefile with revision 6 to indicate that it is still current. So, after you do a clean update at the top of your working copy, it will generally correspond to exactly one revision in the repository.

For each file in a working directory, Subversion records two essential pieces of information in the .svn/ administrative area:

Given this information, by talking to the repository, Subversion can tell which of the following four states a working file is in:

This may sound like a lot to keep track of, but the svn status command will show you the state of any item in your working copy. For more information on that command, see the section called “See an overview of your changes”.

As a general principle, Subversion tries to be as flexible as possible. One special kind of flexibility is the ability to have a working copy containing files and directories with a mix of different working revision numbers. Unfortunately, this flexibility tends to confuse a number of new users. If the earlier example showing mixed revisions perplexed you, here's a primer on why the feature exists and how to make use of it.

The svn import command is a quick way to copy an unversioned tree of files into a repository, creating intermediate directories as necessary. svn import doesn't require a working copy, and your files are immediately committed to the repository. You typically use this when you have an existing tree of files that you want to begin tracking in your Subversion repository. For example:

$ svnadmin create /var/svn/newrepos
$ svn import mytree file:///var/svn/newrepos/some/project \
             -m "Initial import"
Adding         mytree/foo.c
Adding         mytree/bar.c
Adding         mytree/subdir
Adding         mytree/subdir/quux.h

Committed revision 1.

The previous example copied the contents of directory mytree under the directory some/project in the repository:

$ svn list file:///var/svn/newrepos/some/project
bar.c
foo.c
subdir/

Note that after the import is finished, the original tree is not converted into a working copy. To start working, you still need to svn checkout a fresh working copy of the tree.

While Subversion's flexibility allows you to lay out your repository in any way that you choose, we recommend that you create a trunk directory to hold the “main line” of development, a branches directory to contain branch copies, and a tags directory to contain tag copies. For example:

$ svn list file:///var/svn/repos
/trunk
/branches
/tags

You'll learn more about tags and branches in Chapter 4, Branching and Merging. For details and how to set up multiple projects, see the section called “Repository Layout” and the section called “Planning Your Repository Organization” to read more about project roots.

When you perform a Subversion operation that requires you to authenticate, by default Subversion caches your authentication credentials on disk. This is done for convenience so that you don't have to continually reenter your password for future operations. If you're concerned about caching your Subversion passwords, ^[3] you can disable caching either permanently or on a case-by-case basis.

To disable password caching for a particular one-time command, pass the --no-auth-cache option on the command line. To permanently disable caching, you can add the line store-passwords = no to your local machine's Subversion configuration file. See the section called “Client Credentials Caching” for details.

Since Subversion caches auth credentials by default (both username and password), it conveniently remembers who you were acting as the last time you modified your working copy. But sometimes that's not helpful—particularly if you're working in a shared working copy such as a system configuration directory or a web server document root. In this case, just pass the --username option on the command line, and Subversion will attempt to authenticate as that user, prompting you for a password if necessary.

When working on a project with a team, you'll want to update your working copy to receive any changes other developers on the project have made since your last update. Use svn update to bring your working copy into sync with the latest revision in the repository:

$ svn update
U  foo.c
U  bar.c
Updated to revision 2.

In this case, it appears that someone checked in modifications to both foo.c and bar.c since the last time you updated, and Subversion has updated your working copy to include those changes.

When the server sends changes to your working copy via svn update, a letter code is displayed next to each item to let you know what actions Subversion performed to bring your working copy up to date. To find out what these letters mean, run svn help update.

Now you can get to work and make changes in your working copy. It's usually most convenient to decide on a discrete change (or set of changes) to make, such as writing a new feature, fixing a bug, and so on. The Subversion commands that you will use here are svn add, svn delete, svn copy, svn move, and svn mkdir. However, if you are merely editing files that are already in Subversion, you may not need to use any of these commands until you commit.

You can make two kinds of changes to your working copy: file changes and tree changes. You don't need to tell Subversion that you intend to change a file; just make your changes using your text editor, word processor, graphics program, or whatever tool you would normally use. Subversion automatically detects which files have been changed, and in addition, it handles binary files just as easily as it handles text files—and just as efficiently, too. For tree changes, you can ask Subversion to “mark” files and directories for scheduled removal, addition, copying, or moving. These changes may take place immediately in your working copy, but no additions or removals will happen in the repository until you commit them.

Here is an overview of the five Subversion subcommands that you'll use most often to make tree changes:

Once you've finished making changes, you need to commit them to the repository, but before you do so, it's usually a good idea to take a look at exactly what you've changed. By examining your changes before you commit, you can make a more accurate log message. You may also discover that you've inadvertently changed a file, and this gives you a chance to revert those changes before committing. Additionally, this is a good opportunity to review and scrutinize changes before publishing them. You can see an overview of the changes you've made by using svn status, and dig into the details of those changes by using svn diff.

Subversion has been optimized to help you with this task, and it is able to do many things without communicating with the repository. In particular, your working copy contains a hidden cached “pristine” copy of each version-controlled file within the .svn area. Because of this, Subversion can quickly show you how your working files have changed or even allow you to undo your changes without contacting the repository.

?       scratch.c           # file is not under version control
A       stuff/loot/bloo.h   # file is scheduled for addition
C       stuff/loot/lump.c   # file has textual conflicts from an update
D       stuff/fish.c        # file is scheduled for deletion
M       bar.c               # the content in bar.c has local modifications

$ svn status stuff/fish.c
D      stuff/fish.c

$ svn status -v
M               44        23    sally     README
                44        30    sally     INSTALL
M               44        20    harry     bar.c
                44        18    ira       stuff
                44        35    harry     stuff/trout.c
D               44        19    ira       stuff/fish.c
                44        21    sally     stuff/things
A                0         ?     ?        stuff/things/bloo.h
                44        36    harry     stuff/things/gloo.c

$ svn status -u -v
M      *        44        23    sally     README
M               44        20    harry     bar.c
       *        44        35    harry     stuff/trout.c
D               44        19    ira       stuff/fish.c
A                0         ?     ?        stuff/things/bloo.h
Status against revision:   46

$ svn diff
Index: bar.c
===================================================================
--- bar.c	(revision 3)
+++ bar.c	(working copy)
@@ -1,7 +1,12 @@
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+#include <stdio.h>

 int main(void) {
-  printf("Sixty-four slices of American Cheese...\n");
+  printf("Sixty-five slices of American Cheese...\n");
 return 0;
 }

Index: README
===================================================================
--- README	(revision 3)
+++ README	(working copy)
@@ -193,3 +193,4 @@
+Note to self:  pick up laundry.

Index: stuff/fish.c
===================================================================
--- stuff/fish.c	(revision 1)
+++ stuff/fish.c	(working copy)
-Welcome to the file known as 'fish'.
-Information on fish will be here soon.

Index: stuff/things/bloo.h
===================================================================
--- stuff/things/bloo.h	(revision 8)
+++ stuff/things/bloo.h	(working copy)
+Here is a new file to describe
+things about bloo.

$ svn diff > patchfile

Suppose while viewing the output of svn diff you determine that all the changes you made to a particular file are mistakes. Maybe you shouldn't have changed the file at all, or perhaps it would be easier to make different changes starting from scratch.

This is a perfect opportunity to use svn revert:

$ svn revert README
Reverted 'README'

Subversion reverts the file to its premodified state by overwriting it with the cached “pristine” copy from the .svn area. But also note that svn revert can undo any scheduled operations—for example, you might decide that you don't want to add a new file after all:

$ svn status foo
?      foo

$ svn add foo
A         foo

$ svn revert foo
Reverted 'foo'

$ svn status foo
?      foo

Or perhaps you mistakenly removed a file from version control:

$ svn status README

$ svn delete README
D         README

$ svn revert README
Reverted 'README'

$ svn status README

We've already seen how svn status -u can predict conflicts. Suppose you run svn update and some interesting things occur:

$ svn update
U  INSTALL
G  README
Conflict discovered in 'bar.c'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (h) help for more options:

The U and G codes are no cause for concern; those files cleanly absorbed changes from the repository. The files marked with U contained no local changes but were Updated with changes from the repository. The G stands for merGed, which means that the file had local changes to begin with, but the changes coming from the repository didn't overlap with the local changes.

But the next two lines are part of a feature (new in Subversion 1.5) called interactive conflict resolution. This means that the changes from the server overlapped with your own, and you have the opportunity to resolve this conflict. The most commonly used options are displayed, but you can see all of the options by typing h:

…
  (p)  postpone    - mark the conflict to be resolved later
  (df) diff-full   - show all changes made to merged file
  (e)  edit        - change merged file in an editor
  (r)  resolved    - accept merged version of file
  (mf) mine-full   - accept my version of entire file (ignore their changes)
  (tf) theirs-full - accept their version of entire file (lose my changes)
  (l)  launch      - launch external tool to resolve conflict
  (h)  help        - show this list

Let's briefly review each of these options before we go into detail on what each option means.

We'll cover these commands in more detail now, grouping them together by related functionality.

…
Select: (p) postpone, (df) diff-full, (e) edit,
        (h)elp for more options : d
--- .svn/text-base/sandwich.txt.svn-base      Tue Dec 11 21:33:57 2007
+++ .svn/tmp/tempfile.32.tmp     Tue Dec 11 21:34:33 2007
@@ -1 +1,5 @@
-Just buy a sandwich.
+<<<<<<< .mine
+Go pick up a cheesesteak.
+=======
+Bring me a taco!
+>>>>>>> .r32
…

$ svn update
Conflict discovered in 'sandwich.txt'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (h)elp for more options : p
C  sandwich.txt
Updated to revision 2.
$ ls -1
sandwich.txt
sandwich.txt.mine
sandwich.txt.r1
sandwich.txt.r2

$ svn commit -m "Add a few more things"
svn: Commit failed (details follow):
svn: Aborting commit: '/home/sally/svn-work/sandwich.txt' remains in conflict

$ svn resolve --accept working sandwich.txt
Resolved conflicted state of 'sandwich.txt'

$ cat sandwich.txt
Top piece of bread
Mayonnaise
Lettuce
Tomato
Provolone
<<<<<<< .mine
Salami
Mortadella
Prosciutto
=======
Sauerkraut
Grilled Chicken
>>>>>>> .r2
Creole Mustard
Bottom piece of bread

<<<<<<< .mine
Salami
Mortadella
Prosciutto
=======

=======
Sauerkraut
Grilled Chicken
>>>>>>> .r2

Top piece of bread
Mayonnaise
Lettuce
Tomato
Provolone
Salami
Mortadella
Prosciutto
Creole Mustard
Bottom piece of bread

$ svn resolve --accept working sandwich.txt
Resolved conflicted state of 'sandwich.txt'
$ svn commit -m "Go ahead and use my sandwich, discarding Sally's edits."

$ svn update
Conflict discovered in 'sandwich.txt'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (h) help for more options: p
C    sandwich.txt
Updated to revision 2.
$ ls sandwich.*
sandwich.txt  sandwich.txt.mine  sandwich.txt.r2  sandwich.txt.r1
$ svn resolve --accept theirs-full sandwich.txt
Resolved conflicted state of 'sandwich.txt'

$ svn revert sandwich.txt
Reverted 'sandwich.txt'
$ ls sandwich.*
sandwich.txt

Finally! Your edits are finished, you've merged all changes from the server, and you're ready to commit your changes to the repository.

The svn commit command sends all of your changes to the repository. When you commit a change, you need to supply a log message describing your change. Your log message will be attached to the new revision you create. If your log message is brief, you may wish to supply it on the command line using the --message (or -m) option:

$ svn commit -m "Corrected number of cheese slices."
Sending        sandwich.txt
Transmitting file data .
Committed revision 3.

However, if you've been composing your log message as you work, you may want to tell Subversion to get the message from a file by passing the filename with the --file (-F) option:

$ svn commit -F logmsg
Sending        sandwich.txt
Transmitting file data .
Committed revision 4.

If you fail to specify either the --message or --file option, Subversion will automatically launch your favorite editor (see the information on editor-cmd in the section called “Config”) for composing a log message.

$ svn commit
Waiting for Emacs...Done

Log message unchanged or not specified
(a)bort, (c)ontinue, (e)dit
a
$

The repository doesn't know or care whether your changes make any sense as a whole; it checks only to make sure nobody else has changed any of the same files that you did when you weren't looking. If somebody has done that, the entire commit will fail with a message informing you that one or more of your files are out of date:

$ svn commit -m "Add another rule"
Sending        rules.txt
svn: Commit failed (details follow):
svn: File '/sandwich.txt' is out of date
…

(The exact wording of this error message depends on the network protocol and server you're using, but the idea is the same in all cases.)

At this point, you need to run svn update, deal with any merges or conflicts that result, and attempt your commit again.

That covers the basic work cycle for using Subversion. Subversion offers many other features that you can use to manage your repository and working copy, but most of your day-to-day use of Subversion will involve only the commands that we've discussed so far in this chapter. We will, however, cover a few more commands that you'll use fairly often.

To find information about the history of a file or directory, use the svn log command. svn log will provide you with a record of who made changes to a file or directory, at what revision it changed, the time and date of that revision, and—if it was provided—the log message that accompanied the commit:

$ svn log
------------------------------------------------------------------------
r3 | sally | 2008-05-15 23:09:28 -0500 (Thu, 15 May 2008) | 1 line

Added include lines and corrected # of cheese slices.
------------------------------------------------------------------------
r2 | harry | 2008-05-14 18:43:15 -0500 (Wed, 14 May 2008) | 1 line

Added main() methods.
------------------------------------------------------------------------
r1 | sally | 2008-05-10 19:50:31 -0500 (Sat, 10 May 2008) | 1 line

Initial import
------------------------------------------------------------------------

Note that the log messages are printed in reverse chronological order by default. If you wish to see a different range of revisions in a particular order or just a single revision, pass the --revision (-r) option:

$ svn log -r 5:19    # shows logs 5 through 19 in chronological order

$ svn log -r 19:5    # shows logs 5 through 19 in reverse order

$ svn log -r 8       # shows log for revision 8

You can also examine the log history of a single file or directory. For example:

$ svn log foo.c
…
$ svn log http://foo.com/svn/trunk/code/foo.c
…

These will display log messages only for those revisions in which the working file (or URL) changed.

If you want even more information about a file or directory, svn log also takes a --verbose (-v) option. Because Subversion allows you to move and copy files and directories, it is important to be able to track path changes in the filesystem. So, in verbose mode, svn log will include a list of changed paths in a revision in its output:

$ svn log -r 8 -v
------------------------------------------------------------------------
r8 | sally | 2008-05-21 13:19:25 -0500 (Wed, 21 May 2008) | 1 line
Changed paths:
   M /trunk/code/foo.c
   M /trunk/code/bar.h
   A /trunk/code/doc/README

Frozzled the sub-space winch.

------------------------------------------------------------------------

svn log also takes a --quiet (-q) option, which suppresses the body of the log message. When combined with --verbose, it gives just the names of the changed files.

$ svn log -r 2
------------------------------------------------------------------------
$

We've already seen svn diff before—it displays file differences in unified diff format; we used it to show the local modifications made to our working copy before committing to the repository.

In fact, it turns out that there are three distinct uses of svn diff:

$ svn diff
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth open
+Chew with your mouth closed
+Listen when others are speaking
$

$ svn diff -r 3 rules.txt
Index: rules.txt
===================================================================
--- rules.txt	(revision 3)
+++ rules.txt	(working copy)
@@ -1,4 +1,5 @@
 Be kind to others
 Freedom = Responsibility
 Everything in moderation
-Chew with your mouth open
+Chew with your mouth closed
+Listen when others are speaking
$

$ svn diff -r 2:3 rules.txt
Index: rules.txt
===================================================================
--- rules.txt	(revision 2)
+++ rules.txt	(revision 3)
@@ -1,4 +1,4 @@
 Be kind to others
-Freedom = Chocolate Ice Cream
+Freedom = Responsibility
 Everything in moderation
 Chew with your mouth open
$

$ svn diff -c 3 rules.txt
Index: rules.txt
===================================================================
--- rules.txt	(revision 2)
+++ rules.txt	(revision 3)
@@ -1,4 +1,4 @@
 Be kind to others
-Freedom = Chocolate Ice Cream
+Freedom = Responsibility
 Everything in moderation
 Chew with your mouth open
$

$ svn diff -c 5 http://svn.example.com/repos/example/trunk/text/rules.txt
…
$

Using svn cat and svn list, you can view various revisions of files and directories without changing the working revision of your working copy. In fact, you don't even need a working copy to use either one.

$ svn cat -r 2 rules.txt
Be kind to others
Freedom = Chocolate Ice Cream
Everything in moderation
Chew with your mouth open
$

$ svn cat -r 2 rules.txt > rules.txt.v2
$

$ svn list http://svn.collab.net/repos/svn
README
branches/
clients/
tags/
trunk/

$ svn list -v http://svn.collab.net/repos/svn
  20620 harry            1084 Jul 13  2006 README
  23339 harry                 Feb 04 01:40 branches/
  21282 sally                 Aug 27 09:41 developer-resources/
  23198 harry                 Jan 23 17:17 tags/
  23351 sally                 Feb 05 13:26 trunk/

In addition to all of the previous commands, you can use svn update and svn checkout with the --revision option to take an entire working copy “back in time”: ^[7]

$ svn checkout -r 1729 # Checks out a new working copy at r1729
…
$ svn update -r 1729 # Updates an existing working copy to r1729
…

Lastly, if you're building a release and wish to bundle up your files from Subversion but don't want those pesky .svn directories in the way, you can use svn export to create a local copy of all or part of your repository sans .svn directories. As with svn update and svn checkout, you can also pass the --revision option to svn export:

$ svn export http://svn.example.com/svn/repos1 # Exports latest revision
…
$ svn export http://svn.example.com/svn/repos1 -r 1729
# Exports revision r1729
…

Subversion doesn't track either the state or the existence of working copies on the server, so there's no server overhead to keeping working copies around. Likewise, there's no need to let the server know that you're going to delete a working copy.

If you're likely to use a working copy again, there's nothing wrong with just leaving it on disk until you're ready to use it again, at which point all it takes is an svn update to bring it up to date and ready for use.

However, if you're definitely not going to use a working copy again, you can safely delete the entire thing, but you'd be well served to take a look through the working copy for unversioned files. To find these files, run svn status and review any files that are prefixed with a ? to make certain that they're not of importance. After you're done reviewing, you can safely delete your working copy.

When Subversion modifies your working copy (or any information within .svn), it tries to do so as safely as possible. Before changing the working copy, Subversion writes its intentions to a logfile. Next, it executes the commands in the logfile to apply the requested change, holding a lock on the relevant part of the working copy while it works—to prevent other Subversion clients from accessing the working copy mid-change. Finally, Subversion removes the logfile. Architecturally, this is similar to a journaled filesystem. If a Subversion operation is interrupted (e.g, if the process is killed or if the machine crashes), the logfiles remain on disk. By reexecuting the logfiles, Subversion can complete the previously started operation, and your working copy can get itself back into a consistent state.

And this is exactly what svn cleanup does: it searches your working copy and runs any leftover logs, removing working copy locks in the process. If Subversion ever tells you that some part of your working copy is “locked,” this is the command that you should run. Also, svn status will display an L next to locked items:

$ svn status
  L    somedir
M      somedir/foo.c

$ svn cleanup
$ svn status
M      somedir/foo.c

Don't confuse these working copy locks with the ordinary locks that Subversion users create when using the lock-modify-unlock model of concurrent version control; see the sidebar The Three Meanings of “Lock” for clarification.

The Subversion client understands a number of revision keywords. These keywords can be used instead of integer arguments to the --revision (-r) option, and are resolved into specific revision numbers by Subversion:

As can be derived from their descriptions, the PREV, BASE, and COMMITTED revision keywords are used only when referring to a working copy path—they don't apply to repository URLs. HEAD, on the other hand, can be used in conjunction with both of these path types.

Here are some examples of revision keywords in action:

$ svn diff -r PREV:COMMITTED foo.c
# shows the last change committed to foo.c

$ svn log -r HEAD
# shows log message for the latest repository commit

$ svn diff -r HEAD
# compares your working copy (with all of its local changes) to the
# latest version of that tree in the repository

$ svn diff -r BASE:HEAD foo.c
# compares the unmodified version of foo.c with the latest version of
# foo.c in the repository

$ svn log -r BASE:HEAD
# shows all commit logs for the current versioned directory since you
# last updated

$ svn update -r PREV foo.c
# rewinds the last change on foo.c, decreasing foo.c's working revision

$ svn diff -r BASE:14 foo.c
# compares the unmodified version of foo.c with the way foo.c looked
# in revision 14

Revision numbers reveal nothing about the world outside the version control system, but sometimes you need to correlate a moment in real time with a moment in version history. To facilitate this, the --revision (-r) option can also accept as input date specifiers wrapped in curly braces ({ and }). Subversion accepts the standard ISO-8601 date and time formats, plus a few others. Here are some examples. (Remember to use quotes around any date that contains spaces.)

$ svn checkout -r {2006-02-17}
$ svn checkout -r {15:30}
$ svn checkout -r {15:30:00.200000}
$ svn checkout -r {"2006-02-17 15:30"}
$ svn checkout -r {"2006-02-17 15:30 +0230"}
$ svn checkout -r {2006-02-17T15:30}
$ svn checkout -r {2006-02-17T15:30Z}
$ svn checkout -r {2006-02-17T15:30-04:00}
$ svn checkout -r {20060217T1530}
$ svn checkout -r {20060217T1530Z}
$ svn checkout -r {20060217T1530-0500}
…

When you specify a date, Subversion resolves that date to the most recent revision of the repository as of that date, and then continues to operate against that resolved revision number:

$ svn log -r {2006-11-28}
------------------------------------------------------------------------
r12 | ira | 2006-11-27 12:31:51 -0600 (Mon, 27 Nov 2006) | 6 lines
…

You can also use a range of dates. Subversion will find all revisions between both dates, inclusive:

$ svn log -r {2006-11-20}:{2006-11-29}
…

Just as Subversion uses properties to store extra information about the files, directories, and revisions that it contains, you might also find properties to be of similar use. You might find it useful to have a place close to your versioned data to hang custom metadata about that data.

Say you wish to design a web site that houses many digital photos and displays them with captions and a datestamp. Now, your set of photos is constantly changing, so you'd like to have as much of this site automated as possible. These photos can be quite large, so as is common with sites of this nature, you want to provide smaller thumbnail images to your site visitors.

Now, you can get this functionality using traditional files. That is, you can have your image123.jpg and an image123-thumbnail.jpg side by side in a directory. Or if you want to keep the filenames the same, you might have your thumbnails in a different directory, such as thumbnails/image123.jpg. You can also store your captions and datestamps in a similar fashion, again separated from the original image file. But the problem here is that your collection of files multiplies with each new photo added to the site.

Now consider the same web site deployed in a way that makes use of Subversion's file properties. Imagine having a single image file, image123.jpg, with properties set on that file that are named caption, datestamp, and even thumbnail. Now your working copy directory looks much more manageable—in fact, it looks to the casual browser like there are nothing but image files in it. But your automation scripts know better. They know that they can use svn (or better yet, they can use the Subversion language bindings—see the section called “Using the APIs”) to dig out the extra information that your site needs to display without having to read an index file or play path manipulation games.

Custom revision properties are also frequently used. One common such use is a property whose value contains an issue tracker ID with which the revision is associated, perhaps because the change made in that revision fixes a bug filed in the tracker issue with that ID. Other uses include hanging more friendly names on the revision—it might be hard to remember that revision 1935 was a fully tested revision. But if there's, say, a test-results property on that revision with the value all passing, that's meaningful information to have.

Issue(s): IZ2376, IZ1919
Reviewed by:  sally

This fixes a nasty segfault in the wort frabbing process
…

The svn program affords a few ways to add or modify file and directory properties. For properties with short, human-readable values, perhaps the simplest way to add a new property is to specify the property name and value on the command line of the svn propset subcommand:

$ svn propset copyright '(c) 2006 Red-Bean Software' calc/button.c
property 'copyright' set on 'calc/button.c'
$

But we've been touting the flexibility that Subversion offers for your property values. And if you are planning to have a multiline textual, or even binary, property value, you probably do not want to supply that value on the command line. So the svn propset subcommand takes a --file (-F) option for specifying the name of a file that contains the new property value.

$ svn propset license -F /path/to/LICENSE calc/button.c
property 'license' set on 'calc/button.c'
$

There are some restrictions on the names you can use for properties. A property name must start with a letter, a colon (:), or an underscore (_); after that, you can also use digits, hyphens (-), and periods (.). ^[8]

In addition to the propset command, the svn program supplies the propedit command. This command uses the configured editor program (see the section called “Config”) to add or modify properties. When you run the command, svn invokes your editor program on a temporary file that contains the current value of the property (or that is empty, if you are adding a new property). Then, you just modify that value in your editor program until it represents the new value you wish to store for the property, save the temporary file, and then exit the editor program. If Subversion detects that you've actually changed the existing value of the property, it will accept that as the new property value. If you exit your editor without making any changes, no property modification will occur:

$ svn propedit copyright calc/button.c  ### exit the editor without changes
No changes to property 'copyright' on 'calc/button.c'
$

We should note that, as with other svn subcommands, those related to properties can act on multiple paths at once. This enables you to modify properties on whole sets of files with a single command. For example, we could have done the following:

$ svn propset copyright '(c) 2006 Red-Bean Software' calc/*
property 'copyright' set on 'calc/Makefile'
property 'copyright' set on 'calc/button.c'
property 'copyright' set on 'calc/integer.c'
…
$

All of this property adding and editing isn't really very useful if you can't easily get the stored property value. So the svn program supplies two subcommands for displaying the names and values of properties stored on files and directories. The svn proplist command will list the names of properties that exist on a path. Once you know the names of the properties on the node, you can request their values individually using svn propget. This command will, given a property name and a path (or set of paths), print the value of the property to the standard output stream.

$ svn proplist calc/button.c
Properties on 'calc/button.c':
  copyright
  license
$ svn propget copyright calc/button.c
(c) 2006 Red-Bean Software

There's even a variation of the proplist command that will list both the name and the value for all of the properties. Simply supply the --verbose (-v) option.

$ svn proplist -v calc/button.c
Properties on 'calc/button.c':
  copyright : (c) 2006 Red-Bean Software
  license : ================================================================
Copyright (c) 2006 Red-Bean Software.  All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions 
are met:

1. Redistributions of source code must retain the above copyright
notice, this list of conditions, and the recipe for Fitz's famous
red-beans-and-rice.
…

The last property-related subcommand is propdel. Since Subversion allows you to store properties with empty values, you can't remove a property altogether using svn propedit or svn propset. For example, this command will not yield the desired effect:

$ svn propset license '' calc/button.c
property 'license' set on 'calc/button.c'
$ svn proplist -v calc/button.c
Properties on 'calc/button.c':
  copyright : (c) 2006 Red-Bean Software
  license : 
$

You need to use the propdel subcommand to delete properties altogether. The syntax is similar to the other property commands:

$ svn propdel license calc/button.c
property 'license' deleted from 'calc/button.c'.
$ svn proplist -v calc/button.c
Properties on 'calc/button.c':
  copyright : (c) 2006 Red-Bean Software
$

Remember those unversioned revision properties? You can modify those, too, using the same svn subcommands that we just described. Simply add the --revprop command-line parameter and specify the revision whose property you wish to modify. Since revisions are global, you don't need to specify a target path to these property-related commands so long as you are positioned in a working copy of the repository whose revision property you wish to modify. Otherwise, you can simply provide the URL of any path in the repository of interest (including the repository's root URL). For example, you might want to replace the commit log message of an existing revision. ^[9] If your current working directory is part of a working copy of your repository, you can simply run the svn propset command with no target path:

$ svn propset svn:log '* button.c: Fix a compiler warning.' -r11 --revprop
property 'svn:log' set on repository revision '11'
$

But even if you haven't checked out a working copy from that repository, you can still effect the property change by providing the repository's root URL:

$ svn propset svn:log '* button.c: Fix a compiler warning.' -r11 --revprop \
              http://svn.example.com/repos/project
property 'svn:log' set on repository revision '11'
$

Note that the ability to modify these unversioned properties must be explicitly added by the repository administrator (see the section called “Commit Log Message Correction”). That's because the properties aren't versioned, so you run the risk of losing information if you aren't careful with your edits. The repository administrator can set up methods to protect against this loss, and by default, modification of unversioned properties is disabled.

Now that you are familiar with all of the property-related svn subcommands, let's see how property modifications affect the usual Subversion workflow. As we mentioned earlier, file and directory properties are versioned, just like your file contents. As a result, Subversion provides the same opportunities for merging—cleanly or with conflicts—someone else's modifications into your own.

As with file contents, your property changes are local modifications, made permanent only when you commit them to the repository with svn commit. Your property changes can be easily unmade, too—the svn revert command will restore your files and directories to their unedited states—contents, properties, and all. Also, you can receive interesting information about the state of your file and directory properties by using the svn status and svn diff commands.

$ svn status calc/button.c
 M     calc/button.c
$ svn diff calc/button.c
Property changes on: calc/button.c
___________________________________________________________________
Name: copyright
   + (c) 2006 Red-Bean Software

$

Notice how the status subcommand displays M in the second column instead of the first. That is because we have modified the properties on calc/button.c, but not its textual contents. Had we changed both, we would have seen M in the first column, too. (We cover svn status in the section called “See an overview of your changes”).

$ svn update calc
M  calc/Makefile.in
Conflict for property 'linecount' discovered on 'calc/button.c'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (s) show all options: p
 C calc/button.c
Updated to revision 143.
$ 

$ svn status calc
 C     calc/button.c
?      calc/button.c.prej
$ cat calc/button.c.prej 
Trying to change property 'linecount' from '1267' to '1301',
but property has been locally changed from '1267' to '1256'.
$

You might also have noticed the nonstandard way that Subversion currently displays property differences. You can still use svn diff and redirect its output to create a usable patch file. The patch program will ignore property patches—as a rule, it ignores any noise it can't understand. This does, unfortunately, mean that to fully apply a patch generated by svn diff, any property modifications will need to be applied by hand.

Properties are a powerful feature of Subversion, acting as key components of many Subversion features discussed elsewhere in this and other chapters—textual diff and merge support, keyword substitution, newline translation, and so on. But to get the full benefit of properties, they must be set on the right files and directories. Unfortunately, that step can be easily forgotten in the routine of things, especially since failing to set a property doesn't usually result in an obvious error (at least compared to, say, failing to add a file to version control). To help your properties get applied to the places that need them, Subversion provides a couple of simple but useful features.

Whenever you introduce a file to version control using the svn add or svn import commands, Subversion tries to assist by setting some common file properties automatically. First, on operating systems whose filesystems support an execute permission bit, Subversion will automatically set the svn:executable property on newly added or imported files whose execute bit is enabled. (See the section called “File Executability” later in this chapter for more about this property.)

Second, Subversion tries to determine the file's MIME type. If you've configured a mime-types-files runtime configuration parameter, Subversion will try to find a MIME type mapping in that file for your file's extension. If it finds such a mapping, it will set your file's svn:mime-type property to the MIME type it found. If no mapping file is configured, or no mapping for your file's extension could be found, Subversion runs a very basic heuristic to determine whether the file contains nontextual content. If so, it automatically sets the svn:mime-type property on that file to application/octet-stream (the generic “this is a collection of bytes” MIME type). Of course, if Subversion guesses incorrectly, or if you wish to set the svn:mime-type property to something more precise—perhaps image/png or application/x-shockwave-flash—you can always remove or edit that property. (For more on Subversion's use of MIME types, see the section called “File Content Type” later in this chapter.)

Subversion also provides, via its runtime configuration system (see the section called “Runtime Configuration Area”), a more flexible automatic property setting feature that allows you to create mappings of filename patterns to property names and values. Once again, these mappings affect adds and imports, and can not only override the default MIME type decision made by Subversion during those operations, but can also set additional Subversion or custom properties, too. For example, you might create a mapping that says that anytime you add JPEG files—ones whose names match the pattern *.jpg—Subversion should automatically set the svn:mime-type property on those files to image/jpeg. Or perhaps any files that match *.cpp should have svn:eol-style set to native, and svn:keywords set to Id. Automatic property support is perhaps the handiest property-related tool in the Subversion toolbox. See the section called “Config” for more about configuring that support.

Subversion joins the ranks of the many applications that recognize and make use of Multipurpose Internet Mail Extensions (MIME) content types. Besides being a general-purpose storage location for a file's content type, the value of the svn:mime-type file property determines some behavioral characteristics of Subversion itself.

For example, one of the benefits that Subversion typically provides is contextual, line-based merging of changes received from the server during an update into your working file. But for files containing nontextual data, there is often no concept of a “line.” So, for versioned files whose svn:mime-type property is set to a nontextual MIME type (generally, something that doesn't begin with text/, though there are exceptions), Subversion does not attempt to perform contextual merges during updates. Instead, any time you have locally modified a binary working copy file that is also being updated, your file is left untouched and Subversion creates two new files. One file has a .oldrev extension and contains the BASE revision of the file. The other file has a .newrev extension and contains the contents of the updated revision of the file. This behavior is really for the protection of the user against failed attempts at performing contextual merges on files that simply cannot be contextually merged.

Beginning in Subversion 1.5, users can configure a new mime-types-file runtime configuration parameter, which identifies the location of a MIME types mapping file. Subversion will consult this mapping file to determine the MIME type of newly added and imported files.

Also, if the svn:mime-type property is set, then the Subversion Apache module will use its value to populate the Content-type: HTTP header when responding to GET requests. This gives your web browser a crucial clue about how to display a file when you use it to peruse your Subversion repository's contents.

On many operating systems, the ability to execute a file as a command is governed by the presence of an execute permission bit. This bit usually defaults to being disabled, and must be explicitly enabled by the user for each file that needs it. But it would be a monumental hassle to have to remember exactly which files in a freshly checked-out working copy were supposed to have their executable bits toggled on, and then to have to do that toggling. So, Subversion provides the svn:executable property as a way to specify that the executable bit for the file on which that property is set should be enabled, and Subversion honors that request when populating working copies with such files.

This property has no effect on filesystems that have no concept of an executable permission bit, such as FAT32 and NTFS. ^[11] Also, although it has no defined values, Subversion will force its value to * when setting this property. Finally, this property is valid only on files, not on directories.

Unless otherwise noted using a versioned file's svn:mime-type property, Subversion assumes the file contains human-readable data. Generally speaking, Subversion uses this knowledge only to determine whether contextual difference reports for that file are possible. Otherwise, to Subversion, bytes are bytes.

This means that by default, Subversion doesn't pay any attention to the type of end-of-line (EOL) markers used in your files. Unfortunately, different operating systems have different conventions about which character sequences represent the end of a line of text in a file. For example, the usual line-ending token used by software on the Windows platform is a pair of ASCII control characters—a carriage return (CR) followed by a line feed (LF). Unix software, however, just uses the LF character to denote the end of a line.

Not all of the various tools on these operating systems understand files that contain line endings in a format that differs from the native line-ending style of the operating system on which they are running. So, typically, Unix programs treat the CR character present in Windows files as a regular character (usually rendered as ^M), and Windows programs combine all of the lines of a Unix file into one giant line because no carriage return-linefeed (or CRLF) character combination was found to denote the ends of the lines.

This sensitivity to foreign EOL markers can be frustrating for folks who share a file across different operating systems. For example, consider a source code file, and developers that edit this file on both Windows and Unix systems. If all the developers always use tools that preserve the line-ending style of the file, no problems occur.

But in practice, many common tools either fail to properly read a file with foreign EOL markers, or convert the file's line endings to the native style when the file is saved. If the former is true for a developer, he has to use an external conversion utility (such as dos2unix or its companion, unix2dos) to prepare the file for editing. The latter case requires no extra preparation. But both cases result in a file that differs from the original quite literally on every line! Prior to committing his changes, the user has two choices. Either he can use a conversion utility to restore the modified file to the same line-ending style that it was in before his edits were made, or he can simply commit the file—new EOL markers and all.

The result of scenarios like these include wasted time and unnecessary modifications to committed files. Wasted time is painful enough. But when commits change every line in a file, this complicates the job of determining which of those lines were changed in a nontrivial way. Where was that bug really fixed? On what line was a syntax error introduced?

The solution to this problem is the svn:eol-style property. When this property is set to a valid value, Subversion uses it to determine what special processing to perform on the file so that the file's line-ending style isn't flip-flopping with every commit that comes from a different operating system. The valid values are:

In the Subversion repository, a lock is a piece of metadata that grants exclusive access to one user to change a file. This user is said to be the lock owner. Each lock also has a unique identifier, typically a long string of characters, known as the lock token. The repository manages locks, ultimately handling their creation, enforcement, and removal. If any commit transaction attempts to modify or delete a locked file (or delete one of the parent directories of the file), the repository will demand two pieces of information—that the client performing the commit be authenticated as the lock owner, and that the lock token has been provided as part of the commit process as a form of proof that the client knows which lock it is using.

To demonstrate lock creation, let's refer back to our example of multiple graphic designers working on the same binary image files. Harry has decided to change a JPEG image. To prevent other people from committing changes to the file while he is modifying it (as well as alerting them that he is about to change it), he locks the file in the repository using the svn lock command.

$ svn lock banana.jpg -m "Editing file for tomorrow's release."
'banana.jpg' locked by user 'harry'.
$

The preceding example demonstrates a number of new things. First, notice that Harry passed the --message (-m) option to svn lock. Similar to svn commit, the svn lock command can take comments—via either --message (-m) or --file (-F)—to describe the reason for locking the file. Unlike svn commit, however, svn lock will not demand a message by launching your preferred text editor. Lock comments are optional, but still recommended to aid communication.

Second, the lock attempt succeeded. This means that the file wasn't already locked, and that Harry had the latest version of the file. If Harry's working copy of the file had been out of date, the repository would have rejected the request, forcing Harry to svn update and reattempt the locking command. The locking command would also have failed if the file had already been locked by someone else.

As you can see, the svn lock command prints confirmation of the successful lock. At this point, the fact that the file is locked becomes apparent in the output of the svn status and svn info reporting subcommands.

$ svn status
     K banana.jpg

$ svn info banana.jpg
Path: banana.jpg
Name: banana.jpg
URL: http://svn.example.com/repos/project/banana.jpg
Repository UUID: edb2f264-5ef2-0310-a47a-87b0ce17a8ec
Revision: 2198
Node Kind: file
Schedule: normal
Last Changed Author: frank
Last Changed Rev: 1950
Last Changed Date: 2006-03-15 12:43:04 -0600 (Wed, 15 Mar 2006)
Text Last Updated: 2006-06-08 19:23:07 -0500 (Thu, 08 Jun 2006)
Properties Last Updated: 2006-06-08 19:23:07 -0500 (Thu, 08 Jun 2006)
Checksum: 3b110d3b10638f5d1f4fe0f436a5a2a5
Lock Token: opaquelocktoken:0c0f600b-88f9-0310-9e48-355b44d4a58e
Lock Owner: harry
Lock Created: 2006-06-14 17:20:31 -0500 (Wed, 14 Jun 2006)
Lock Comment (1 line):
Editing file for tomorrow's release.

$

The fact that the svn info command, which does not contact the repository when run against working copy paths, can display the lock token reveals an important piece of information about those tokens: they are cached in the working copy. The presence of the lock token is critical. It gives the working copy authorization to make use of the lock later on. Also, the svn status command shows a K next to the file (short for locKed), indicating that the lock token is present.

Now that Harry has locked banana.jpg, Sally is unable to change or delete that file:

$ svn delete banana.jpg
D         banana.jpg
$ svn commit -m "Delete useless file."
Deleting       banana.jpg
svn: Commit failed (details follow):
svn: Server sent unexpected return value (423 Locked) in response to DELETE\
 request for '/repos/project/!svn/wrk/64bad3a9-96f9-0310-818a-df4224ddc35d/\
banana.jpg'
$

But Harry, after touching up the banana's shade of yellow, is able to commit his changes to the file. That's because he authenticates as the lock owner and also because his working copy holds the correct lock token:

$ svn status
M    K banana.jpg
$ svn commit -m "Make banana more yellow"
Sending        banana.jpg
Transmitting file data .
Committed revision 2201.
$ svn status
$

Notice that after the commit is finished, svn status shows that the lock token is no longer present in the working copy. This is the standard behavior of svn commit—it searches the working copy (or list of targets, if you provide such a list) for local modifications and sends all the lock tokens it encounters during this walk to the server as part of the commit transaction. After the commit completes successfully, all of the repository locks that were mentioned are released—even on files that weren't committed. This is meant to discourage users from being sloppy about locking or from holding locks for too long. If Harry haphazardly locks 30 files in a directory named images because he's unsure of which files he needs to change, yet changes only four of those files, when he runs svn commit images, the process will still release all 30 locks.

This behavior of automatically releasing locks can be overridden with the --no-unlock option to svn commit. This is best used for those times when you want to commit changes, but still plan to make more changes and thus need to retain existing locks. You can also make this your default behavior by setting the no-unlock runtime configuration option (see the section called “Runtime Configuration Area”).

Of course, locking a file doesn't oblige one to commit a change to it. The lock can be released at any time with a simple svn unlock command:

$ svn unlock banana.c
'banana.c' unlocked.

When a commit fails due to someone else's locks, it's fairly easy to learn about them. The easiest way is to run svn status --show-updates:

$ svn status -u
M              23   bar.c
M    O         32   raisin.jpg
       *       72   foo.h
Status against revision:     105
$

In this example, Sally can see not only that her copy of foo.h is out of date, but also that one of the two modified files she plans to commit is locked in the repository. The O symbol stands for “Other,” meaning that a lock exists on the file and was created by somebody else. If she were to attempt a commit, the lock on raisin.jpg would prevent it. Sally is left wondering who made the lock, when, and why. Once again, svn info has the answers:

$ svn info http://svn.example.com/repos/project/raisin.jpg
Path: raisin.jpg
Name: raisin.jpg
URL: http://svn.example.com/repos/project/raisin.jpg
Repository UUID: edb2f264-5ef2-0310-a47a-87b0ce17a8ec
Revision: 105
Node Kind: file
Last Changed Author: sally
Last Changed Rev: 32
Last Changed Date: 2006-01-25 12:43:04 -0600 (Sun, 25 Jan 2006)
Lock Token: opaquelocktoken:fc2b4dee-98f9-0310-abf3-653ff3226e6b
Lock Owner: harry
Lock Created: 2006-02-16 13:29:18 -0500 (Thu, 16 Feb 2006)
Lock Comment (1 line):
Need to make a quick tweak to this image.
$

Just as you can use svn info to examine objects in the working copy, you can also use it to examine objects in the repository. If the main argument to svn info is a working copy path, then all of the working copy's cached information is displayed; any mention of a lock means that the working copy is holding a lock token (if a file is locked by another user or in another working copy, svn info on a working copy path will show no lock information at all). If the main argument to svn info is a URL, the information reflects the latest version of an object in the repository, and any mention of a lock describes the current lock on the object.

So in this particular example, Sally can see that Harry locked the file on February 16 to “make a quick tweak.” It being June, she suspects that he probably forgot all about the lock. She might phone Harry to complain and ask him to release the lock. If he's unavailable, she might try to forcibly break the lock herself or ask an administrator to do so.

A repository lock isn't sacred—in Subversion's default configuration state, locks can be released not only by the person who created them, but by anyone. When somebody other than the original lock creator destroys a lock, we refer to this as breaking the lock.

From the administrator's chair, it's simple to break locks. The svnlook and svnadmin programs have the ability to display and remove locks directly from the repository. (For more information about these tools, see the section called “An Administrator's Toolkit”.)

$ svnadmin lslocks /var/svn/repos
Path: /project2/images/banana.jpg
UUID Token: opaquelocktoken:c32b4d88-e8fb-2310-abb3-153ff1236923
Owner: frank
Created: 2006-06-15 13:29:18 -0500 (Thu, 15 Jun 2006)
Expires: 
Comment (1 line):
Still improving the yellow color.

Path: /project/raisin.jpg
UUID Token: opaquelocktoken:fc2b4dee-98f9-0310-abf3-653ff3226e6b
Owner: harry
Created: 2006-02-16 13:29:18 -0500 (Thu, 16 Feb 2006)
Expires: 
Comment (1 line):
Need to make a quick tweak to this image.

$ svnadmin rmlocks /var/svn/repos /project/raisin.jpg
Removed lock on '/project/raisin.jpg'.
$

The more interesting option is to allow users to break each other's locks over the network. To do this, Sally simply needs to pass the --force to the svn unlock command:

$ svn status -u
M              23   bar.c
M    O         32   raisin.jpg
       *       72   foo.h
Status against revision:     105
$ svn unlock raisin.jpg
svn: 'raisin.jpg' is not locked in this working copy
$ svn info raisin.jpg | grep URL
URL: http://svn.example.com/repos/project/raisin.jpg
$ svn unlock http://svn.example.com/repos/project/raisin.jpg
svn: Unlock request failed: 403 Forbidden (http://svn.example.com)
$ svn unlock --force http://svn.example.com/repos/project/raisin.jpg
'raisin.jpg' unlocked.
$

Now, Sally's initial attempt to unlock failed because she ran svn unlock directly on her working copy of the file, and no lock token was present. To remove the lock directly from the repository, she needs to pass a URL to svn unlock. Her first attempt to unlock the URL fails, because she can't authenticate as the lock owner (nor does she have the lock token). But when she passes --force, the authentication and authorization requirements are ignored, and the remote lock is broken.

Simply breaking a lock may not be enough. In the running example, Sally may not only want to break Harry's long-forgotten lock, but relock the file for her own use. She can accomplish this by using svn unlock with --force and then svn lock back-to-back, but there's a small chance that somebody else might lock the file between the two commands. The simpler thing to do is to steal the lock, which involves breaking and relocking the file all in one atomic step. To do this, Sally passes the --force option to svn lock:

$ svn lock raisin.jpg
svn: Lock request failed: 423 Locked (http://svn.example.com)
$ svn lock --force raisin.jpg
'raisin.jpg' locked by user 'sally'.
$

In any case, whether the lock is broken or stolen, Harry may be in for a surprise. Harry's working copy still contains the original lock token, but that lock no longer exists. The lock token is said to be defunct. The lock represented by the lock token has either been broken (no longer in the repository) or stolen (replaced with a different lock). Either way, Harry can see this by asking svn status to contact the repository:

$ svn status
     K raisin.jpg
$ svn status -u
     B         32   raisin.jpg
$ svn update
  B  raisin.jpg
$ svn status
$

If the repository lock was broken, then svn status --show-updates displays a B (Broken) symbol next to the file. If a new lock exists in place of the old one, then a T (sTolen) symbol is shown. Finally, svn update notices any defunct lock tokens and removes them from the working copy.

We've seen how svn lock and svn unlock can be used to create, release, break, and steal locks. This satisfies the goal of serializing commit access to a file. But what about the larger problem of preventing wasted time?

For example, suppose Harry locks an image file and then begins editing it. Meanwhile, miles away, Sally wants to do the same thing. She doesn't think to run svn status --show-updates, so she has no idea that Harry has already locked the file. She spends hours editing the file, and when she tries to commit her change, she discovers that either the file is locked or that she's out of date. Regardless, her changes aren't mergeable with Harry's. One of these two people has to throw away his or her work, and a lot of time has been wasted.

Subversion's solution to this problem is to provide a mechanism to remind users that a file ought to be locked before the editing begins. The mechanism is a special property: svn:needs-lock. If that property is attached to a file (regardless of its value, which is irrelevant), Subversion will try to use filesystem-level permissions to make the file read-only—unless, of course, the user has explicitly locked the file. When a lock token is present (as a result of using svn lock), the file becomes read/write. When the lock is released, the file becomes read-only again.

The theory, then, is that if the image file has this property attached, Sally would immediately notice something is strange when she opens the file for editing: many applications alert users immediately when a read-only file is opened for editing, and nearly all would prevent her from saving changes to the file. This reminds her to lock the file before editing, whereby she discovers the preexisting lock:

$ /usr/local/bin/gimp raisin.jpg
gimp: error: file is read-only!
$ ls -l raisin.jpg
-r--r--r--   1 sally   sally   215589 Jun  8 19:23 raisin.jpg
$ svn lock raisin.jpg
svn: Lock request failed: 423 Locked (http://svn.example.com)
$ svn info http://svn.example.com/repos/project/raisin.jpg | grep Lock
Lock Token: opaquelocktoken:fc2b4dee-98f9-0310-abf3-653ff3226e6b
Lock Owner: harry
Lock Created: 2006-06-08 07:29:18 -0500 (Thu, 08 June 2006)
Lock Comment (1 line):
Making some tweaks.  Locking for the next two hours.
$

Note that this property is a communication tool that works independently from the locking system. In other words, any file can be locked, whether or not this property is present. And conversely, the presence of this property doesn't make the repository require a lock when committing.

Unfortunately, the system isn't flawless. It's possible that even when a file has the property, the read-only reminder won't always work. Sometimes applications misbehave and “hijack” the read-only file, silently allowing users to edit and save the file anyway. There's not much that Subversion can do in this situation—at the end of the day, there's simply no substitution for good interpersonal communication. ^[16]

You can create, modify, and delete changelists using the svn changelist command. More accurately, you use this command to set or unset the changelist association of a particular working copy file. A changelist is effectively created the first time you label a file with that changelist; it is deleted when you remove that label from the last file that had it. Let's examine a usage scenario that demonstrates these concepts.

Harry is fixing some bugs in the calculator application's mathematics logic. His work leads him to change a couple of files:

$ svn status
M      integer.c
M      mathops.c
$

While testing his bug fix, Harry notices that his changes bring to light a tangentially related bug in the user interface logic found in button.c. Harry decides that he'll go ahead and fix that bug, too, as a separate commit from his math fixes. Now, in a small working copy with only a handful of files and few logical changes, Harry can probably keep his two logical change groupings mentally organized without any problem. But today he's going to use Subversion's changelists feature as a special favor to the authors of this book.

Harry first creates a changelist and associates with it the two files he's already changed. He does this by using the svn changelist command to assign the same arbitrary changelist name to those files:

$ svn changelist math-fixes integer.c mathops.c
Path 'integer.c' is now a member of changelist 'math-fixes'.
Path 'mathops.c' is now a member of changelist 'math-fixes'.
$ svn status

--- Changelist 'math-fixes':
M      integer.c
M      mathops.c
$

As you can see, the output of svn status reflects this new grouping.

Harry now sets off to fix the secondary UI problem. Since he knows which file he'll be changing, he assigns that path to a changelist, too. Unfortunately, Harry carelessly assigns this third file to the same changelist as the previous two files:

$ svn changelist math-fixes button.c
Path 'button.c' is now a member of changelist 'math-fixes'.
$ svn status

--- Changelist 'math-fixes':
       button.c
M      integer.c
M      mathops.c
$

Fortunately, Harry catches his mistake. At this point, he has two options. He can remove the changelist association from button.c, and then assign a different changelist name:

$ svn changelist --remove button.c
Path 'button.c' is no longer a member of a changelist.
$ svn changelist ui-fix button.c
Path 'button.c' is now a member of changelist 'ui-fix'.
$

Or, he can skip the removal and just assign a new changelist name. In this case, Subversion will first warn Harry that button.c is being removed from the first changelist:

$ svn changelist ui-fix button.c
svn: warning: Removing 'button.c' from changelist 'math-fixes'.
Path 'button.c' is now a member of changelist 'ui-fix'.
$ svn status

--- Changelist 'ui-fix':
       button.c

--- Changelist 'math-fixes':
M      integer.c
M      mathops.c
$

Harry now has two distinct changelists present in his working copy, and svn status will group its output according to these changelist determinations. Notice that even though Harry hasn't yet modified button.c, it still shows up in the output of svn status as interesting because it has a changelist assignment. Changelists can be added to and removed from files at any time, regardless of whether they contain local modifications.

Harry now fixes the user interface problem in button.c.

$ svn status

--- Changelist 'ui-fix':
M      button.c

--- Changelist 'math-fixes':
M      integer.c
M      mathops.c
$

The visual grouping that Harry sees in the output of svn status as shown in our previous section is nice, but not entirely useful. The status command is but one of many operations that he might wish to perform on his working copy. Fortunately, many of Subversion's other operations understand how to operate on changelists via the use of the --changelist option.

When provided with a --changelist option, Subversion commands will limit the scope of their operation to only those files to which a particular changelist name is assigned. If Harry now wants to see the actual changes he's made to the files in his math-fixes changelist, he could explicitly list only the files that make up that changelist on the svn diff command line.

$ svn diff integer.c mathops.c
Index: integer.c
===================================================================
--- integer.c	(revision 1157)
+++ integer.c	(working copy)
…
Index: mathops.c
===================================================================
--- mathops.c	(revision 1157)
+++ mathops.c	(working copy)
…
$

That works okay for a few files, but what if Harry's change touched 20 or 30 files? That would be an annoyingly long list of explicitly named files. Now that he's using changelists, though, Harry can avoid explicitly listing the set of files in his changelist from now on, and instead provide just the changelist name:

$ svn diff --changelist math-fixes
Index: integer.c
===================================================================
--- integer.c	(revision 1157)
+++ integer.c	(working copy)
…
Index: mathops.c
===================================================================
--- mathops.c	(revision 1157)
+++ mathops.c	(working copy)
…
$

And when it's time to commit, Harry can again use the --changelist option to limit the scope of the commit to files in a certain changelist. He might commit his user interface fix by doing the following:

$ svn ci -m "Fix a UI bug found while working on math logic." \
      --changelist ui-fix
Sending        button.c
Transmitting file data .
Committed revision 1158.
$

In fact, the svn commit command provides a second changelists-related option: --keep-changelists. Normally, changelist assignments are removed from files after they are committed. But if --keep-changelists is provided, Subversion will leave the changelist assignment on the committed (and now unmodified) files. In any case, committing files assigned to one changelist leaves other changelists undisturbed.

$ svn status

--- Changelist 'math-fixes':
M      integer.c
M      mathops.c
$

Even the svn changelist command accepts the --changelist option. This allows you to quickly and easily rename or remove a changelist:

$ svn changelist math-bugs --changelist math-fixes --depth infinity .
svn: warning: Removing 'integer.c' from changelist 'math-fixes'.
Path 'integer.c' is now a member of changelist 'math-bugs'.
svn: warning: Removing 'mathops.c' from changelist 'math-fixes'.
Path 'mathops.c' is now a member of changelist 'math-bugs'.
$ svn changelist --remove --changelist math-bugs --depth infinity .
Path 'integer.c' is no longer a member of a changelist.
Path 'mathops.c' is no longer a member of a changelist.
$

Finally, you can specify multiple instances of the --changelist option on a single command line. Doing so limits the operation you are performing to files found in any of the specified changesets.

Subversion's changelist feature is a handy tool for grouping working copy files, but it does have a few limitations. Changelists are artifacts of a particular working copy, which means that changelist assignments cannot be propagated to the repository or otherwise shared with other users. Changelists can be assigned only to files—Subversion doesn't currently support the use of changelists with directories. Finally, you can have at most one changelist assignment on a given working copy file. Here is where the blog post category and photo service tag analogies break down—if you find yourself needing to assign a file to multiple changelists, you're out of luck.

The Subversion client spends most of its time managing working copies. When it needs information from a remote repository, however, it makes a network request, and the server responds with an appropriate answer. The details of the network protocol are hidden from the user—the client attempts to access a URL, and depending on the URL scheme, a particular protocol is used to contact the server (see the sidebar Repository URLs).

When the server process receives a client request, it often demands that the client identify itself. It issues an authentication challenge to the client, and the client responds by providing credentials back to the server. Once authentication is complete, the server responds with the original information that the client asked for. Notice that this system is different from systems such as CVS, where the client preemptively offers credentials (“logs in”) to the server before ever making a request. In Subversion, the server “pulls” credentials by challenging the client at the appropriate moment, rather than the client “pushing” them. This makes certain operations more elegant. For example, if a server is configured to allow anyone in the world to read a repository, the server will never issue an authentication challenge when a client attempts to svn checkout.

If the particular network requests issued by the client result in a new revision being created in the repository (e.g., svn commit), Subversion uses the authenticated username associated with those requests as the author of the revision. That is, the authenticated user's name is stored as the value of the svn:author property on the new revision (see the section called “Subversion Properties”). If the client was not authenticated (i.e., if the server never issued an authentication challenge), the revision's svn:author property is empty.

Many servers are configured to require authentication on every request. This would be a big annoyance to users if they were forced to type their passwords over and over again. Fortunately, the Subversion client has a remedy for this—a built-in system for caching authentication credentials on disk. By default, whenever the command-line client successfully responds to a server's authentication challenge, it saves the credentials in the user's private runtime configuration area (~/.subversion/auth/ on Unix-like systems or %APPDATA%/Subversion/auth/ on Windows; see the section called “Runtime Configuration Area” for more details about the runtime configuration system). Successful credentials are cached on disk and keyed on a combination of the server's hostname, port, and authentication realm.

When the client receives an authentication challenge, it first looks for the appropriate credentials in the user's disk cache. If seemingly suitable credentials are not present, or if the cached credentials ultimately fail to authenticate, the client will, by default, fall back to prompting the user for the necessary information.

The security-conscious reader will suspect immediately that there is reason for concern here. “Caching passwords on disk? That's terrible! You should never do that!”

The Subversion developers recognize the legitimacy of such concerns, and so Subversion works with available mechanisms provided by the operating system and environment to try to minimize the risk of leaking this information. Here's a breakdown of what this means for users on the most common platforms:

Of course, for the truly paranoid, none of these mechanisms meets the test of perfection. So for those folks willing to sacrifice convenience for the ultimate in security, Subversion provides various ways of disabling its credentials caching system altogether.

To disable caching for a single command, pass the --no-auth-cache option:

$ svn commit -F log_msg.txt --no-auth-cache
Authentication realm: <svn://host.example.com:3690> example realm
Username:  joe
Password for 'joe':

Adding         newfile
Transmitting file data .
Committed revision 2324.

# password was not cached, so a second commit still prompts us

$ svn delete newfile
$ svn commit -F new_msg.txt
Authentication realm: <svn://host.example.com:3690> example realm
Username:  joe
…

Or, if you want to disable credential caching permanently, you can edit the config file in your runtime configuration area and set the store-auth-creds option to no. This will prevent the storing of credentials used in any Subversion interactions you perform on the affected computer. This can be extended to cover all users on the computer, too, by modifying the system-wide runtime configuration area (described in the section called “Configuration Area Layout”).

[auth]
store-auth-creds = no

Sometimes users will want to remove specific credentials from the disk cache. To do this, you need to navigate into the auth/ area and manually delete the appropriate cache file. Credentials are cached in individual files; if you look inside each file, you will see keys and values. The svn:realmstring key describes the particular server realm that the file is associated with:

$ ls ~/.subversion/auth/svn.simple/
5671adf2865e267db74f09ba6f872c28
3893ed123b39500bca8a0b382839198e
5c3c22968347b390f349ff340196ed39

$ cat ~/.subversion/auth/svn.simple/5671adf2865e267db74f09ba6f872c28

K 8
username
V 3
joe
K 8
password
V 4
blah
K 15
svn:realmstring
V 45
<https://svn.domain.com:443> Joe's repository
END

Once you have located the proper cache file, just delete it.

One last word about svn's authentication behavior, specifically regarding the --username and --password options. Many client subcommands accept these options, but it is important to understand that using these options does not automatically send credentials to the server. As discussed earlier, the server “pulls” credentials from the client when it deems necessary; the client cannot “push” them at will. If a username and/or password are passed as options, they will be presented to the server only if the server requests them. These options are typically used to authenticate as a different user than Subversion would have chosen by default (such as your system login name) or when trying to avoid interactive prompting (such as when calling svn from a script).

Here is a final summary that describes how a Subversion client behaves when it receives an authentication challenge.

If the client successfully authenticates by any of these methods, it will attempt to cache the credentials on disk (unless the user has disabled this behavior, as mentioned earlier).

Creating a branch is very simple—you make a copy of the project in the repository using the svn copy command. Subversion is able to copy not only single files, but whole directories as well. In this case, you want to make a copy of the /calc/trunk directory. Where should the new copy live? Wherever you wish—it's a matter of project policy. Let's say that your team has a policy of creating branches in the /calc/branches area of the repository, and you want to name your branch my-calc-branch. You'll want to create a new directory, /calc/branches/my-calc-branch, which begins its life as a copy of /calc/trunk.

You may already have seen svn copy used to copy one file to another within a working copy. But it can also be used to do a “remote” copy entirely within the repository. Just copy one URL to another:

$ svn copy http://svn.example.com/repos/calc/trunk \
           http://svn.example.com/repos/calc/branches/my-calc-branch \
      -m "Creating a private branch of /calc/trunk."

Committed revision 341.

This command causes a near-instantaneous commit in the repository, creating a new directory in revision 341. The new directory is a copy of /calc/trunk. This is shown in Figure 4.3, “Repository with new copy”. ^[20] While it's also possible to create a branch by using svn copy to duplicate a directory within the working copy, this technique isn't recommended. It can be quite slow, in fact! Copying a directory on the client side is a linear-time operation, in that it actually has to duplicate every file and subdirectory on the local disk. Copying a directory on the server, however, is a constant-time operation, and it's the way most people create branches.

Figure 4.3. Repository with new copy

Now that you've created a branch of the project, you can check out a new working copy to start using it:

$ svn checkout http://svn.example.com/repos/calc/branches/my-calc-branch
A  my-calc-branch/Makefile
A  my-calc-branch/integer.c
A  my-calc-branch/button.c
Checked out revision 341.

There's nothing special about this working copy; it simply mirrors a different directory in the repository. When you commit changes, however, Sally won't see them when she updates, because her working copy is of /calc/trunk. (Be sure to read the section called “Traversing Branches” later in this chapter: the svn switch command is an alternative way of creating a working copy of a branch.)

Let's pretend that a week goes by, and the following commits happen:

Now two independent lines of development (shown in Figure 4.4, “The branching of one file's history”) are happening on integer.c.

Figure 4.4. The branching of one file's history

Things get interesting when you look at the history of changes made to your copy of integer.c:

$ pwd
/home/user/my-calc-branch

$ svn log -v integer.c
------------------------------------------------------------------------
r343 | user | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines
Changed paths:
   M /calc/branches/my-calc-branch/integer.c

* integer.c:  frozzled the wazjub.

------------------------------------------------------------------------
r341 | user | 2002-11-03 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines
Changed paths:
   A /calc/branches/my-calc-branch (from /calc/trunk:340)

Creating a private branch of /calc/trunk.

------------------------------------------------------------------------
r303 | sally | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  changed a docstring.

------------------------------------------------------------------------
r98 | sally | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines
Changed paths:
   A /calc/trunk/integer.c

* integer.c:  adding this file to the project.

------------------------------------------------------------------------

Notice that Subversion is tracing the history of your branch's integer.c all the way back through time, even traversing the point where it was copied. It shows the creation of the branch as an event in the history, because integer.c was implicitly copied when all of /calc/trunk/ was copied. Now look at what happens when Sally runs the same command on her copy of the file:

$ pwd
/home/sally/calc

$ svn log -v integer.c
------------------------------------------------------------------------
r344 | sally | 2002-11-07 15:27:56 -0600 (Thu, 07 Nov 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  fix a bunch of spelling errors.

------------------------------------------------------------------------
r303 | sally | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines
Changed paths:
   M /calc/trunk/integer.c

* integer.c:  changed a docstring.

------------------------------------------------------------------------
r98 | sally | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines
Changed paths:
   A /calc/trunk/integer.c

* integer.c:  adding this file to the project.

------------------------------------------------------------------------

Sally sees her own revision 344 change, but not the change you made in revision 343. As far as Subversion is concerned, these two commits affected different files in different repository locations. However, Subversion does show that the two files share a common history. Before the branch copy was made in revision 341, the files used to be the same file. That's why you and Sally both see the changes made in revisions 303 and 98.

You should remember two important lessons from this section. First, Subversion has no internal concept of a branch—it knows only how to make copies. When you copy a directory, the resultant directory is only a “branch” because you attach that meaning to it. You may think of the directory differently, or treat it differently, but to Subversion it's just an ordinary directory that happens to carry some extra historical information.

Second, because of this copy mechanism, Subversion's branches exist as normal filesystem directories in the repository. This is different from other version control systems, where branches are typically defined by adding extra-dimensional “labels” to collections of files. The location of your branch directory doesn't matter to Subversion. Most teams follow a convention of putting all branches into a /branches directory, but you're free to invent any policy you wish.

Before we proceed further, we should warn you that there's going to be a lot of discussion of “changes” in the pages ahead. A lot of people experienced with version control systems use the terms “change” and “changeset” interchangeably, and we should clarify what Subversion understands as a changeset.

Everyone seems to have a slightly different definition of changeset, or at least a different expectation of what it means for a version control system to have one. For our purposes, let's say that a changeset is just a collection of changes with a unique name. The changes might include textual edits to file contents, modifications to tree structure, or tweaks to metadata. In more common speak, a changeset is just a patch with a name you can refer to.

In Subversion, a global revision number N names a tree in the repository: it's the way the repository looked after the Nth commit. It's also the name of an implicit changeset: if you compare tree N with tree N−1, you can derive the exact patch that was committed. For this reason, it's easy to think of revision N as not just a tree, but a changeset as well. If you use an issue tracker to manage bugs, you can use the revision numbers to refer to particular patches that fix bugs—for example, “this issue was fixed by r9238.” Somebody can then run svn log -r 9238 to read about the exact changeset that fixed the bug, and run svn diff -c 9238 to see the patch itself. And (as you'll see shortly) Subversion's svn merge command is able to use revision numbers. You can merge specific changesets from one branch to another by naming them in the merge arguments: passing -c 9238 to svn merge would merge changeset r9238 into your working copy.

Continuing with our running example, let's suppose that a week has passed since you started working on your private branch. Your new feature isn't finished yet, but at the same time you know that other people on your team have continued to make important changes in the project's /trunk. It's in your best interest to replicate those changes to your own branch, just to make sure they mesh well with your changes. In fact, this is a best practice: frequently keeping your branch in sync with the main development line helps prevent “surprise” conflicts when it comes time for you to fold your changes back into the trunk.

Subversion is aware of the history of your branch and knows when it divided away from the trunk. To replicate the latest, greatest trunk changes to your branch, first make sure your working copy of the branch is “clean”—that it has no local modifications reported by svn status. Then simply run:

$ pwd
/home/user/my-calc-branch

$ svn merge http://svn.example.com/repos/calc/trunk
--- Merging r345 through r356 into '.':
U    button.c
U    integer.c

This basic syntax—svn merge URL—tells Subversion to merge all recent changes from the URL to the current working directory (which is typically the root of your working copy). After running the prior example, your branch working copy now contains new local modifications, and these edits are duplications of all of the changes that have happened on the trunk since you first created your branch:

$ svn status
 M     .
M      button.c
M      integer.c

At this point, the wise thing to do is look at the changes carefully with svn diff, and then build and test your branch. Notice that the current working directory (“.”) has also been modified; the svn diff will show that its svn:mergeinfo property has been either created or modified. This is important merge-related metadata that you should not touch, since it will be needed by future svn merge commands. (We'll learn more about this metadata later in the chapter.)

After performing the merge, you might also need to resolve some conflicts (just as you do with svn update) or possibly make some small edits to get things working properly. (Remember, just because there are no syntactic conflicts doesn't mean there aren't any semantic conflicts!) If you encounter serious problems, you can always abort the local changes by running svn revert . -R (which will undo all local modifications) and start a long “what's going on?” discussion with your collaborators. If things look good, however, you can submit these changes into the repository:

$ svn commit -m "Merged latest trunk changes to my-calc-branch."
Sending        .
Sending        button.c
Sending        integer.c
Transmitting file data ..
Committed revision 357.

At this point, your private branch is now “in sync” with the trunk, so you can rest easier knowing that as you continue to work in isolation, you're not drifting too far away from what everyone else is doing.

$ cd my-calc-branch
$ svn diff -r 341:HEAD http://svn.example.com/repos/calc/trunk > patchfile
$ patch -p0  < patchfile
Patching file integer.c using Plan A...
Hunk #1 succeeded at 147.
Hunk #2 succeeded at 164.
Hunk #3 succeeded at 241.
Hunk #4 succeeded at 249.
done

Suppose that another week has passed. You've committed more changes to your branch, and your comrades have continued to improve the trunk as well. Once again, you'd like to replicate the latest trunk changes to your branch and bring yourself in sync. Just run the same merge command again!

$ svn merge http://svn.example.com/repos/calc/trunk
--- Merging r357 through r380 into '.':
U    integer.c
U    Makefile
A    README

Subversion knows which trunk changes you've already replicated to your branch, so it carefully replicates only those changes you don't yet have. Once again, you'll have to build, test, and svn commit the local modifications to your branch.

What happens when you finally finish your work, though? Your new feature is done, and you're ready to merge your branch changes back to the trunk (so your team can enjoy the bounty of your labor). The process is simple. First, bring your branch in sync with the trunk again, just as you've been doing all along:

$ svn merge http://svn.example.com/repos/calc/trunk
--- Merging r381 through r385 into '.':
U    button.c
U    README

$ # build, test, ...

$ svn commit -m "Final merge of trunk changes to my-calc-branch."
Sending        .
Sending        button.c
Sending        README
Transmitting file data ..
Committed revision 390.

Now, you use svn merge to replicate your branch changes back into the trunk. You'll need an up-to-date working copy of /trunk. You can do this by either doing an svn checkout, dredging up an old trunk working copy from somewhere on your disk, or using svn switch (see the section called “Traversing Branches”.) However you get a trunk working copy, remember that it's a best practice to do your merge into a working copy that has no local edits and has been recently updated (i.e., is not a mixture of local revisions). If your working copy isn't “clean” in these ways, you can run into some unnecessary conflict-related headaches and svn merge will likely return an error.

Once you have a clean working copy of the trunk, you're ready to merge your branch back into it:

$ pwd
/home/user/calc-trunk

$ svn update  # (make sure the working copy is up to date)
At revision 390.

$ svn merge --reintegrate http://svn.example.com/repos/calc/branches/my-calc-branch
--- Merging differences between repository URLs into '.':
U    button.c
U    integer.c
U    Makefile
 U   .

$ # build, test, verify, ...

$ svn commit -m "Merge my-calc-branch back into trunk!"
Sending        .
Sending        button.c
Sending        integer.c
Sending        Makefile
Transmitting file data ..
Committed revision 391.

Congratulations, your branch has now been remerged back into the main line of development. Notice our use of the --reintegrate option this time around. The option is critical for reintegrating changes from a branch back into its original line of development—don't forget it! It's needed because this sort of “merge back” is a different sort of work than what you've been doing up until now. Previously, we had been asking svn merge to grab the “next set” of changes from one line of development (the trunk) and duplicate them to another (your branch). This is fairly straightforward, and each time Subversion knows how to pick up where it left off. In our prior examples, you can see that first it merges the ranges 345:356 from trunk to branch; later on, it continues by merging the next contiguously available range, 356:380. When doing the final sync, it merges the range 380:385.

When merging your branch back to the trunk, however, the underlying mathematics is quite different. Your feature branch is now a mishmosh of both duplicated trunk changes and private branch changes, so there's no simple contiguous range of revisions to copy over. By specifying the --reintegrate option, you're asking Subversion to carefully replicate only those changes unique to your branch. (And in fact, it does this by comparing the latest trunk tree with the latest branch tree: the resulting difference is exactly your branch changes!)

Now that your private branch is merged to trunk, you may wish to remove it from the repository:

$ svn delete http://svn.example.com/repos/calc/branches/my-calc-branch \
      -m "Remove my-calc-branch."
Committed revision 392.

But wait! Isn't the history of that branch valuable? What if somebody wants to audit the evolution of your feature someday and look at all of your branch changes? No need to worry. Remember that even though your branch is no longer visible in the /branches directory, its existence is still an immutable part of the repository's history. A simple svn log command on the /branches URL will show the entire history of your branch. Your branch can even be resurrected at some point, should you desire (see the section called “Resurrecting Deleted Items”).

In Subversion 1.5, once a --reintegrate merge is done from branch to trunk, the branch is no longer usable for further work. It's not able to correctly absorb new trunk changes, nor can it be properly reintegrated to trunk again. For this reason, if you want to keep working on your feature branch, we recommend destroying it and then re-creating it from the trunk:

$ svn delete http://svn.example.com/repos/calc/branches/my-calc-branch \
      -m "Remove my-calc-branch."
Committed revision 392.

$ svn copy http://svn.example.com/repos/calc/trunk \
           http://svn.example.com/repos/calc/branches/new-branch
      -m "Create a new branch from trunk."
Committed revision 393.

$ cd my-calc-branch

$ svn switch http://svn.example.com/repos/calc/branches/new-branch
Updated to revision 393.

The final command in the prior example—svn switch—is a way of updating an existing working copy to reflect a different repository directory. We'll discuss this more in the section called “Traversing Branches”.

The basic mechanism Subversion uses to track changesets—that is, which changes have been merged to which branches—is by recording data in properties. Specifically, merge data is tracked in the svn:mergeinfo property attached to files and directories. (If you're not familiar with Subversion properties, now is the time to skim the section called “Properties”.)

You can examine the property, just like any other:

$ cd my-calc-branch
$ svn propget svn:mergeinfo .
/trunk:341-390

It is not recommended that you change the value of this property yourself, unless you really know what you're doing. This property is automatically maintained by Subversion whenever you run svn merge. Its value indicates which changes (at a given path) have been replicated into the directory in question. In this case, the path is /trunk and the directory which has received the specific changes is /branches/my-calc-branch.

There's also a subcommand, svn mergeinfo, which can be helpful in seeing not only which changesets a directory has absorbed, but also which changesets it's still eligible to receive. This gives a sort of preview of the next set of changes that svn merge will replicate to your branch.

$ cd my-calc-branch

# Which changes have already been merged from trunk to branch?
$ svn mergeinfo http://svn.example.com/repos/calc/trunk
r341
r342
r343
…
r388
r389
r390

# Which changes are still eligible to merge from trunk to branch?
$ svn mergeinfo http://svn.example.com/repos/calc/trunk --show-revs eligible
r391
r392
r393
r394
r395

The svn mergeinfo command requires a “source” URL (where the changes would be coming from), and takes an optional “target” URL (where the changes would be merged to). If no target URL is given, it assumes that the current working directory is the target. In the prior example, because we're querying our branch working copy, the command assumes we're interested in receiving changes to /branches/mybranch from the specified trunk URL.

Another way to get a more precise preview of a merge operation is to use the --dry-run option:

$ svn merge http://svn.example.com/repos/calc/trunk --dry-run
U    integer.c

$ svn status
#  nothing printed, working copy is still unchanged.

The --dry-run option doesn't actually apply any local changes to the working copy. It shows only status codes that would be printed in a real merge. It's useful for getting a “high-level” preview of the potential merge, for those times when running svn diff gives too much detail.

Of course, the best way to preview a merge operation is to just do it! Remember, running svn merge isn't an inherently risky thing (unless you've made local modifications to your working copy—but we've already stressed that you shouldn't be merging into such an environment). If you don't like the results of the merge, simply run svn revert . -R to revert the changes from your working copy and retry the command with different options. The merge isn't final until you actually svn commit the results.

An extremely common use for svn merge is to roll back a change that has already been committed. Suppose you're working away happily on a working copy of /calc/trunk, and you discover that the change made way back in revision 303, which changed integer.c, is completely wrong. It never should have been committed. You can use svn merge to “undo” the change in your working copy, and then commit the local modification to the repository. All you need to do is to specify a reverse difference. (You can do this by specifying --revision 303:302, or by an equivalent --change -303.)

$ svn merge -c -303 http://svn.example.com/repos/calc/trunk
--- Reverse-merging r303 into 'integer.c':
U    integer.c

$ svn status
 M     .
M      integer.c

$ svn diff
…
# verify that the change is removed
…

$ svn commit -m "Undoing change committed in r303."
Sending        integer.c
Transmitting file data .
Committed revision 350.

As we mentioned earlier, one way to think about a repository revision is as a specific changeset. By using the -r option, you can ask svn merge to apply a changeset, or a whole range of changesets, to your working copy. In our case of undoing a change, we're asking svn merge to apply changeset #303 to our working copy backward.

Keep in mind that rolling back a change like this is just like any other svn merge operation, so you should use svn status and svn diff to confirm that your work is in the state you want it to be in, and then use svn commit to send the final version to the repository. After committing, this particular changeset is no longer reflected in the HEAD revision.

Again, you may be thinking: well, that really didn't undo the commit, did it? The change still exists in revision 303. If somebody checks out a version of the calc project between revisions 303 and 349, she'll still see the bad change, right?

Yes, that's true. When we talk about “removing” a change, we're really talking about removing it from the HEAD revision. The original change still exists in the repository's history. For most situations, this is good enough. Most people are only interested in tracking the HEAD of a project anyway. There are special cases, however, where you really might want to destroy all evidence of the commit. (Perhaps somebody accidentally committed a confidential document.) This isn't so easy, it turns out, because Subversion was deliberately designed to never lose information. Revisions are immutable trees that build upon one another. Removing a revision from history would cause a domino effect, creating chaos in all subsequent revisions and possibly invalidating all working copies. ^[21]

The great thing about version control systems is that information is never lost. Even when you delete a file or directory, it may be gone from the HEAD revision, but the object still exists in earlier revisions. One of the most common questions new users ask is, “How do I get my old file or directory back?”

The first step is to define exactly which item you're trying to resurrect. Here's a useful metaphor: you can think of every object in the repository as existing in a sort of two-dimensional coordinate system. The first coordinate is a particular revision tree, and the second coordinate is a path within that tree. So every version of your file or directory can be defined by a specific coordinate pair. (Remember the “peg revision” syntax—foo.c@224—mentioned back in the section called “Peg and Operative Revisions”.)

First, you might need to use svn log to discover the exact coordinate pair you wish to resurrect. A good strategy is to run svn log --verbose in a directory that used to contain your deleted item. The --verbose (-v) option shows a list of all changed items in each revision; all you need to do is find the revision in which you deleted the file or directory. You can do this visually, or by using another tool to examine the log output (via grep, or perhaps via an incremental search in an editor).

$ cd parent-dir
$ svn log -v
…
------------------------------------------------------------------------
r808 | joe | 2003-12-26 14:29:40 -0600 (Fri, 26 Dec 2003) | 3 lines
Changed paths:
   D /calc/trunk/real.c
   M /calc/trunk/integer.c

Added fast fourier transform functions to integer.c.
Removed real.c because code now in double.c.
…

In the example, we're assuming that you're looking for a deleted file real.c. By looking through the logs of a parent directory, you've spotted that this file was deleted in revision 808. Therefore, the last version of the file to exist was in the revision right before that. Conclusion: you want to resurrect the path /calc/trunk/real.c from revision 807.

That was the hard part—the research. Now that you know what you want to restore, you have two different choices.

One option is to use svn merge to apply revision 808 “in reverse.” (We already discussed how to undo changes in the section called “Undoing Changes”.) This would have the effect of re-adding real.c as a local modification. The file would be scheduled for addition, and after a commit, the file would again exist in HEAD.

In this particular example, however, this is probably not the best strategy. Reverse-applying revision 808 would not only schedule real.c for addition, but the log message indicates that it would also undo certain changes to integer.c, which you don't want. Certainly, you could reverse-merge revision 808 and then svn revert the local modifications to integer.c, but this technique doesn't scale well. What if 90 files were changed in revision 808?

A second, more targeted strategy is not to use svn merge at all, but rather to use the svn copy command. Simply copy the exact revision and path “coordinate pair” from the repository to your working copy:

$ svn copy http://svn.example.com/repos/calc/trunk/real.c@807 ./real.c

$ svn status
A  +   real.c

$ svn commit -m "Resurrected real.c from revision 807, /calc/trunk/real.c."
Adding         real.c
Transmitting file data .
Committed revision 1390.

The plus sign in the status output indicates that the item isn't merely scheduled for addition, but scheduled for addition “with history.” Subversion remembers where it was copied from. In the future, running svn log on this file will traverse back through the file's resurrection and through all the history it had prior to revision 807. In other words, this new real.c isn't really new; it's a direct descendant of the original, deleted file. This is usually considered a good and useful thing. If, however, you wanted to resurrect the file without maintaining a historical link to the old file, this technique works just as well:

$ svn cat http://svn.example.com/repos/calc/trunk/real.c@807 > ./real.c

$ svn add real.c
A         real.c

$ svn commit -m "Re-created real.c from revision 807."
Adding         real.c
Transmitting file data .
Committed revision 1390.

Although our example shows us resurrecting a file, note that these same techniques work just as well for resurrecting deleted directories. Also note that a resurrection doesn't have to happen in your working copy—it can happen entirely in the repository:

$ svn copy http://svn.example.com/repos/calc/trunk/real.c@807 \
           http://svn.example.com/repos/calc/trunk/ \
      -m "Resurrect real.c from revision 807."
Committed revision 1390.

$ svn update
A    real.c
Updated to revision 1390.

Just as the term “changeset” is often used in version control systems, so is the term cherrypicking. This word refers to the act of choosing one specific changeset from a branch and replicating it to another. Cherrypicking may also refer to the act of duplicating a particular set of (not necessarily contiguous!) changesets from one branch to another. This is in contrast to more typical merging scenarios, where the “next” contiguous range of revisions is duplicated automatically.

Why would people want to replicate just a single change? It comes up more often than you'd think. For example, let's go back in time and imagine that you haven't yet merged your private feature branch back to the trunk. At the water cooler, you get word that Sally made an interesting change to integer.c on the trunk. Looking over the history of commits to the trunk, you see that in revision 355 she fixed a critical bug that directly impacts the feature you're working on. You might not be ready to merge all the trunk changes to your branch just yet, but you certainly need that particular bug fix in order to continue your work.

$ svn diff -c 355 http://svn.example.com/repos/calc/trunk

Index: integer.c
===================================================================
--- integer.c	(revision 354)
+++ integer.c	(revision 355)
@@ -147,7 +147,7 @@
     case 6:  sprintf(info->operating_system, "HPFS (OS/2 or NT)"); break;
     case 7:  sprintf(info->operating_system, "Macintosh"); break;
     case 8:  sprintf(info->operating_system, "Z-System"); break;
-    case 9:  sprintf(info->operating_system, "CP/MM");
+    case 9:  sprintf(info->operating_system, "CP/M"); break;
     case 10:  sprintf(info->operating_system, "TOPS-20"); break;
     case 11:  sprintf(info->operating_system, "NTFS (Windows NT)"); break;
     case 12:  sprintf(info->operating_system, "QDOS"); break;

Just as you used svn diff in the prior example to examine revision 355, you can pass the same option to svn merge:

$ svn merge -c 355 http://svn.example.com/repos/calc/trunk
U    integer.c

$ svn status
M      integer.c

You can now go through the usual testing procedures before committing this change to your branch. After the commit, Subversion marks r355 as having been merged to the branch so that future “magic” merges that synchronize your branch with the trunk know to skip over r355. (Merging the same change to the same branch almost always results in a conflict!)

$ cd my-calc-branch

$ svn propget svn:mergeinfo .
/trunk:341-349,355

# Notice that r355 isn't listed as "eligible" to merge, because
# it's already been merged.
$ svn mergeinfo http://svn.example.com/repos/calc/trunk --show-revs eligible
r350
r351
r352
r353
r354
r356
r357
r358
r359
r360

$ svn merge http://svn.example.com/repos/calc/trunk
--- Merging r350 through r354 into '.':
 U   .
U    integer.c
U    Makefile
--- Merging r356 through r360 into '.':
 U   .
U    integer.c
U    button.c

This use case of replicating (or backporting) bug fixes from one branch to another is perhaps the most popular reason for cherrypicking changes; it comes up all the time, for example, when a team is maintaining a “release branch” of software. (We discuss this pattern in the section called “Release Branches”.)

A word of warning: while svn diff and svn merge are very similar in concept, they do have different syntax in many cases. Be sure to read about them in Chapter 9, Subversion Complete Reference for details, or ask svn help. For example, svn merge requires a working copy path as a target, that is, a place where it should apply the generated patch. If the target isn't specified, it assumes you are trying to perform one of the following common operations:

If you are merging a directory and haven't specified a target path, svn merge assumes the first case and tries to apply the changes into your current directory. If you are merging a file, and that file (or a file by the same name) exists in your current working directory, svn merge assumes the second case and tries to apply the changes to a local file with the same name.

You've now seen some examples of the svn merge command, and you're about to see several more. If you're feeling confused about exactly how merging works, you're not alone. Many users (especially those new to version control) are initially perplexed about the proper syntax of the command and about how and when the feature should be used. But fear not, this command is actually much simpler than you think! There's a very easy technique for understanding exactly how svn merge behaves.

The main source of confusion is the name of the command. The term “merge” somehow denotes that branches are combined together, or that some sort of mysterious blending of data is going on. That's not the case. A better name for the command might have been svn diff-and-apply, because that's all that happens: two repository trees are compared, and the differences are applied to a working copy.

If you're using svn merge to do basic copying of changes between branches, it will generally do the right thing automatically. For example, a command such as the following:

$ svn merge http://svn.example.com/repos/calc/some-branch

will attempt to duplicate any changes made on some-branch into your current working directory, which is presumably a working copy that shares some historical connection to the branch. The command is smart enough to only duplicate changes that your working copy doesn't yet have. If you repeat this command once a week, it will only duplicate the “newest” branch changes that happened since you last merged.

If you choose to use the svn merge command in all its full glory by giving it specific revision ranges to duplicate, the command takes three main arguments:

Once these three arguments are specified, the two trees are compared, and the differences are applied to the target working copy as local modifications. When the command is done, the results are no different than if you had hand-edited the files or run various svn add or svn delete commands yourself. If you like the results, you can commit them. If you don't like the results, you can simply svn revert all of the changes.

The syntax of svn merge allows you to specify the three necessary arguments rather flexibly. Here are some examples:

$ svn merge http://svn.example.com/repos/branch1@150 \
            http://svn.example.com/repos/branch2@212 \
            my-working-copy

$ svn merge -r 100:200 http://svn.example.com/repos/trunk my-working-copy

$ svn merge -r 100:200 http://svn.example.com/repos/trunk

The first syntax lays out all three arguments explicitly, naming each tree in the form URL@REV and naming the working copy target. The second syntax can be used as a shorthand for situations when you're comparing two different revisions of the same URL. The last syntax shows how the working copy argument is optional; if omitted, it defaults to the current directory.

While the first example shows the “full” syntax of svn merge, it needs to be used very carefully; it can result in merges which do not record any svn:mergeinfo metadata at all. The next section talks a bit more about this.

Subversion tries to generate merge metadata whenever it can, to make future invocations of svn merge smarter. There are still situations, however, where svn:mergeinfo data is not created or changed. Remember to be a bit wary of these scenarios:

Just like the svn update command, svn merge applies changes to your working copy. And therefore it's also capable of creating conflicts. The conflicts produced by svn merge, however, are sometimes different, and this section explains those differences.

To begin with, assume that your working copy has no local edits. When you svn update to a particular revision, the changes sent by the server will always apply “cleanly” to your working copy. The server produces the delta by comparing two trees: a virtual snapshot of your working copy, and the revision tree you're interested in. Because the left hand side of the comparison is exactly equal to what you already have, the delta is guaranteed to correctly convert your working copy into the right hand tree.

But svn merge has no such guarantees and can be much more chaotic: the advanced user can ask the server to compare any two trees at all, even ones that are unrelated to the working copy! This means there's large potential for human error. Users will sometimes compare the wrong two trees, creating a delta that doesn't apply cleanly. svn merge will do its best to apply as much of the delta as possible, but some parts may be impossible. Just as the Unix patch command sometimes complains about “failed hunks,” svn merge will similarly complain about “skipped targets”:

$ svn merge -r 1288:1351 http://svn.example.com/repos/branch
U    foo.c
U    bar.c
Skipped missing target: 'baz.c'
U    glub.c
U    sputter.h

Conflict discovered in 'glorb.h'.
Select: (p) postpone, (df) diff-full, (e) edit,
        (h) help for more options:

In the previous example, it might be the case that baz.c exists in both snapshots of the branch being compared, and the resultant delta wants to change the file's contents, but the file doesn't exist in the working copy. Whatever the case, the “skipped” message means that the user is most likely comparing the wrong two trees; it's the classic sign of user error. When this happens, it's easy to recursively revert all the changes created by the merge (svn revert . --recursive), delete any unversioned files or directories left behind after the revert, and rerun svn merge with different arguments.

Also notice that the preceeding example shows a conflict happening on glorb.h. We already stated that the working copy has no local edits: how can a conflict possibly happen? Again, because the user can use svn merge to define and apply any old delta to the working copy, that delta may contain textual changes that don't cleanly apply to a working file, even if the file has no local modifications.

Another small difference between svn update and svn merge is the names of the full-text files created when a conflict happens. In the section called “Resolve Conflicts (Merging Others' Changes)”, we saw that an update produces files named filename.mine, filename.rOLDREV, and filename.rNEWREV. When svn merge produces a conflict, though, it creates three files named filename.working, filename.left, and filename.right. In this case, the terms “left” and “right” are describing which side of the double-tree comparison the file came from. In any case, these differing names will help you distinguish between conflicts that happened as a result of an update and ones that happened as a result of a merge.

Sometimes there's a particular changeset that you don't want to be automatically merged. For example, perhaps your team's policy is to do new development work on /trunk, but to be more conservative about backporting changes to a stable branch you use for releasing to the public. On one extreme, you can manually cherrypick single changesets from the trunk to the branch—just the changes that are stable enough to pass muster. Maybe things aren't quite that strict, though; perhaps most of the time you'd like to just let svn merge automatically merge most changes from trunk to branch. In this case, you'd like a way to mask a few specific changes out, that is, prevent them from ever being automatically merged.

In Subversion 1.5, the only way to block a changeset is to make the system believe that the change has already been merged. To do this, one can invoke a merge command with the --record-only option:

$ cd my-calc-branch

$ svn propget svn:mergeinfo .
/trunk:1680-3305

# Let's make the metadata list r3328 as already merged.
$ svn merge -c 3328 --record-only http://svn.example.com/repos/calc/trunk

$ svn status
M     .

$ svn propget svn:mergeinfo .
/trunk:1680-3305,3328

$ svn commit -m "Block r3328 from being merged to the branch."
…

This technique works, but it's also a little bit dangerous. The main problem is that we're not clearly differentiating between the ideas of “I already have this change” and “I don't have this change.” We're effectively lying to the system, making it think that the change was previously merged. This puts the responsibility on you—the user—to remember that the change wasn't actually merged, it just wasn't wanted. There's no way to ask Subversion for a list of “blocked changelists.” If you want to track them (so that you can unblock them someday). you'll need to record them in a text file somewhere, or perhaps in an invented property. In Subversion 1.5, unfortunately, this is the only way to manage blocked revisions; the plans are to make a better interface for this in future versions.

One of the main features of any version control system is to keep track of who changed what, and when they did it. The svn log and svn blame commands are just the tools for this: when invoked on individual files, they show not only the history of changesets that affected the file, but also exactly which user wrote which line of code, and when she did it.

When changes start getting replicated between branches, however, things start to get complicated. For example, if you were to ask svn log about the history of your feature branch, it would show exactly every revision that ever affected the branch:

$ cd my-calc-branch
$ svn log -q
------------------------------------------------------------------------
r390 | user | 2002-11-22 11:01:57 -0600 (Fri, 22 Nov 2002) | 1 line
------------------------------------------------------------------------
r388 | user | 2002-11-21 05:20:00 -0600 (Thu, 21 Nov 2002) | 2 lines
------------------------------------------------------------------------
r381 | user | 2002-11-20 15:07:06 -0600 (Wed, 20 Nov 2002) | 2 lines
------------------------------------------------------------------------
r359 | user | 2002-11-19 19:19:20 -0600 (Tue, 19 Nov 2002) | 2 lines
------------------------------------------------------------------------
r357 | user | 2002-11-15 14:29:52 -0600 (Fri, 15 Nov 2002) | 2 lines
------------------------------------------------------------------------
r343 | user | 2002-11-07 13:50:10 -0600 (Thu, 07 Nov 2002) | 2 lines
------------------------------------------------------------------------
r341 | user | 2002-11-03 07:17:16 -0600 (Sun, 03 Nov 2002) | 2 lines
------------------------------------------------------------------------
r303 | sally | 2002-10-29 21:14:35 -0600 (Tue, 29 Oct 2002) | 2 lines
------------------------------------------------------------------------
r98 | sally | 2002-02-22 15:35:29 -0600 (Fri, 22 Feb 2002) | 2 lines
------------------------------------------------------------------------

But is this really an accurate picture of all the changes that happened on the branch? What's being left out here is the fact that revisions 390, 381, and 357 were actually the results of merging changes from the trunk. If you look at one of these logs in detail, the multiple trunk changesets that comprised the branch change are nowhere to be seen:

$ svn log -v -r 390
------------------------------------------------------------------------
r390 | user | 2002-11-22 11:01:57 -0600 (Fri, 22 Nov 2002) | 1 line
Changed paths:
   M /branches/my-calc-branch/button.c
   M /branches/my-calc-branch/README

Final merge of trunk changes to my-calc-branch.

We happen to know that this merge to the branch was nothing but a merge of trunk changes. How can we see those trunk changes as well? The answer is to use the --use-merge-history (-g) option. This option expands those “child” changes that were part of the merge.

$ svn log -v -r 390 -g
------------------------------------------------------------------------
r390 | user | 2002-11-22 11:01:57 -0600 (Fri, 22 Nov 2002) | 1 line
Changed paths:
   M /branches/my-calc-branch/button.c
   M /branches/my-calc-branch/README

Final merge of trunk changes to my-calc-branch.
------------------------------------------------------------------------
r383 | sally | 2002-11-21 03:19:00 -0600 (Thu, 21 Nov 2002) | 2 lines
Changed paths:
   M /branches/my-calc-branch/button.c
Merged via: r390

Fix inverse graphic error on button.
------------------------------------------------------------------------
r382 | sally | 2002-11-20 16:57:06 -0600 (Wed, 20 Nov 2002) | 2 lines
Changed paths:
   M /branches/my-calc-branch/README
Merged via: r390

Document my last fix in README.

By making the log operation use merge history, we see not just the revision we queried (r390), but also the two revisions that came along on the ride with it—a couple of changes made by Sally to the trunk. This is a much more complete picture of history!

The svn blame command also takes the --use-merge-history (-g) option. If this option is neglected, somebody looking at a line-by-line annotation of button.c may get the mistaken impression that you were responsible for the lines that fixed a certain error:

$ svn blame button.c
…
   390    user    retval = inverse_func(button, path);
   390    user    return retval;
   390    user    }
…

And while it's true that you did actually commit those three lines in revision 390, two of them were actually written by Sally back in revision 383:

$ svn blame button.c -g
…
G    383    sally   retval = inverse_func(button, path);
G    383    sally   return retval;
     390    user    }
…

Now we know who to really blame for those two lines of code!

When conversing with a Subversion developer, you might very likely hear reference to the term ancestry. This word is used to describe the relationship between two objects in a repository: if they're related to each other, one object is said to be an ancestor of the other.

For example, suppose you commit revision 100, which includes a change to a file foo.c. Then foo.c@99 is an “ancestor” of foo.c@100. On the other hand, suppose you commit the deletion of foo.c in revision 101, and then add a new file by the same name in revision 102. In this case, foo.c@99 and foo.c@102 may appear to be related (they have the same path), but in fact are completely different objects in the repository. They share no history or “ancestry.”

The reason for bringing this up is to point out an important difference between svn diff and svn merge. The former command ignores ancestry, while the latter command is quite sensitive to it. For example, if you asked svn diff to compare revisions 99 and 102 of foo.c, you would see line-based diffs; the diff command is blindly comparing two paths. But if you asked svn merge to compare the same two objects, it would notice that they're unrelated and first attempt to delete the old file, then add the new file; the output would indicate a deletion followed by an add:

D    foo.c
A    foo.c
      

Most merges involve comparing trees that are ancestrally related to one another; therefore, svn merge defaults to this behavior. Occasionally, however, you may want the merge command to compare two unrelated trees. For example, you may have imported two source-code trees representing different vendor releases of a software project (see the section called “Vendor Branches”). If you ask svn merge to compare the two trees, you'd see the entire first tree being deleted, followed by an add of the entire second tree! In these situations, you'll want svn merge to do a path-based comparison only, ignoring any relations between files and directories. Add the --ignore-ancestry option to your merge command, and it will behave just like svn diff. (And conversely, the --notice-ancestry option will cause svn diff to behave like the svn merge command.)

A common desire is to refactor source code, especially in Java-based software projects. Files and directories are shuffled around and renamed, often causing great disruption to everyone working on the project. Sounds like a perfect case to use a branch, doesn't it? Just create a branch, shuffle things around, and then merge the branch back to the trunk, right?

Alas, this scenario doesn't work so well right now and is considered one of Subversion's current weak spots. The problem is that Subversion's svn update command isn't as robust as it should be, particularly when dealing with copy and move operations.

When you use svn copy to duplicate a file, the repository remembers where the new file came from, but it fails to transmit that information to the client which is running svn update or svn merge. Instead of telling the client, “Copy that file you already have to this new location,” it sends down an entirely new file. This can lead to problems, especially because the same thing happens with renamed files. A lesser-known fact about Subversion is that it lacks “true renames”—the svn move command is nothing more than an aggregation of svn copy and svn delete.

For example, suppose that while working on your private branch, you rename integer.c to whole.c. Effectively you've created a new file in your branch that is a copy of the original file, and deleted the original file. Meanwhile, back on trunk, Sally has committed some improvements to integer.c. Now you decide to merge your branch to the trunk:

$ cd calc/trunk

$ svn merge --reintegrate http://svn.example.com/repos/calc/branches/my-calc-branch
--- Merging differences between repository URLs into '.':
D   integer.c
A   whole.c
U   .
      

This doesn't look so bad at first glance, but it's also probably not what you or Sally expected. The merge operation has deleted the latest version of the integer.c file (the one containing Sally's latest changes), and blindly added your new whole.c file—which is a duplicate of the older version of integer.c. The net effect is that merging your “rename” to the branch has removed Sally's recent changes from the latest revision!

This isn't true data loss. Sally's changes are still in the repository's history, but it may not be immediately obvious that this has happened. The moral of this story is that until Subversion improves, be very careful about merging copies and renames from one branch to another.

If you've just upgraded your server to Subversion 1.5 or later, there's a significant risk that pre-1.5 Subversion clients can mess up your automated merge tracking. Why is this? When a pre-1.5 Subversion client performs svn merge, it doesn't modify the value of the svn:mergeinfo property at all. So the subsequent commit, despite being the result of a merge, doesn't tell the repository about the duplicated changes—that information is lost. Later on, when “merge-aware” clients attempt automatic merging, they're likely to run into all sorts of conflicts resulting from repeated merges.

If you and your team are relying on the merge-tracking features of Subversion, you may want to configure your repository to prevent older clients from committing changes. The easy way to do this is by inspecting the “capabilities” parameter in the start-commit hook script. If the client reports itself as having mergeinfo capabilities, the hook script can allow the commit to start. If the client doesn't report that capability, have the hook deny the commit. We'll learn more about hook scripts in the next chapter; see the section called “Implementing Repository Hooks” and start-commit for details.

The bottom line is that Subversion's merge-tracking feature has an extremely complex internal implementation, and the svn:mergeinfo property is the only window the user has into the machinery. Because the feature is relatively new, a numbers of edge cases and possible unexpected behaviors may pop up.

For example, sometimes mergeinfo will be generated when running a simple svn copy or svn move command. Sometimes mergeinfo will appear on files that you didn't expect to be touched by an operation. Sometimes mergeinfo won't be generated at all, when you expect it to. Furthermore, the management of mergeinfo metadata has a whole set of taxonomies and behaviors around it, such as “explicit” versus “implicit” mergeinfo, “operative” versus “inoperative” revisions, specific mechanisms of mergeinfo “elision,” and even “inheritance” from parent to child directories.

We've chosen not to cover these detailed topics in this book for a couple of reasons. First, the level of detail is absolutely overwhelming for a typical user. Second, as Subversion continues to improve, we feel that a typical user shouldn't have to understand these concepts; they'll eventually fade into the background as pesky implementation details. All that said, if you enjoy this sort of thing, you can get a fantastic overview in a paper posted at CollabNet's website: http://www.collab.net/community/subversion/articles/merge-info.html.

For now, if you want to steer clear of bugs and odd behaviors in automatic merging, the CollabNet article recommends that you stick to these simple best practices:

Once again, svn copy comes to the rescue. If you want to create a snapshot of /calc/trunk exactly as it looks in the HEAD revision, make a copy of it:

$ svn copy http://svn.example.com/repos/calc/trunk \
           http://svn.example.com/repos/calc/tags/release-1.0 \
      -m "Tagging the 1.0 release of the 'calc' project."

Committed revision 902.

This example assumes that a /calc/tags directory already exists. (If it doesn't, you can create it using svn mkdir.) After the copy completes, the new release-1.0 directory is forever a snapshot of how the /trunk directory looked in the HEAD revision at the time you made the copy. Of course, you might want to be more precise about exactly which revision you copy, in case somebody else may have committed changes to the project when you weren't looking. So if you know that revision 901 of /calc/trunk is exactly the snapshot you want, you can specify it by passing -r 901 to the svn copy command.

But wait a moment: isn't this tag creation procedure the same procedure we used to create a branch? Yes, in fact, it is. In Subversion, there's no difference between a tag and a branch. Both are just ordinary directories that are created by copying. Just as with branches, the only reason a copied directory is a “tag” is because humans have decided to treat it that way: as long as nobody ever commits to the directory, it forever remains a snapshot. If people start committing to it, it becomes a branch.

If you are administering a repository, there are two approaches you can take to managing tags. The first approach is “hands off”: as a matter of project policy, decide where your tags will live, and make sure all users know how to treat the directories they copy. (That is, make sure they know not to commit to them.) The second approach is more paranoid: you can use one of the access control scripts provided with Subversion to prevent anyone from doing anything but creating new copies in the tags area (see Chapter 6, Server Configuration). The paranoid approach, however, isn't usually necessary. If a user accidentally commits a change to a tag directory, you can simply undo the change as discussed in the previous section. This is version control, after all!

Sometimes you may want your “snapshot” to be more complicated than a single directory at a single revision.

For example, pretend your project is much larger than our calc example: suppose it contains a number of subdirectories and many more files. In the course of your work, you may decide that you need to create a working copy that is designed to have specific features and bug fixes. You can accomplish this by selectively backdating files or directories to particular revisions (using svn update with the -r option liberally), by switching files and directories to particular branches (making use of svn switch), or even just by making a bunch of local changes. When you're done, your working copy is a hodgepodge of repository locations from different revisions. But after testing, you know it's the precise combination of data you need to tag.

Time to make a snapshot. Copying one URL to another won't work here. In this case, you want to make a snapshot of your exact working copy arrangement and store it in the repository. Luckily, svn copy actually has four different uses (which you can read about in Chapter 9, Subversion Complete Reference), including the ability to copy a working copy tree to the repository:

$ ls
my-working-copy/

$ svn copy my-working-copy \
           http://svn.example.com/repos/calc/tags/mytag \
           -m "Tag my existing working copy state."

Committed revision 940.

Now there is a new directory in the repository, /calc/tags/mytag, which is an exact snapshot of your working copy—mixed revisions, URLs, local changes, and all.

Other users have found interesting uses for this feature. Sometimes there are situations where you have a bunch of local changes made to your working copy, and you'd like a collaborator to see them. Instead of running svn diff and sending a patch file (which won't capture directory, symlink, or property changes), you can use svn copy to “upload” your working copy to a private area of the repository. Your collaborator can then either check out a verbatim copy of your working copy or use svn merge to receive your exact changes.

While this is a nice method for uploading a quick snapshot of your working copy, note that this is not a good way to initially create a branch. Branch creation should be an event unto itself, and this method conflates the creation of a branch with extra changes to files, all within a single revision. This makes it very difficult (later on) to identify a single revision number as a branch point.

There are some standard, recommended ways to organize a repository. Most people create a trunk directory to hold the “main line” of development, a branches directory to contain branch copies, and a tags directory to contain tag copies. If a repository holds only one project, often people create these top-level directories:

/trunk
/branches
/tags

If a repository contains multiple projects, admins typically index their layout by project (see the section called “Planning Your Repository Organization” to read more about “project roots”):

/paint/trunk
/paint/branches
/paint/tags
/calc/trunk
/calc/branches
/calc/tags

Of course, you're free to ignore these common layouts. You can create any sort of variation, whatever works best for you or your team. Remember that whatever you choose, it's not a permanent commitment. You can reorganize your repository at any time. Because branches and tags are ordinary directories, the svn move command can move or rename them however you wish. Switching from one layout to another is just a matter of issuing a series of server-side moves; if you don't like the way things are organized in the repository, just juggle the directories around.

Remember, though, that while moving directories may be easy to do, you need to be considerate of your users as well. Your juggling can be disorienting to users with existing working copies. If a user has a working copy of a particular repository directory, your svn move operation might remove the path from the latest revision. When the user next runs svn update, she will be told that her working copy represents a path that no longer exists, and the user will be forced to svn switch to the new location.

Another nice feature of Subversion's model is that branches and tags can have finite lifetimes, just like any other versioned item. For example, suppose you eventually finish all your work on your personal branch of the calc project. After merging all of your changes back into /calc/trunk, there's no need for your private branch directory to stick around anymore:

$ svn delete http://svn.example.com/repos/calc/branches/my-calc-branch \
             -m "Removing obsolete branch of calc project."

Committed revision 375.

And now your branch is gone. Of course, it's not really gone: the directory is simply missing from the HEAD revision, no longer distracting anyone. If you use svn checkout, svn switch, or svn list to examine an earlier revision, you'll still be able to see your old branch.

If browsing your deleted directory isn't enough, you can always bring it back. Resurrecting data is very easy in Subversion. If there's a deleted directory (or file) that you'd like to bring back into HEAD, simply use svn copy to copy it from the old revision:

$ svn copy http://svn.example.com/repos/calc/branches/my-calc-branch@374 \
           http://svn.example.com/repos/calc/branches/my-calc-branch \
           -m "Restore my-calc-branch."

Committed revision 376.

In our example, your personal branch had a relatively short lifetime: you may have created it to fix a bug or implement a new feature. When your task is done, so is the branch. In software development, though, it's also common to have two “main” branches running side by side for very long periods. For example, suppose it's time to release a stable version of the calc project to the public, and you know it's going to take a couple of months to shake bugs out of the software. You don't want people to add new features to the project, but you don't want to tell all developers to stop programming either. So instead, you create a “stable” branch of the software that won't change much:

$ svn copy http://svn.example.com/repos/calc/trunk \
           http://svn.example.com/repos/calc/branches/stable-1.0 \
           -m "Creating stable branch of calc project."

Committed revision 377.

And now developers are free to continue adding cutting-edge (or experimental) features to /calc/trunk, and you can declare a project policy that only bug fixes are to be committed to /calc/branches/stable-1.0. That is, as people continue to work on the trunk, a human selectively ports bug fixes over to the stable branch. Even after the stable branch has shipped, you'll probably continue to maintain the branch for a long time—that is, as long as you continue to support that release for customers. We'll discuss this more in the next section.

Most software has a typical life cycle: code, test, release, repeat. There are two problems with this process. First, developers need to keep writing new features while quality assurance teams take time to test supposedly stable versions of the software. New work cannot halt while the software is tested. Second, the team almost always needs to support older, released versions of software; if a bug is discovered in the latest code, it most likely exists in released versions as well, and customers will want to get that bug fix without having to wait for a major new release.

Here's where version control can help. The typical procedure looks like this:

This entire process repeats as the software matures: when the 2.0 work is complete, a new 2.0 release branch is created, tested, tagged, and eventually released. After some years, the repository ends up with a number of release branches in “maintenance” mode, and a number of tags representing final shipped versions.

A feature branch is the sort of branch that's been the dominant example in this chapter (the one you've been working on while Sally continues to work on /trunk). It's a temporary branch created to work on a complex change without interfering with the stability of /trunk. Unlike release branches (which may need to be supported forever), feature branches are born, used for a while, merged back to the trunk, and then ultimately deleted. They have a finite span of usefulness.

Again, project policies vary widely concerning exactly when it's appropriate to create a feature branch. Some projects never use feature branches at all: commits to /trunk are a free-for-all. The advantage to this system is that it's simple—nobody needs to learn about branching or merging. The disadvantage is that the trunk code is often unstable or unusable. Other projects use branches to an extreme: no change is ever committed to the trunk directly. Even the most trivial changes are created on a short-lived branch, carefully reviewed, and merged to the trunk. Then the branch is deleted. This system guarantees an exceptionally stable and usable trunk at all times, but at the cost of tremendous process overhead.

Most projects take a middle-of-the-road approach. They commonly insist that /trunk compile and pass regression tests at all times. A feature branch is required only when a change requires a large number of destabilizing commits. A good rule of thumb is to ask this question: if the developer worked for days in isolation and then committed the large change all at once (so that /trunk were never destabilized), would it be too large a change to review? If the answer to that question is “yes,” the change should be developed on a feature branch. As the developer commits incremental changes to the branch, they can be easily reviewed by peers.

Finally, there's the issue of how to best keep a feature branch in “sync” with the trunk as work progresses. As we mentioned earlier, there's a great risk to working on a branch for weeks or months; trunk changes may continue to pour in, to the point where the two lines of development differ so greatly that it may become a nightmare trying to merge the branch back to the trunk.

This situation is best avoided by regularly merging trunk changes to the branch. Make up a policy: once a week, merge the last week's worth of trunk changes to the branch.

At some point, you'll be ready to merge the “synchronized” feature branch back to the trunk. To do this, begin by doing a final merge of the latest trunk changes to the branch. When that's done, the latest versions of branch and trunk will be absolutely identical except for your branch changes. You would then merge back with the --reintegrate option:

$ cd trunk-working-copy

$ svn update
At revision 1910.

$ svn merge --reintegrate http://svn.example.com/repos/calc/branches/mybranch
--- Merging differences between repository URLs into '.':
U    real.c
U    integer.c
A    newdirectory
A    newdirectory/newfile
 U   .
…

Another way of thinking about this pattern is that your weekly sync of trunk to branch is analogous to running svn update in a working copy, while the final merge step is analogous to running svn commit from a working copy. After all, what else is a working copy but a very shallow private branch? It's a branch that's capable of storing only one change at a time.

Managing vendor branches generally works like this: first, you create a top-level directory (such as /vendor) to hold the vendor branches. Then you import the third-party code into a subdirectory of that top-level directory. You then copy that subdirectory into your main development branch (e.g., /trunk) at the appropriate location. You always make your local changes in the main development branch. With each new release of the code you are tracking, you bring it into the vendor branch and merge the changes into /trunk, resolving whatever conflicts occur between your local changes and the upstream changes.

An example will help to clarify this algorithm. We'll use a scenario where your development team is creating a calculator program that links against a third-party complex number arithmetic library, libcomplex. We'll begin with the initial creation of the vendor branch and the import of the first vendor drop. We'll call our vendor branch directory libcomplex, and our code drops will go into a subdirectory of our vendor branch called current. And since svn import creates all the intermediate parent directories it needs, we can actually accomplish both of these steps with a single command:

$ svn import /path/to/libcomplex-1.0 \
             http://svn.example.com/repos/vendor/libcomplex/current \
             -m 'importing initial 1.0 vendor drop'
…

We now have the current version of the libcomplex source code in /vendor/libcomplex/current. Now, we tag that version (see the section called “Tags”) and then copy it into the main development branch. Our copy will create a new directory called libcomplex in our existing calc project directory. It is in this copied version of the vendor data that we will make our customizations:

$ svn copy http://svn.example.com/repos/vendor/libcomplex/current  \
           http://svn.example.com/repos/vendor/libcomplex/1.0      \
           -m 'tagging libcomplex-1.0'
…
$ svn copy http://svn.example.com/repos/vendor/libcomplex/1.0  \
           http://svn.example.com/repos/calc/libcomplex        \
           -m 'bringing libcomplex-1.0 into the main branch'
…

We check out our project's main branch—which now includes a copy of the first vendor drop—and we get to work customizing the libcomplex code. Before we know it, our modified version of libcomplex is now completely integrated into our calculator program. ^[25]

A few weeks later, the developers of libcomplex release a new version of their library—version 1.1—which contains some features and functionality that we really want. We'd like to upgrade to this new version, but without losing the customizations we made to the existing version. What we essentially would like to do is to replace our current baseline version of libcomplex 1.0 with a copy of libcomplex 1.1, and then re-apply the custom modifications we previously made to that library to the new version. But we actually approach the problem from the other direction, applying the changes made to libcomplex between versions 1.0 and 1.1 to our modified copy of it.

To perform this upgrade, we check out a copy of our vendor branch and replace the code in the current directory with the new libcomplex 1.1 source code. We quite literally copy new files on top of existing files, perhaps exploding the libcomplex 1.1 release tarball atop our existing files and directories. The goal here is to make our current directory contain only the libcomplex 1.1 code and to ensure that all that code is under version control. Oh, and we want to do this with as little version control history disturbance as possible.

After replacing the 1.0 code with 1.1 code, svn status will show files with local modifications as well as, perhaps, some unversioned files. If we did what we were supposed to do, the unversioned files are only those new files introduced in the 1.1 release of libcomplex—we run svn add on those to get them under version control. If the 1.1 code no longer has certain files that were in the 1.0 tree, it may be hard to notice them; you'd have to compare the two trees with some external tool and then svn delete any files present in 1.0 but not in 1.1. (Although it might also be just fine to let these same files live on in unused obscurity!) Finally, once our current working copy contains only the libcomplex 1.1 code, we commit the changes we made to get it looking that way.

Our current branch now contains the new vendor drop. We tag the new version as 1.1 (in the same way we previously tagged the version 1.0 vendor drop), and then merge the differences between the tag of the previous version and the new current version into our main development branch:

$ cd working-copies/calc
$ svn merge http://svn.example.com/repos/vendor/libcomplex/1.0      \
            http://svn.example.com/repos/vendor/libcomplex/current  \
            libcomplex
… # resolve all the conflicts between their changes and our changes
$ svn commit -m 'merging libcomplex-1.1 into the main branch'
…

In the trivial use case, the new version of our third-party tool would look, from a files-and-directories point of view, just like the previous version. None of the libcomplex source files would have been deleted, renamed, or moved to different locations—the new version would contain only textual modifications against the previous one. In a perfect world, our modifications would apply cleanly to the new version of the library, with absolutely no complications or conflicts.

But things aren't always that simple, and in fact it is quite common for source files to get moved around between releases of software. This complicates the process of ensuring that our modifications are still valid for the new version of code, and things can quickly degrade into a situation where we have to manually re-create our customizations in the new version. Once Subversion knows about the history of a given source file—including all its previous locations—the process of merging in the new version of the library is pretty simple. But we are responsible for telling Subversion how the source file layout changed from vendor drop to vendor drop.

Vendor drops that contain more than a few deletes, additions, and moves complicate the process of upgrading to each successive version of the third-party data. So Subversion supplies the svn_load_dirs.pl script to assist with this process. This script automates the importing steps we mentioned in the general vendor branch management procedure to make sure mistakes are minimized. You will still be responsible for using the merge commands to merge the new versions of the third-party data into your main development branch, but svn_load_dirs.pl can help you more quickly and easily arrive at that stage.

In short, svn_load_dirs.pl is an enhancement to svn import that has several important characteristics:

svn_load_dirs.pl takes three mandatory arguments. The first argument is the URL to the base Subversion directory to work in. This argument is followed by the URL—relative to the first argument—into which the current vendor drop will be imported. Finally, the third argument is the local directory to import. Using our previous example, a typical run of svn_load_dirs.pl might look like this:

$ svn_load_dirs.pl http://svn.example.com/repos/vendor/libcomplex \
                   current                                        \
                   /path/to/libcomplex-1.1
…

You can indicate that you'd like svn_load_dirs.pl to tag the new vendor drop by passing the -t command-line option and specifying a tag name. This tag is another URL relative to the first program argument.

$ svn_load_dirs.pl -t libcomplex-1.1                              \
                   http://svn.example.com/repos/vendor/libcomplex \
                   current                                        \
                   /path/to/libcomplex-1.1
…

When you run svn_load_dirs.pl, it examines the contents of your existing “current” vendor drop and compares them with the proposed new vendor drop. In the trivial case, no files will be in one version and not the other, and the script will perform the new import without incident. If, however, there are discrepancies in the file layouts between versions, svn_load_dirs.pl will ask you how to resolve those differences. For example, you will have the opportunity to tell the script that you know that the file math.c in version 1.0 of libcomplex was renamed to arithmetic.c in libcomplex 1.1. Any discrepancies not explained by moves are treated as regular additions and deletions.

The script also accepts a separate configuration file for setting properties on files and directories matching a regular expression that are added to the repository. This configuration file is specified to svn_load_dirs.pl using the -p command-line option. Each line of the configuration file is a whitespace-delimited set of two or four values: a Perl-style regular expression against which to match the added path, a control keyword (either break or cont), and then optionally a property name and value.

\.png$              break   svn:mime-type   image/png
\.jpe?g$            break   svn:mime-type   image/jpeg
\.m3u$              cont    svn:mime-type   audio/x-mpegurl
\.m3u$              break   svn:eol-style   LF
.*                  break   svn:eol-style   native

For each added path, the configured property changes whose regular expression matches the path are applied in order, unless the control specification is break (which means that no more property changes should be applied to that path). If the control specification is cont—an abbreviation for continue—matching will continue with the next line of the configuration file.

Any whitespace in the regular expression, property name, or property value must be surrounded by either single or double quotes. You can escape quotes that are not used for wrapping whitespace by preceding them with a backslash (\) character. The backslash escapes only quotes when parsing the configuration file, so do not protect any other characters beyond what is necessary for the regular expression.

While Subversion allows you to move around versioned files and directories without any loss of information, and even provides ways of moving whole sets of versioned history from one repository to another, doing so can greatly disrupt the workflow of those who access the repository often and come to expect things to be at certain locations. So before creating a new repository, try to peer into the future a bit; plan ahead before placing your data under version control. By conscientiously “laying out” your repository or repositories and their versioned contents ahead of time, you can prevent many future headaches.

Let's assume that as repository administrator, you will be responsible for supporting the version control system for several projects. Your first decision is whether to use a single repository for multiple projects, or to give each project its own repository, or some compromise of these two.

There are benefits to using a single repository for multiple projects, most obviously the lack of duplicated maintenance. A single repository means that there is one set of hook programs, one thing to routinely back up, one thing to dump and load if Subversion releases an incompatible new version, and so on. Also, you can move data between projects easily, without losing any historical versioning information.

The downside of using a single repository is that different projects may have different requirements in terms of the repository event triggers, such as needing to send commit notification emails to different mailing lists, or having different definitions about what does and does not constitute a legitimate commit. These aren't insurmountable problems, of course—it just means that all of your hook scripts have to be sensitive to the layout of your repository rather than assuming that the whole repository is associated with a single group of people. Also, remember that Subversion uses repository-global revision numbers. While those numbers don't have any particular magical powers, some folks still don't like the fact that even though no changes have been made to their project lately, the youngest revision number for the repository keeps climbing because other projects are actively adding new revisions. ^[27]

A middle-ground approach can be taken, too. For example, projects can be grouped by how well they relate to each other. You might have a few repositories with a handful of projects in each repository. That way, projects that are likely to want to share data can do so easily, and as new revisions are added to the repository, at least the developers know that those new revisions are at least remotely related to everyone who uses that repository.

After deciding how to organize your projects with respect to repositories, you'll probably want to think about directory hierarchies within the repositories themselves. Because Subversion uses regular directory copies for branching and tagging (see Chapter 4, Branching and Merging), the Subversion community recommends that you choose a repository location for each project root—the “topmost” directory that contains data related to that project—and then create three subdirectories beneath that root: trunk, meaning the directory under which the main project development occurs; branches, which is a directory in which to create various named branches of the main development line; and tags, which is a collection of tree snapshots that are created, and perhaps destroyed, but never changed. ^[28]

For example, your repository might look like this:

/
   calc/
      trunk/
      tags/
      branches/
   calendar/
      trunk/
      tags/
      branches/
   spreadsheet/
      trunk/
      tags/
      branches/
   …

Note that it doesn't matter where in your repository each project root is. If you have only one project per repository, the logical place to put each project root is at the root of that project's respective repository. If you have multiple projects, you might want to arrange them in groups inside the repository, perhaps putting projects with similar goals or shared code in the same subdirectory, or maybe just grouping them alphabetically. Such an arrangement might look like this:

/
   utils/
      calc/
         trunk/
         tags/
         branches/
      calendar/
         trunk/
         tags/
         branches/
      …
   office/
      spreadsheet/
         trunk/
         tags/
         branches/
      …

Lay out your repository in whatever way you see fit. Subversion does not expect or enforce a particular layout—in its eyes, a directory is a directory is a directory. Ultimately, you should choose the repository arrangement that meets the needs of the people who work on the projects that live there.

In the name of full disclosure, though, we'll mention another very common layout. In this layout, the trunk, tags, and branches directories live in the root directory of your repository, and your projects are in subdirectories beneath those, like so:

/
   trunk/
      calc/
      calendar/
      spreadsheet/
      …
   tags/
      calc/
      calendar/
      spreadsheet/
      …
   branches/
      calc/
      calendar/
      spreadsheet/
      …

There's nothing particularly incorrect about such a layout, but it may or may not seem as intuitive for your users. Especially in large, multiproject situations with many users, those users may tend to be familiar with only one or two of the projects in the repository. But the projects-as-branch-siblings approach tends to deemphasize project individuality and focus on the entire set of projects as a single entity. That's a social issue, though. We like our originally suggested arrangement for purely practical reasons—it's easier to ask about (or modify, or migrate elsewhere) the entire history of a single project when there's a single repository path that holds the entire history—past, present, tagged, and branched—for that project and that project alone.

Before creating your Subversion repository, an obvious question you'll need to answer is where the thing is going to live. This is strongly connected to myriad other questions involving how the repository will be accessed (via a Subversion server or directly), by whom (users behind your corporate firewall or the whole world out on the open Internet), what other services you'll be providing around Subversion (repository browsing interfaces, email-based commit notification, etc.), your data backup strategy, and so on.

We cover server choice and configuration in Chapter 6, Server Configuration, but the point we'd like to briefly make here is simply that the answers to some of these other questions might have implications that force your hand when deciding where your repository will live. For example, certain deployment scenarios might require accessing the repository via a remote filesystem from multiple computers, in which case (as you'll read in the next section) your choice of a repository backend data store turns out not to be a choice at all because only one of the available backends will work in this scenario.

Addressing each possible way to deploy Subversion is both impossible and outside the scope of this book. We simply encourage you to evaluate your options using these pages and other sources as your reference material and to plan ahead.

As of version 1.1, Subversion provides two options for the type of underlying data store—often referred to as “the backend” or, somewhat confusingly, “the (versioned) filesystem”—that each repository uses. One type of data store keeps everything in a Berkeley DB (or BDB) database environment; repositories that use this type are often referred to as being “BDB-backed.” The other type stores data in ordinary flat files, using a custom format. Subversion developers have adopted the habit of referring to this latter data storage mechanism as FSFS ^[29] —a versioned filesystem implementation that uses the native OS filesystem directly—rather than via a database library or some other abstraction layer—to store data.

Table 5.1, “Repository data store comparison” gives a comparative overview of Berkeley DB and FSFS repositories.

There are advantages and disadvantages to each of these two backend types. Neither of them is more “official” than the other, though the newer FSFS is the default data store as of Subversion 1.2. Both are reliable enough to trust with your versioned data. But as you can see in Table 5.1, “Repository data store comparison”, the FSFS backend provides quite a bit more flexibility in terms of its supported deployment scenarios. More flexibility means you have to work a little harder to find ways to deploy it incorrectly. Those reasons—plus the fact that not using Berkeley DB means there's one fewer component in the system—largely explain why today almost everyone uses the FSFS backend when creating new repositories.

Fortunately, most programs that access Subversion repositories are blissfully ignorant of which backend data store is in use. And you aren't even necessarily stuck with your first choice of a data store—in the event that you change your mind later, Subversion provides ways of migrating your repository's data into another repository that uses a different backend data store. We talk more about that later in this chapter.

The following subsections provide a more detailed look at the available backend data store types.

Subversion repository creation is an incredibly simple task. The svnadmin utility that comes with Subversion provides a subcommand (svnadmin create) for doing just that.

$ # Create a repository
$ svnadmin create /var/svn/repos
$

This creates a new repository in the directory /var/svn/repos, and with the default filesystem data store. Prior to Subversion 1.2, the default was to use Berkeley DB; the default is now FSFS. You can explicitly choose the filesystem type using the --fs-type argument, which accepts as a parameter either fsfs or bdb.

$ # Create an FSFS-backed repository
$ svnadmin create --fs-type fsfs /var/svn/repos
$

# Create a Berkeley-DB-backed repository
$ svnadmin create --fs-type bdb /var/svn/repos
$

After running this simple command, you have a Subversion repository.

Present in the db/ subdirectory of your repository is the implementation of the versioned filesystem. Your new repository's versioned filesystem begins life at revision 0, which is defined to consist of nothing but the top-level root (/) directory. Initially, revision 0 also has a single revision property, svn:date, set to the time at which the repository was created.

Now that you have a repository, it's time to customize it.

A hook is a program triggered by some repository event, such as the creation of a new revision or the modification of an unversioned property. Some hooks (the so-called “pre hooks”) run in advance of a repository operation and provide a means by which to both report what is about to happen and prevent it from happening at all. Other hooks (the “post hooks”) run after the completion of a repository event and are useful for performing tasks that examine—but don't modify—the repository. Each hook is handed enough information to tell what that event is (or was), the specific repository changes proposed (or completed), and the username of the person who triggered the event.

The hooks subdirectory is, by default, filled with templates for various repository hooks:

$ ls repos/hooks/
post-commit.tmpl          post-unlock.tmpl  pre-revprop-change.tmpl
post-lock.tmpl            pre-commit.tmpl   pre-unlock.tmpl
post-revprop-change.tmpl  pre-lock.tmpl     start-commit.tmpl
$

There is one template for each hook that the Subversion repository supports; by examining the contents of those template scripts, you can see what triggers each script to run and what data is passed to that script. Also present in many of these templates are examples of how one might use that script, in conjunction with other Subversion-supplied programs, to perform common useful tasks. To actually install a working hook, you need only place some executable program or script into the repos/hooks directory, which can be executed as the name (such as start-commit or post-commit) of the hook.

On Unix platforms, this means supplying a script or program (which could be a shell script, a Python program, a compiled C binary, or any number of other things) named exactly like the name of the hook. Of course, the template files are present for more than just informational purposes—the easiest way to install a hook on Unix platforms is to simply copy the appropriate template file to a new file that lacks the .tmpl extension, customize the hook's contents, and ensure that the script is executable. Windows, however, uses file extensions to determine whether a program is executable, so you would need to supply a program whose basename is the name of the hook and whose extension is one of the special extensions recognized by Windows for executable programs, such as .exe for programs and .bat for batch files.

Subversion executes hooks as the same user who owns the process that is accessing the Subversion repository. In most cases, the repository is being accessed via a Subversion server, so this user is the same user as whom the server runs on the system. The hooks themselves will need to be configured with OS-level permissions that allow that user to execute them. Also, this means that any programs or files (including the Subversion repository) accessed directly or indirectly by the hook will be accessed as the same user. In other words, be alert to potential permission-related problems that could prevent the hook from performing the tasks it is designed to perform.

There are serveral hooks implemented by the Subversion repository, and you can get details about each of them in the section called “Repository Hooks”. As a repository administrator, you'll need to decide which hooks you wish to implement (by way of providing an appropriately named and permissioned hook program), and how. When you make this decision, keep in mind the big picture of how your repository is deployed. For example, if you are using server configuration to determine which users are permitted to commit changes to your repository, you don't need to do this sort of access control via the hook system.

There is no shortage of Subversion hook programs and scripts that are freely available either from the Subversion community itself or elsewhere. These scripts cover a wide range of utility—basic access control, policy adherence checking, issue tracker integration, email- or syndication-based commit notification, and beyond. Or, if you wish to write your own, see Chapter 8, Embedding Subversion.

A Berkeley DB environment is an encapsulation of one or more databases, logfiles, region files, and configuration files. The Berkeley DB environment has its own set of default configuration values for things such as the number of database locks allowed to be taken out at any given time, the maximum size of the journaling logfiles, and so on. Subversion's filesystem logic additionally chooses default values for some of the Berkeley DB configuration options. However, sometimes your particular repository, with its unique collection of data and access patterns, might require a different set of configuration option values.

The producers of Berkeley DB understand that different applications and database environments have different requirements, so they have provided a mechanism for overriding at runtime many of the configuration values for the Berkeley DB environment. BDB checks for the presence of a file named DB_CONFIG in the environment directory (namely, the repository's db subdirectory), and parses the options found in that file. Subversion itself creates this file when it creates the rest of the repository. The file initially contains some default options, as well as pointers to the Berkeley DB online documentation so that you can read about what those options do. Of course, you are free to add any of the supported Berkeley DB options to your DB_CONFIG file. Just be aware that while Subversion never attempts to read or interpret the contents of the file and makes no direct use of the option settings in it, you'll want to avoid any configuration changes that may cause Berkeley DB to behave in a fashion that is at odds with what Subversion might expect. Also, changes made to DB_CONFIG won't take effect until you recover the database environment (using svnadmin recover).