We've already covered in detail how Subversion stores and retrieves various versions of files and directories in its repository. Whole chapters have been devoted to this most fundamental piece of functionality provided by the tool. And if the versioning support stopped there, Subversion would still be complete from a version control perspective. But it doesn't stop there.
In addition to versioning your directories and files, Subversion provides interfaces for adding, modifying, and removing versioned metadata on each of your versioned directories and files. We refer to this metadata as properties, and they can be thought of as two-column tables that map property names to arbitrary values attached to each item in your working copy. Generally speaking, the names and values of the properties can be whatever you want them to be, with the constraint that the names must be human-readable text. And the best part about these properties is that they, too, are versioned, just like the textual contents of your files. You can modify, commit, and revert property changes as easily as committing textual changes. And you receive other people's property changes as you update your working copy.
In this section, we will examine the utility—both to users of Subversion, and to Subversion itself—of property support. You'll learn about the property-related svn subcommands, and how property modifications affect your normal Subversion workflow. Hopefully, you'll be convinced that Subversion properties can enhance your version control experience.
Properties can be very useful additions to your working copy. In fact, Subversion itself uses properties to house special information, and as a way to denote that certain special processing might be needed. Likewise, you can use properties for your own purposes. Of course, anything you can do with properties you could also do using regular versioned files, but consider the following example of Subversion property use.
Say you wish to design a website that houses many digital photos, and displays them with captions and a datestamp. Now, your set of photos is constantly changing, so you'd like to have as much of this site automated as possible. These photos can be quite large, so as is common with sites of this nature, you want to provide smaller thumbnail images to your site visitors. You can do this with traditional files. That is, you can have your image123.jpg and an image123-thumbnail.jpg side-by-side in a directory. Or if you want to keep the filenames the same, you might have your thumbnails in a different directory, like thumbnails/image123.jpg. You can also store your captions and datestamps in a similar fashion, again separated from the original image file. Soon, your tree of files is a mess, and grows in multiples with each new photo added to the site.
Now consider the same setup using Subversion's file properties. Imagine having a single image file, image123.jpg, and then properties set on that file named caption, datestamp, and even thumbnail. Now your working copy directory looks much more manageable—in fact, it looks like there are nothing but image files in it. But your automation scripts know better. They know that they can use svn (or better yet, they can use the Subversion language bindings—see the section called “Using Languages Other than C and C++”) to dig out the extra information that your site needs to display without having to read an index file or play path manipulation games.
How (and if) you use Subversion properties is up to you. As we mentioned, Subversion has it own uses for properties, which we'll discuss a little later in this chapter. But first, let's discuss how to manipulate options using the svn program.
The svn command affords a few ways to add or modify file and directory properties. For properties with short, human-readable values, perhaps the simplest way to add a new property is to specify the property name and value on the command-line of the propset subcommand.
$ svn propset copyright '(c) 2003 Red-Bean Software' calc/button.c property 'copyright' set on 'calc/button.c' $
But we've been touting the flexibility that Subversion offers for your property values. And if you are planning to have a multi-line textual, or even binary, property value, you probably do not want to supply that value on the command-line. So the propset subcommand takes a --file (-F) option for specifying the name of a file which contains the new property value.
$ svn propset license -F /path/to/LICENSE calc/button.c property 'license' set on 'calc/button.c' $
In addition to the propset command, the svn program supplies the propedit command. This command uses the configured editor program (see the section called “Config”) to add or modify properties. When you run the command, svn invokes your editor program on a temporary file that contains the current value of the property (or which is empty, if you are adding a new property). Then, you just modify that value in your editor program until it represents the new value you wish to store for the property, save the temporary file, and then exit the editor program. If Subversion detects that you've actually changed the existing value of the property, it will accept that as the new property value. If you exit your editor without making any changes, no property modification will occur.
$ svn propedit copyright calc/button.c ### exit the editor without changes No changes to property 'copyright' on 'calc/button.c' $
We should note that, as with other svn subcommands, those related to properties can act on multiple paths at once. This enables you to modify properties on whole sets of files with a single command. For example, we could have done:
$ svn propset copyright '(c) 2002 Red-Bean Software' calc/* property 'copyright' set on 'calc/Makefile' property 'copyright' set on 'calc/button.c' property 'copyright' set on 'calc/integer.c' … $
All of this property adding and editing isn't really very useful if you can't easily get the stored property value. So the svn program supplies two subcommands for displaying the names and values of properties stored on files and directories. The svn proplist command will list the names of properties that exist on a path. Once you know the names of the properties on the node, you can request their values individually using svn propget. This command will, given a path (or set of paths) and a property name, print the value of the property to the standard output stream.
$ svn proplist calc/button.c Properties on 'calc/button.c': copyright license $ svn propget copyright calc/button.c (c) 2003 Red-Bean Software
There's even a variation of the proplist command that will list both the name and value of all of the properties. Simply supply the --verbose (-v) option.
$ svn proplist --verbose calc/button.c Properties on 'calc/button.c': copyright : (c) 2003 Red-Bean Software license : ================================================================ Copyright (c) 2003 Red-Bean Software. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions, and the recipe for Fitz's famous red-beans-and-rice. …
The last property-related subcommand is propdel. Since Subversion allows you to store properties with empty values, you can't remove a property altogether using propedit or propset. For example, this command will not yield the desired effect:
$ svn propset license '' calc/button.c property 'license' set on 'calc/button.c' $ svn proplist --verbose calc/button.c Properties on 'calc/button.c': copyright : (c) 2003 Red-Bean Software license : $
You need to use the propdel command to delete properties altogether. The syntax is similar to the other property commands:
$ svn propdel license calc/button.c property 'license' deleted from ''. $ svn proplist --verbose calc/button.c Properties on 'calc/button.c': copyright : (c) 2003 Red-Bean Software $
Now that you are familiar with all of the property-related svn subcommands, let's see how property modifications affect the usual Subversion workflow. As we mentioned earlier, file and directory properties are versioned, just like your file contents. As a result, Subversion provides the same opportunities for merging—in cleanly or conflicting fashions—someone else's modifications into your own.
And as with file contents, your property changes are local modifications, only made permanent when you commit them to the repository with svn commit. Your property changes can be easily unmade, too—the svn revert command will restore your files and directories to their un-edited states, contents, properties, and all. Also, you can receive interesting information about the state of your file and directory properties by using the svn status and svn diff commands.
$ svn status calc/button.c M calc/button.c $ svn diff calc/button.c Property changes on: calc/button.c ___________________________________________________________________ Name: copyright + (c) 2003 Red-Bean Software $
Notice how the status subcommand displays M in the second column instead of the first. That is because we have modified the properties on calc/button.c, but not modified its textual contents. Had we changed both, we would have seen M in the first column, too (see the section called “svn status”).
You might also have noticed the non-standard way that Subversion currently displays property differences. You can still run svn diff and redirect the output to create a usable patch file. The patch program will ignore property patches—as a rule, it ignores any noise it can't understand. This does unfortunately mean that to fully apply a patch generated by svn diff, any property modifications will need to be applied by hand.
As you can see, the presence of property modifications has no outstanding effect on the typical Subversion workflow. Your general patterns of updating your working copy, checking the status of your files and directories, reporting on the modifications you have made, and committing those modifications to the repository are completely immune to the presence or absence of properties. The svn program has some additional subcommands for actually making property changes, but that is the only noticeable asymmetry.
Subversion has no particular policy regarding properties—you can use them for any purpose. Subversion asks only that you not use property names that begin with the prefix svn:. That's the namespace that it sets aside for its own use. In fact, Subversion defines certain properties that have magical effects on the files and directories to which they are attached. In this section, we'll untangle the mystery, and describe how these special properties make your life just a little easier.
The svn:executable property is used to control a versioned file's filesystem-level execute permission bit in a semi-automated way. This property has no defined values—its mere presence indicates a desire that the execute permission bit be kept enabled by Subversion. Removing this property will restore full control of the execute bit back to the operating system.
On many operating systems, the ability to execute a file as a command is governed by the presence of an execute permission bit. This bit usually defaults to being disabled, and must be explicitly enabled by the user for each file that needs it. In a working copy, new files are being created all the time as new versions of existing files are received during an update. This means that you might enable the execute bit on a file, then update your working copy, and if that file was changed as part of the update, its execute bit might get disabled. So, Subversion provides the svn:executable property as a way to keep the execute bit enabled.
This property has no effect on filesystems that have no concept of an executable permission bit, such as FAT32 and NTFS. [28] Also, although it has no defined values, Subversion will force its value to * when setting this property. Finally, this property is valid only on files, not on directories.
The svn:mime-type property serves many purposes in Subversion. Besides being a general-purpose storage location for a file's Multipurpose Internet Mail Extensions (MIME) classification, the value of this property determines some behavioral characteristics of Subversion itself.
For example, if a file's svn:mime-type property is set to a non-text MIME type (generally, something that doesn't begin with text/, though there are exceptions), Subversion will assume that the file contains binary—that is, not human-readable—data. One of the benefits that Subversion typically provides is contextual, line-based merging of changes received from the server during an update into your working file. But for files believed to contain binary data, there is no concept of a “line”. So, for those files, Subversion does not attempt to perform contextual merges during updates. Instead, any time you have locally modified a binary working copy file that is also being updated, your file is renamed with a .orig extension, and then Subversion stores a new working copy file that contains the changes received during the update, but not your own local modifications, at the original filename. This behavior is really for the protection of the user against failed attempts at performing contextual merges on files that simply cannot be contextually merged.
Also, if the svn:mime-type property is set, then the Subversion Apache module will use its value to populate the Content-type: HTTP header when responding to GET requests. This gives a crucial clue about how to display a file when perusing your repository with a web browser.
The svn:ignore property contains a list of file patterns which certain Subversion operations will ignore. Perhaps the most commonly used special property, it works in conjunction with the global-ignores run-time configuration option (see the section called “Config”) to filter unversioned files and directories out of commands like svn status, svn add, and svn import.
The rationale behind the svn:ignore property is easily explained. Subversion does not assume that every file or subdirectory in a working copy directory is intended for version control. Resources must be explicitly placed under Subversion's management using the svn add or svn import commands. As a result, there are often many resources in a working copy that are not versioned.
Now, the svn status command displays as part of its output every unversioned file or subdirectory in a working copy that is not already filtered out by the global-ignores option (or its built-in default value). This is done so that users can see if perhaps they've forgotten to add a resource to version control.
But Subversion cannot possibly guess the names of every resource that should be ignored. Also, quite often there are things that should be ignored in every working copy of a particular repository. To force every user of that repository to add patterns for those resources to their run-time configuration areas would be not just a burden, but has the potential to clash with the configuration needs of other working copies that the user has checked out.
The solution is to store ignore patterns that are unique to the resources likely to appear in a given directory with the directory itself. Common examples of unversioned resources that are basically unique to a directory, yet likely to appear there, include output from program compilations. Or—to use an example more appropriate to this book—the HTML, PDF, or PostScript files generated as the result of a conversion of some source DocBook XML files to a more legible output format.
For this purpose, the svn:ignore property is the solution. Its value is a multi-line collection of file patterns, one pattern per line. The property is set on the directory in which you wish the patterns to be applied. [29] For example, say you have the following output from svn status:
$ svn status calc M calc/button.c ? calc/calculator ? calc/data.c ? calc/debug_log ? calc/debug_log.1 ? calc/debug_log.2.gz ? calc/debug_log.3.gz
In this example, you have made some property modifications to button.c, but in your working copy you also have some unversioned files: the latest calculator program that you've compiled from your source code, a source file named data.c, and a set of debugging output log files. Now, you know that your build system always results in the calculator program being generated. [30] And you know that your test suite always leaves those debugging log files lying around. These facts are true for all working copies, not just your own. And you know that you aren't interested in seeing those things every time you run svn status. So you use svn propedit svn:ignore calc to add some ignore patterns to the calc directory. For example, you might add this as the new value of the svn:ignore property:
calculator debug_log*
After you've added this property, you will now have a local property modification on the calc directory. But notice what else is different about your svn status output:
$ svn status M calc M calc/button.c ? calc/data.c
Now, all the cruft is missing from the output! Of course, those files are still in your working copy. Subversion is simply not reminding you that they are present and unversioned. And now with all the trivial noise removed from the display, you are left with more interesting items—such as that source code file that you probably forgot to add to version control.
If you want to see the ignored files, you can pass the --no-ignore option to subversion:
$ svn status --no-ignore M calc/button.c I calc/calculator ? calc/data.c I calc/debug_log I calc/debug_log.1 I calc/debug_log.2.gz I calc/debug_log.3.gz
The list of patterns to ignore is also used by svn add and svn import. Both of these operations involve asking Subversion to begin managing some set of files and directories. Rather than force the user to pick and choose which files in a tree she wishes to start versioning, Subversion uses the ignore patterns to determine which files should not be swept into the version control system as part of a larger recursive addition or import operation.
Subversion has the ability to substitute keywords—pieces of useful, dynamic information about a versioned file—into the contents of the file itself. Keywords generally describe information about the last time the file was known to be modified. Because this information changes each time the file changes, and more importantly, just after the file changes, it is a hassle for any process except the version control system to keep the data completely up-to-date. Left to human authors, the information would inevitably grow stale.
For example, say you have a document in which you would like to display the last date on which it was modified. You could burden every author of that document to, just before committing their changes, also tweak the part of the document that describes when it was last changed. But sooner or later, someone would forget to do that. Instead simply ask Subversion to perform keyword substitution on the LastChangedDate keyword. You control where the keyword is inserted into your document by placing a keyword anchor at the desired location in the file. This anchor is just a string of text formatted as $KeywordName$.
Subversion defines the list of keywords available for substitution. That list contains the following five keywords, some of which have shorter aliases that you can also use:
This keyword describes the last time the file was known to have been changed in the repository, and looks something like $LastChangedDate: 2002-07-22 21:42:37 -0700 (Mon, 22 Jul 2002) $. It may be abbreviated as Date.
This keyword describes the last known revision in which this file changed in the repository, and looks something like $LastChangedRevision: 144 $. It may be abbreviated as Revision or Rev.
This keyword describes the last known user to change this file in the repository, and looks something like $LastChangedBy: harry $. It may be abbreviated as Author.
This keyword describes the full URL to the latest version of the file in the repository, and looks something like $HeadURL: http://svn.collab.net/repos/trunk/README $. It may be abbreviated as URL.
This keyword is a compressed combination of the other keywords. Its substitution looks something like $Id: calc.c 148 2002-07-28 21:30:43Z sally $, and is interpreted to mean that the file calc.c was last changed in revision 148 on the evening of July 28, 2002 by the user sally.
Simply adding keyword anchor text to your file does nothing special. Subversion will never attempt to perform textual substitutions on your file contents unless explicitly asked to do so. After all, you might be writing a document [31] about how to use keywords, and you don't want Subversion to substitute your beautiful examples of un-substituted keyword anchors!
To tell Subversion whether or not to substitute keywords on a particular file, we again turn to the property-related subcommands. The svn:keywords property, when set on a versioned file, controls which keywords will be substituted on that file. The value is a space-delimited list of the keyword names or aliases found in the previous table.
For example, say you have a versioned file named weather.txt that looks like this:
Here is the latest report from the front lines. $LastChangedDate$ $Rev$ Cumulus clouds are appearing more frequently as summer approaches.
With no svn:keywords property set on that file, Subversion will do nothing special. Now, let's enable substitution of the LastChangedDate keyword.
$ svn propset svn:keywords "LastChangedDate Author" weather.txt property 'svn:keywords' set on 'weather.txt' $
Now you have made a local property modification on the weather.txt file. You will see no changes to the file's contents (unless you made some of your own prior to setting the property). Notice that the file contained a keyword anchor for the Rev keyword, yet we did not include that keyword in the property value we set. Subversion will happily ignore requests to substitute keywords that are not present in the file, and will not substitute keywords that are not present in the svn:keywords property value.
Immediately after you commit this property change, Subversion will update your working file with the new substitute text. Instead of seeing your keyword anchor $LastChangedDate$, you'll see its substituted result. That result also contains the name of the keyword, and continues to be bounded by the dollar sign ($) characters. And as we predicted, the Rev keyword was not substituted because we didn't ask for it to be.
Here is the latest report from the front lines. $LastChangedDate: 2002-07-22 21:42:37 -0700 (Mon, 22 Jul 2002) $ $Rev$ Cumulus clouds are appearing more frequently as summer approaches.
If someone else now commits a change to weather.txt, your copy of that file will continue to display the same substituted keyword value as before—until you update your working copy. At that time the keywords in your weather.txt file will be re-substituted with information that reflects the most recent known commit to that file.
Unless otherwise noted using a versioned file's svn:mime-type property, Subversion assumes the file contains human-readable data. Generally speaking, Subversion only uses this knowledge to determine if contextual difference reports for that file are possible. Otherwise, to Subversion, bytes are bytes.
This means that by default, Subversion doesn't pay any attention to the type of end-of-line (EOL) markers used in your files. Unfortunately, different operating system use different tokens to represent the end of a line of text in a file. For example, the usual line ending token used by software on the Windows platform is a pair of ASCII control characters—carriage return (CR) and line feed (LF). Unix software, however, just uses the LF character to denote the end of a line.
Not all of the various tools on these operating systems are prepared to understand files that contain line endings in a format that differs from the native line ending style of the operating system on which they are running. Common results are that Unix programs treat the CR character present in Windows files as a regular character (usually rendered as ^M), and that Windows programs combine all of the lines of a Unix file into one giant line because no carriage return-linefeed (or CRLF) character combination was found to denote the end of line.
This sensitivity to foreign EOL markers can become frustrating for folks who share a file across different operating systems. For example, consider a source code file, and developers that edit this file on both Windows and Unix systems. If all the developers always use tools which preserve the line ending style of the file, no problems occur.
But in practice, many common tools either fail to properly read a file with foreign EOL markers, or they convert the file's line endings to the native style when the file is saved. If the former is true for a developer, he has to use an external conversion utility (such as dos2unix or its companion, unix2dos) to prepare the file for editing. The latter case requires no extra preparation. But both cases result in a file that differs from the original quite literally on every line! Prior to committing his changes, the user has two choices. Either he can use a conversion utility to restore the modified file to the same line ending style that it was in before his edits were made. Or, he can simply commit the file—new EOL markers and all.
The result of scenarios like these include wasted time and unnecessary modifications to committed files. Wasted time is painful enough. But when commits change every line in a file, this complicates the job of determining which of those lines were changed in a non-trivial way. Where was that bug really fixed? On what line was a syntax error introduced?
The solution to this problem is the svn:eol-style property. When this property is set to a valid value, Subversion uses it to determine what special processing to perform on the file so that the file's line ending style isn't flip-flopping with every commit that comes from a different operating system. The valid values are:
This causes the file to contain the EOL markers that are native to the operating system on which Subversion was run. In other words, if a user on a Windows machine checks out a working copy that contains a file with an svn:eol-style property set to native, that file will contain CRLF EOL markers. A Unix user checking out a working copy which contains the same file will see LF EOL markers in his copy of the file.
Note that Subversion will actually store the file in the repository using normalized LF EOL markers regardless of the operating system. This is basically transparent to the user, though.
This causes the file to contain CRLF sequences for EOL markers, regardless of the operating system in use.
This causes the file to contain LF characters for EOL markers, regardless of the operating system in use.
This causes the file to contain CR characters for EOL markers, regardless of the operating system in use. This line ending style is not very common. It was used on older Macintosh platforms (on which Subversion doesn't even run).
The svn:externals property contains instructions for Subversion to populate a versioned directory with one or more other checked-out Subversion working copies. For more information on this keyword and its use, see the section called “Externals Definitions”.
Properties are a powerful feature of Subversion, acting as key components of many Subversion features discussed elsewhere in this and other chapters—textual diff and merge support, keyword substitution, newline translation, etc. But to get the full benefit of properties, they must be set on the right files and directories. Unfortunately, that can be a step easily forgotten in the routine of things, especially since failing to set a property doesn't usually result in an obvious error condition (at least compared to, say, failing to add a file to version control). To help your properties get applied to the places that need them, Subversion provides a couple of simple but useful features.
Whenever you introduce a file to version control using the svn add or svn import commands, Subversion runs a very basic heuristic to determine if that file consists of human-readable or non-human-readable content. If the latter is the decision made, Subversion will automatically set the svn:mime-type property on that file to application/octet-stream (the generic “this is a collection of bytes” MIME type). Of course, if Subversion guesses incorrectly, or if you wish to set the svn:mime-type property to something more precise—perhaps image/png or application/x-shockwave-flash—you can always remove or edit that property.
Subversion also provides the auto-props feature, which allows you to create mappings of filename patterns to property names and values. These mappings are made in your runtime configuration area. They again affect adds and imports, and not only can override any default MIME type decision made by Subversion during those operations, they can also set additional Subversion or custom properties, too. For example, you might create a mapping that says that any time you add JPEG files—ones that match the pattern *.jpg—Subversion should automatically set the svn:mime-type property on those files to image/jpeg. Or perhaps any files that match *.cpp should have svn:eol-style set to native, and svn:keywords set to Id. Auto-prop support is perhaps the handiest property related tool in the Subversion toolbox. See the section called “Config” for more about configuring that support.
[27] Fixing spelling errors, grammatical gotchas, and “just-plain-wrongness” in commit log messages is perhaps the most common use-case for the --revprop option.
[28] The Windows filesystems use file extensions (such as .EXE, .BAT, and .COM) to denote executable files.
[29] The patterns are strictly for that directory—they do not carry recursively into subdirectories.
[30] Isn't that the whole point of a build system?
[31] … or maybe even a section of a book …