At CollabNet we have been discussing the differences and similarities between the Open Source community, and the Enterprise community. We want to be sensitive to that fine line between information awareness and marketing, without falling to one side or the other. Some people at CollabNet are hesitant to point to anything that costs money to the Open Source community for fear of offending, yet much of the content could indeed be very useful in shortening that learning curve.

As someone who is also helping to maintain the CollabNet site, and as Community Manager, I needed to learn what I needed to know as quickly as possible. I am not only learning to use Subversion, but I’m also learning to use the collaborative tools provided by TeamForge.You can see the tools of CEE if you look at any project on openCollabNet. We will be upgrading the site to TeamForge in the future.

Yes, we eat our own dog food, so to speak, and I’m glad. I have been impressed with the functionality and the ease of use of this platform compared to others I have used. To shorten my learning curve I have found some great training through articles, white papers, webinars, and web courses.

It is my hope that the resources I list below help to shorten your learning curve, no matter whether you want to stick with only the free materials or the ones that we charge for.

For those of you who are Subversion experts, you can stop reading here, though I value your opinion and experience and would appreciate any comments or suggestions you’d like to make. Also, some of you may not be aware of the variety collaboration tools that TeamForge provides.

Newsletters

Many bypass this option when registering for CollabNet, but the newsletter can point to useful information about learning Subversion, as well as other content you might not want to miss. You can subscribe by clicking on your user name at the top of the CollabNet site. Once in your profile, scroll down and click the Newsletter box. Or you can read the archived newsletters.

Webinars

Webinars are a great way of seeing software in action, and hearing information about the tools. Be sure to be logged into the site so you don't need to fill in any forms.

• What's New in Subversion 1.6
• Subversion Best Practices
• CollabNet TeamForge The Power of Centralization

Discussion Forums

Discussion forums are an excellent place to ask questions that fellow developers and users can answer. Be sure to return the favor by answering the questions when you can provide solutions based on your own experience. You'll need to be logged in to participate.

• Subversion for Admins
• Subversion End Users & Developers

Release Notes, Data Sheets, and Articles

We also have some good notes, data sheets, and articles. Be sure you are logged in so you needn't fill out contact information.

• Subversion 1.6
• Browsing a Subversion or CVS Repository
• CollabNet Subversion Datasheet
• CollabNet Training for Subversion

Training Courses

CollabNet offers a complete, role-based training curriculum across the entire CollabNet product line. Courses are delivered in a number of formats to best meet your education needs.

Subversion 1.5 - Individual Modules:

• Subversion 1.5 for Developers - Section 01 - Introduction to Version Control (10 min) - $28
• Subversion 1.5 for Developers - Section 02 - Introducing Subversion (19 min) - $28
• Subversion 1.5 for Developers - Section 03 - Global Revisioning and Working Copies (21 min) - $38
• Subversion 1.5 for Developers - Section 04 - Standard Work Cycle (33 min) - $38
• See More . . .

Instructor Led Courses

• Subversion 1.6 for Developers - Standard
• Subversion 1.6 for Developers - Enterprise
• Subversion 1.6 for the 1.x Developer
• Subversion 1.6 for Administrators
• See More . . .

FAQs

FAQs are probably one of the best free resources you can read for learning about Subversion.

• Subversion Client FAQ
• Subversion Server FAQ
• Subversion Migration FAQ

As I discover more resources for learning Subversion and TeamForge, I will write new blogs, and add to the newsletter as I discover.

Enjoy your Subversion learning path!

This article explains in-depth how to build a Subversion server on OS X.  While building a Subversion server is the purpose of this article, a lot of the information you learn is operating system agnostic. So, whether you're building a Linux-based Subversion server or a Windows-based Subversion server, a lot of the same rules and procedures will apply. That being said, let's get started.

# Load Subversion Apache Modules
LoadModule dav_svn_module /opt/subversion/lib/svn-apache/mod_dav_svn.so

# Work around authz and SVNListParentPath issue
RedirectMatch ^(/repos/svn)$ $1/

# Enable Subversion logging
CustomLog logs/svn_logfile "%t %u %{SVN-ACTION}e" env=SVN-ACTION

<Location /repos/svn/>
 # Enable Subversion
 DAV svn

 # Directory containing all repository for this path
 SVNParentPath /opt/repos/svn

 # List repositories collection
 SVNListParentPath On
</Location>

Subversion 1.5 introduced that idea of sharding for FSFS-backed repositories. For every commit to a FSFS repository, Subversion creates a single file which describes all the changes in that revision. Prior to 1.5, all of those files were stored in a single directory, which had several drawbacks: incremental backups took a long time, the repository could not be dymnically grown across different filesystems, and some filesystems have degraded performance when the number of directory entries grows too large. With sharding, these revision files were split into separate subdirectories, eliminating a large number of these problems.

Even with sharding, the filesystem still has some inefficiencies. For instance, due to the block size of the underlying filesystem, having many files can still lead to wasted space on disk, especially with many small commits. Subversion can open and read data from many revisions over the course of an operation, and using a large number of files means that Subversion can not exploit various operating system-level caches. Backing up and restoring a repository, although quicker, can still take a long time because of the large number of files spread across the repository.

One of the great ideas that came out of the 2008 Subversion Developers' Summit was the notion that FSFS filesystems could be packed, that is, all the files in a completed shard could be glued together to create a single monster revision file.  This pack file would save space on disk, give the operating system a chance to do some caching, and generally improve the snappiness of the system.

In order to use FSFS packing, you simply need to ensure that target repository has been upgraded to the latest format, and then pack the repository using svnadmin.  Note that repositories do not automatically pack themselves, so for heavily used repositories, you may want to install a cron job or post commit hook to do the packing.  Users can continue to use the repository while it is being packed:

$ svnadmin upgrade repo
Repository lock acquired.
Please wait; upgrading the repository may take some time...

Upgrade completed.
$ svnadmin pack repo
Packing shard 0...done.
Packing shard 1...done.
Packing shard 2...done.
Packing shard 3...done.
Packing shard 4...done.
...
Packing shard 36...done.
$

To give an idea of the potential space savings, on my local 1.5-era copy of Subversion's own repository I get the following results:

$ du -sh svnrepo-1.5/
659M	svnrepo-1.5/
$ 

While on a packed 1.6 copy of the same repository, with rep-sharing enabled, I see the following:

$ du -sh svnrepo-1.6/
593M	svnrepo-1.6/
$ 

That's more than a 10% decrease in space, at no cost in performance. These space savings will vary depending upon your own repository and use habits, but we're excited about the improvements in the FSFS backend in Subversion 1.6.

My previous blog entry discussing Subversion, Apache and LDAP is nearing two years old. It was written when Apache 2.0.x was still the mainstream and when Apache 2.2.x was released, changes in the LDAP modules and their respective configuration directives has left my previous entry very confusing for those wanting to use Apache 2.2.x. The purpose of the Definitive Guide is to provide a single location for questions for Apache 2.0.x and 2.2.x, while also providing more depth about things to consider when building your Apache-based Subversion server using LDAP for authentication.

The Configuration

For those of you that just want to get to the point, where you can copy and paste and move on, here you go:

Example Apache 2.2.x Configuration Snippet

# Load Apache LDAP modules
LoadModule ldap_module modules/mod_ldap.so
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so

# Load Subversion Apache Modules
LoadModule dav_svn_module     modules/mod_dav_svn.so # Use full path to SUBVERSION_HOME/bin/mod_dav_svn.so on Windows
LoadModule authz_svn_module   modules/mod_authz_svn.so # Use full path to SUBVERSION_HOME/bin/mod_authz_svn.so on Windows

# Work around authz and SVNListParentPath issue
RedirectMatch ^(/repos)$ $1/

# Enable Subversion logging
CustomLog logs/svn_logfile "%t %u %{SVN-ACTION}e" env=SVN-ACTION

<Location /repos/>
  # Enable Subversion
  DAV svn

  # Directory containing all repository for this path
  SVNParentPath /subversion/svn-repos

  # List repositories colleciton
  SVNListParentPath On

  # Enable WebDAV automatic versioning
  SVNAutoversioning On

  # Repository Display Name
  SVNReposName "Your Subversion Repository"

  # Do basic password authentication in the clear
  AuthType Basic

  # The name of the protected area or "realm"
  AuthName "Your Subversion Repository"

  # Make LDAP the authentication mechanism
  AuthBasicProvider ldap

  # Make LDAP authentication is final
  AuthzLDAPAuthoritative on

  # Active Directory requires an authenticating DN to access records
  AuthLDAPBindDN "CN=ldapuser,CN=Users,DC=your,DC=domain"

  # This is the password for the AuthLDAPBindDN user in Active Directory
  AuthLDAPBindPassword ldappassword

  # The LDAP query URL
  AuthLDAPURL "ldap://your.domain:389/DC=your,DC=domain?sAMAccountName?sub?(objectClass=*)"

  # Require a valid user
  Require valid-user

  # Authorization file
  AuthzSVNAccessFile /subversion/apache2/auth/repos.acl
</Location>

Example Apache 2.0.x Configuration Snippet

# Load Apache LDAP modules
LoadModule ldap_module modules/mod_ldap.so
LoadModule auth_ldap_module modules/mod_auth_ldap.so

# Load Subversion Apache Modules
LoadModule dav_svn_module     modules/mod_dav_svn.so # Use full path to SUBVERSION_HOME/bin/mod_dav_svn.so on Windows
LoadModule authz_svn_module   modules/mod_authz_svn.so # Use full path to SUBVERSION_HOME/bin/mod_authz_svn.so on Windows

# Work around authz and SVNListParentPath issue
RedirectMatch ^(/repos)$ $1/

# Enable Subversion logging
CustomLog logs/svn_logfile "%t %u %{SVN-ACTION}e" env=SVN-ACTION

<Location /repos/>
  # Enable Subversion
  DAV svn

  # Directory containing all repository for this path
  SVNParentPath /subversion/svn-repos

  # List repositories colleciton
  SVNListParentPath On

  # Enable WebDAV automatic versioning
  SVNAutoversioning On

  # Repository Display Name
  SVNReposName "Your Subversion Repository"

  # LDAP Authentication is final
  AuthLDAPAuthoritative on

  # Do basic password authentication in the clear
  AuthType Basic

  # The name of the protected area or "realm"
  AuthName "Your Subversion Repository"

  # Active Directory requires an authenticating DN to access records
  AuthLDAPBindDN "CN=ldapuser,CN=Users,DC=your,DC=domain"

  # This is the password for the AuthLDAPBindDN user in Active Directory
  AuthLDAPBindPassword ldappassword

  # The LDAP query URL
  AuthLDAPURL "ldap://your.domain:389/DC=your,DC=domain?sAMAccountName?sub?(objectClass=*)"

  # Require authentication
  Require valid-user

  # Authorization file
  AuthzSVNAccessFile /subversion/apache2/auth/repos.acl
</Location>

(The configurations above were for pointing to an Active Directory (AD) server.

Understanding the Configuration

So...the above Apache configurations are what I personally use when building an Apache-based server. Obviously there are changes that need to be made depending on the environment in but for now, it's a great start. To make the best of this opportunity, let's talk about the miscellaneous parts of the configuration.

SVNListParentPath and Subversion's authz

One of the first problems people run into when building an Apache-based Subversion server is when they want to have mod_dav_svn serve a list of repositories. Everything works fine until they enable Subversion's authorization (authz) support. What happens is the server will be configured properly and secured properly but when you go to the repository collection list, which in our case is http://localhost/repos, you are forbidden to view the collection even if you have access. Well, with the RedirectMatch closer to the top of the configuration, you fix this issue. How you might be asking and the reason is that when you enable authz, you must have a trailing slash at the end of the collection url. With the RedirectMatch, we automatically redirect urls to the collection listing when there is no trailing slash. Problem solved.

Custom Subversion Logging

Subversion uses Apache's WebDAV support for providing access to its repositories when using Apache. Unfortunately, when you look at Apache's access logs to try and see your Subversion usage, you end up with a lot of WebDAV communication being logged and you only see a portion of the actual client/server communication. This is because mod_dav_svn uses Apache subrequests and Apache does not log subrequests. Even if it did, turning the Subversion communication in the Apache access log into something meaningful would be nearly impossible. That being said, the configuration above has been setup to use one of Subversion's features: Apache Logging which takes the guess work out.

Subversion Configuration

The other Subversion-specific parts of the Apache configuration are pretty self-explanitory. To summarize what is enabled with the above:

• SVNListParentPath: Enables the ability to browse the location root and get a list of repositories being served by that url base
• SVNAutoversioning: Enables the use of WebDAV clients to make changes to the repository contents without using a Subversion client
• SVNParentPath: Enables serving N number of repositories for the url base
• SVNReposName: Enables you to put in your own text to be visible in the web browser when browsing your repository contents via the built-in repository browser provided by mod_dav_svn
• AuthzSVNAccessFile: Tells Subversion's mod_authz_svn module where to find the authz file.

For more details about the Subversion-specific Apache directives, and a list of even more ways you can configure your Apache-based Subversion server, view the mod_dav_svn and the mod_authz_svn documentation.

LDAP Configuration

The LDAP portion of the Apache configuration is where most people run into problems. That being said, we'll spend a little more time explaining the Apache LDAP configuration. The most important thing to note is the subtle differences between Apache 2.0.x and Apache 2.2.x:

Apache 2.0.x           | Apache 2.2.x
-----------------------------------------------
AuthLDAPAuthoritative  | AuthzLDAPAuthoritative
AuthLDAPBindDN         | AuthLDAPBindDN
AuthLDAPBindPassword   | AuthLDAPBindPassword
AuthLDAPURL            | AuthLDAPURL
                       | AuthBasicProvider

You should note that the Apache LDAP module names have also changed between Apache 2.0.x and 2.2.x. Now that we see the naming changes, let's talk about how to properly use these Apache directives to get the LDAP-based authentication you're looking for. (I will be using the Apache 2.2.x names for the Apache directives. If you're still using Apache 2.0.x, please refer to the table above for how to take my documentation and apply it to Apache 2.0.x.)

• AuthzLDAPAuthoritative: Tells Apache whether or not a failed authentication request can be passed to other Apache modules
• AuthLDAPBindDN: The distinguished name of the user account that Apache will use to connect to the directory system to perform its user authentication
• AuthLDAPBindPassword: The password for the user account configured via the AuthLDAPBindDN directive
• AuthLDAPURL: This is a url that tells where the directory server is, where to look for users at, what user attribute is used to identify a user and other miscellaneous things specific to the LDAP query syntax (More on this later.)
• AuthBasicProvider: This tells Apache which authentication module you want to use for Basic authentication

All of the directives above are pretty straight forward except for the AuthLDAPURL directive. This directive we will discuss in more detail below. For any other Apache configuration questions, please resort to the Apache Documentation for your respective Apache version.

The LDAP Query URL

For most, the AuthLDAPURL directive is the most challenging to understand. There is good reason for this. That one directive actually consists of 6+ pieces of information that will be different for each Subversion server. Let's break our example AuthLDAPURL into its pieces and discuss the importance, and nuances, of each.

For simplicity, here is the url again, in its entirety: ldap://your.domain:389/DC=your,DC=domain?sAMAccountName?sub?(objectClass=*)

• Url scheme: [ldap] This is nothing more than a url scheme. It will usually be either 'ldap' or 'ldaps' in the event that you're using SSL for accessing your directory server.
• Hostname: [your.domain] This is the ip address or hostname of your directory server.
• Port: [389] This is the port the server is listening on for directory server communication.
• Search Base: [DC=your,DC=domain] This is the distinguished name to the path in the directory tree that you want to search for users.
• Username attribute: [sAMAccountName] This is the attribute contains the login name being used.
• Query scope: [sub] This tells the directory server what type of query to perform.
• Filter: [(objectClass=*)] This tells the directory server to filter the query for objects matching a particular filter

For more details on constructing an ldap url, which is a standard and not specific to Apache, view RFC 2255.

Working with Active Directory

Active Directory is known as a Multi-Master Directory System. This being said, each directory server in AD does not always have all the necessary information to perform all directory server requests. The best way to handle this is to have Apache query a Global Catalog. A Global Catalog server has the ability to search at the whole forest for users. This means if you want to do domain-wide searches or larger, you need to point to a Global Catalog and you need to update your Apache configuration accordingly. When using a Global Catalog, you should be using port 3268 when performing your queries.

Searching for Users

In the example url above, the sAMAccountName attribute is used to identify the username. This attribute is Windows/Active Directory specific so for those of you using OpenLDAP or another option, that attribute probably will not exist. Change your attribute accordingly. An example is if you wanted to use the Common Name to login, you could specify "CN" as the attribute.

LDAP Query Tuning

The last thing we will talk about is the ability to use filters to make your LDAP query a little more specific. In the example url above we used "(objectClass=*)", which will search for all objects. If you know that you only want to search for a particular object type, like the "user" type, you could use "(objectClass=user)" instead.

Conclusion

Building an Apache-based Subversion server with LDAP as the authentication mechanism can be daunting for some. I hope this has made things easier for you.

The Subversion project released the first publicly available release candidate for the 1.6.0 release on Friday Feb. 20.  You can download the source for this release candidate from the Subversion project on tigris.org.  The release notes for the 1.6 release are still being assembled but you can follow the latest state of the document from the project website.

As we did with the Subversion 1.5.0 release, CollabNet is providing binary packages of this release candidate to make it as easy as possible for the community to evaluate the release and provide their feedback.  You can download binaries for Windows and Linux right now.  We will be adding Solaris and OSX binaries soon.

Follow this blog for more entries on specific features in Subversion 1.6.  There are several new features as you can see in the release notes.  The biggest new feature is arguably the support for tree conflicts.  This will need a separate post of its own to explain, perhaps more than one.  For those that cannot wait, I can only point to the folder with the notes the developers were keeping on the feature.  Not all of those files represent what finally went into the feature so keep that in mind and keep an eye out for more posts.

The Subversion project has issued a beta1 release on the path to the final GA. The next release will likely be rc1 but, as always, that depends on the feedback we get from testers.

CollabNet has created binaries for Linux, Windows and Solaris (with OS X coming soon) from this beta1.  A noteworthy change as we get closer to RC1 is that we are now providing our CollabNet Subversion packaging for all three platforms. Solaris and Windows are new with this beta release. In addition, for all three of these platforms the server installation can automatically install and configure ViewVC as part of the server installation.

You can download these binaries from openCollabNet. There are also updated versions of the CollabNet Merge Client, Subclipse and TortoiseSVN available on this site.

We have a forum where you can post questions, comments or problems that you encounter using this version. If after some discussion a report is indeed deemed a defect, we forward the bug to the Subversion community using their normal processes. Feedback on the GUI merge clients will go to CollabNet Engineering.

For those that missed the announcement of the Alpha2 release, here are a few things to be aware of:

• Subversion 1.5 client has an updated working copy format. Once an SVN 1.5 client updates a working copy it will be converted to the new format. This means that older SVN clients will no longer be able to read these working copies. So either be careful, or be sure to update all your clients to the same version. There is a Python script that you can use to downgrade your working copy to the 1.4 format.
• Subversion 1.5 server can be used to serve an existing repository, but you will not be able to use the new merge tracking features with this repository until it is upgraded. You have a few options here:
1. Dump existing repository, then create a new repository using the 1.5 binaries and load the dump file.
2. Create a new repository using 1.5 binaries and use svnsync 1.5 binary to transfer the contents to the new repository.
3. Run a new command svnadmin upgrade to upgrade an existing repository. This runs fast as it just updates the internal repository format. There are some downsides to this approach though:
• SVN 1.5 repositories maintain a new index called the node-origins index  This is needed to speed up certain merge operations. The index will be lazy-populated on upgraded repositories, but can be updated on demand by running a new tool named svn-populate-node-origins-index(.exe). If you have a large repository you will likely want to populate the index. New repositories, or ones that are dumped/loaded or svnsync'ed will not need this.
• For fsfs repositories, the node-origins index will be slightly faster and take up less disk space if the repository is reloaded or synced. For existing repositories, the index has to be maintained in a separate set of files that takes up extra disk space and I/O.
• For new 1.5 fsfs repositories, there is a new sharding mechanism for storing revisions on disk. Instead of storing all revisions in one single directory, they are now spread across multiple directories which will be better on certain filesystems.  There is a Python script you can run to shard an exising fsfs repository though.
4. If you were using an earlier version of the 1.5 binaries, note that the on-disk format of the repository has been changed in the most recent builds. You should dump your repository with your current binaries, then install the new binaries, create a new repository and load the repository.

Please take the opportunity to give these beta versions on the path to GA a try. Your feedback will lead to a better final release and likely also help us deliver Subversion 1.5 as soon as possible.

The Subversion project has issued an alpha2 release on the path to the final GA. The next release might be alpha3, it might be beta1, that depends on the feedback we get from testers. One thing is certain, and that is the sooner we can find any remaining bugs, the sooner the GA release will be available.  Likewise, hearing that you have used this version with some success will also help us make decisions.

CollabNet has created binaries for Linux and Windows (with OS X coming soon) from this alpha2. You can download them from openCollabNet. There are also updated versions of the CollabNet Merge Client, Subclipse and TortoiseSVN available on this site.

We have a forum where you can post questions, comments or problems that you encounter using this version. If after some discussion a report is indeed deemed a defect, we forward the bug to the Subversion community using their normal processes. Feedback on the GUI merge clients will go to CollabNet Engineering.

A few things to be aware of:

• Subversion 1.5 client has an updated working copy format.  Once an SVN 1.5 client updates a working copy it will be converted to the new format. This means that older SVN clients will no longer be able to read these working copies. So either be careful, or be sure to update all your clients to the same version. There is a Python script that you can use to downgrade your working copy to the 1.4 format.
• Subversion 1.5 server can be used to serve an existing repository, but you will not be able to use the new merge tracking features with this repository until it is upgraded. You have a few options here:
1. Dump existing repository, then create a new repository using the 1.5 binaries and load the dump file.
2. Create a new repository using 1.5 binaries and use svnsync 1.5 binary to transfer the contents to the new repository.
3. Run a new command svnadmin upgrade to upgrade an existing repository. This runs fast as it just updates the internal repository format. There are some downsides to this approach though:
• SVN 1.5 repositories maintain a new index called the node-origins index  This is needed to speed up certain merge operations. The index will be lazy-populated on upgraded repositories, but can be updated on demand by running a new tool named svn-populate-node-origins-index(.exe). If you have a large repository you will likely want to populate the index. New repositories, or ones that are dumped/loaded or svnsync'ed will not need this.
• For fsfs repositories, the node-origins index will be slightly faster and take up less disk space if the repository is reloaded or synced. For existing repositories, the index has to be maintained in a separate set of files that takes up extra disk space and I/O.
• For new 1.5 fsfs repositories, there is a new sharding mechanism for storing revisions on disk. Instead of storing all revisions in one single directory, they are now spread across multiple directories which will be better on certain filesystems.  There is a Python script you can run to shard an exising fsfs repository though.
4. If you were using an earlier version of the 1.5 binaries, note that the on-disk format of the repository has been changed in the most recent builds. You should dump your repository with your current binaries, then install the new binaries, create a new repository and load the repository.

Please take the opportunity to give these alpha versions on the path to GA a try. Your feedback will lead to a better final release and likely also help us deliver Subversion 1.5 as soon as possible.

Yesterday at the Pre-SubConf Subversion Workshop in Munich, I presented a short bit about a new Subversion 1.5 feature — WebDAV write-thru proxy support. This feature allows for a Subversion server deployment based around Apache 2.2.x and mod_proxy in which there is a single master server and associated repository, and one or more slave servers which handle read operations while passing through (proxying) write operations to that single master server. In this deployment scenario, each slave server has its own copy of the master repository which is kept in sync by some process, typically one driven by hook scripts on the master server itself.

Why might someone want to use such an arrangement? Well, users are far more likely to be doing read operations (such as checkouts, updates, status checks, log history requests, diff calculations, etc.) than write operations (such as commit, revision property changes, and lock/unlock requests). You might wish to employ several servers to share the burden of handling those read requests (a load-balancing scenario). Or perhaps you have a worldwide organization (such as CollabNet's) with offices in, say, the United States, Eastern Europe, and India. If you deploy a single centralized server across the whole organization, you will almost certainly wind up favoring some users over others in terms of the performance of the network links between them and the Subversion server. WebDAV write-thru proxies would allow you to keep geographically local slave servers for each of your global regions, universally optimizing the performance of all of your users' read requests.

For the purposes of my presentation, I whipped up a simple WebDAV write-thru proxy scenario with a single slave server, and using svnsync as the mechanism for propogating changes from the master server to the slave server. The following describes how you can do the same.

Two of the questions most asked during the webinar “Branching and Merging with Subversion 1.5” were:

• “What are the differences between Subversion 1.5's merge tracking features and svnmerge.py?”
• “Will there be a migration utility?”

I’m in Munich at the moment and today attended CollabNet’s Subversion Power Workshop that precedes SubConf, the first conference dedicated to Subversion. Answers to both questions were covered during the merge tracking sessions.

Before talking about the advantages of Subversion 1.5 merge tracking over svnmerge.py, I think it is important to say that svnmerge.py has been a very useful tool contributed and maintained by Subversion community members. We are all thankful for their contributions. That said, Subversion 1.5 does offer several advantages over svnmerge.py:

• SVN 1.5 tracks merge history at a per-path level of granularity and tracks file level merges. svnmerge.py only tracks merge info for the entire branch, it does not track finer levels (e.g.: files). This allows SVN to implement cherry picking, which svnmerge.py does not support.
• svnmerge.py doesn’t always properly merge merge-properties (like svnmerge-integrated and svnmerge-blocked), leading to property conflicts when merging across multiple branches. SVN 1.5 solves this.
• SVN 1.5 includes support for third-party tools, such as call-backs for conflict resolution. If a merge conflict arises, Subversion can call your GUI client and the client can then fire up a conflict resolution tool. This improves workflow and usability. You can see the integration between SVN 1.5 merging and GUI clients in action by trying CollabNet’s new GUI Merge Client.
• SVN’s merge tracking includes more comprehensive auditing, like querying the merge history to answer questions about what was merged when and where.

A utility to migrate svnmerge.py merge tracking history from a previous version to Subversion 1.5 was committed to the Subversion trunk just a few days ago. What can it do?

• Convert the latest merge history for multiple branches. In terms of history, information about the latest merges for each of your branches is the most important information because it allows for an easier merge the next time you merge. If you want to keep things simple and only migrate the latest merge history, you can.
• The migration utility can also scan your entire repo for merge history, or accept a series of path prefixes to look at: svnmerge-migrate-history.py /path/to/repos trunk tasks branches.

We’ve uploaded a copy of the script to the download section of the Merge Tracking Early Adopter Program. You can download it and test it out on your own repository (not for production please, Subversion 1.5 is still under development). It should work well but tell us if you run into any problems. We know of one limitation: don’t run it on a Subversion 1.5 repository that already has merge history, this has not been tested yet (then again: why would you want to do that?). Right now, only the .py script is available but the Subversion developers plan to release a Windows executable later.

One of the most common questions that occurs whenever a new release of Subversion (or any product) comes out, has to do with the considerations you need to take into account before upgrading.  I will try to cover some of those in this post.

Subversion's compatibility guidelines guarantee that any 1.x client can talk to any 1.x server.  So you do not have to upgrade the client and server at the same time.  The 1.5 client can talk to 1.0-1.4 servers and any 1.0-1.4 clients can talk to a 1.5 server.

Those same guidelines say that you never have to reload a repository when upgrading the server.  You can upgrade your server from 1.0-1.4 to 1.5 and you do not need to do anything to your repositories.  The new server will silently make any updates that are needed, when they are needed.  In the case of 1.5, a new SQLite database file will be added to the repository to store indexes that speed up the new merge tracking function.  Even though you do not have to reload your repository, there are often small efficiencies you can gain by doing a dump/load of your repository.  In the case of this release, there is a new storage mechanism for fsfs repositories.  Instead of storing all of the revisions in a single folder, we now "shard" the repository by creating a sub-directory for each 1000 revisions. This does not improve performance of Subversion, but it might improve performance of things like backup utilities or other tools you use to work with the file system on the server.  In this case, you do not have to dump/load to get this feature, there is a Python script you can run that will do the sharding for you.  This script will run relatively quickly, certainly orders of magnitude faster than a dump/load.

Some features of Subversion 1.5 will require that you have a 1.5 server and a 1.5 client to enjoy the full benefits of the feature.  For example, the merge tracking feature requires that both the client and server are running 1.5.  If one of them is running a lower version than you get the merge functionality that existed in that version of Subversion.  If you want to take advantage of merge tracking you really need to update your server and all of the clients that are doing merges.  Doing a merge with an old client will work, but will not set any of the merge tracking information when the change is committed.  This simply means that someone using a 1.5 client later will attempt to merge the same info again and get conflicts.  The svn merge --record-only command can be used to fix this after the fact.

When using the new sparse-checkouts functionality it really helps to have a 1.5 server.  The feature will work with an older server, but in most cases the server will still send the extra information (file contents) to the client and the client has to discard what it does not want.

There is an important client-side compatibility feature to take into account.  Some of the new features of 1.5 such as sparse-checkouts and changelists necessitate a new client working copy format number.  So this means as soon as you perform an operation that needs to re-write your working copy (checkout/update/switch/commit/propset etc.) then the internal format of the working copy will be upgraded to the new format for Subversion 1.5.  Once this happens, you will no longer be able to use a 1.4 client with that same working copy.  The same sort of change occurred with the 1.3 to 1.4 upgrade process.  We know that this really impacted many users that were stuck using an older client, so we are including a Python script that you can run to downgrade a working copy back to the 1.4 format.  We will distribute this as an EXE for Windows users.  Just to make this perfectly clear, you can have a 1.4 client and 1.5 client on the same system.  You just need to be careful to not mix their usage on the same working copy.  Also, pure-read operations like (status/info/ls) do not update the working copy.  So simply viewing a working copy with a new version of TortoiseSVN or SCPlugin installed does not modify the working copy format.  Conversely, however, an old version of one of these tools will not be able to read a 1.5 working copy.

Hopefully this answers the questions about upgrading to Subversion 1.5.  It is going to be a great release and I certainly hope people choose to upgrade when it comes out.  If you have additional questions that I did not cover, please post them in the comments or on the Subversion Users forum on openCollabNet.