Today is the last day developer.com accepts nominations for its "Product of the year 2008" award. Let’s all go to the nomination page and enter Subversion in these categories:

Development Tool of the Year

Open Source Tool of the Year

It takes less than two minutes to fill out the form.

Subversion well-deserves an award in these areas. If a lot of us nominate Subversion, then it will be part of the vote that starts November 15th (I'll remind you here).

A lot of people asked us about TortoiseSVN and its plans for merge tracking. For those of you who do not know, TortoiseSVN is a Subversion client for Windows users. It plugs into the Windows Shell and makes Subversion client commands available through right-click options within Windows Explorer and any of the open/save dialogues. TortoiseSVN is a great tool that rightfully won a Sourceforge.net Community Choice Award 2007. You can download it from openCollabNet and it is also part of CollabNet’s stack of supported integrations for Subversion.

I use TortoiseSVN myself all the time so I too was curious to find out about the TortoiseSVN plans for merge tracking. Some information is on their web site but I contacted the project lead for TortoiseSVN, Stefan Küng, and asked about the status. Here is his reply and some screenshots (thanks Stefan):

Merge tracking is usually done by the Subversion library without the user knowing about it. I.e., the user doesn't see that Subversion skips already merged revisions when merging from trunk to a branch. That's why there's not really much done in TSVN UI-wise. Where we thought we have to show the user more information, we did. Otherwise we tried to hide the specifics as well as possible.

What we did for now:

The log dialog, when started from the merge dialog to select revisions to merge shows the already merged revisions in grey instead of black. This indicates to the user that the grey revisions shouldn't be merged (again).

The log dialog shows merged revisions indented, with each indentation level being a merge level. This UI may change before the final 1.5 release because I'm not really happy with the way the indentation looks right now - it's hard to see the different levels. I may add some vertical lines there or do something else to improve the indentation.

When showing blames, merged lines are shown with an italic font, and the user can optionally show the merged paths of every line.

Nightly builds are on the TortoiseSVN web site but they might not always work with the Subversion 1.5 pre-release binaries currently downloadable from the Merge Tracking Early Adopter Program. Next week we will update the Subversion 1.5 binaries to the latest development status and we’ll post a TortoiseSVN build that works with our latest Subversion build.

One more thing: On October 30th CollabNet will do another webinar on Subversion 1.5, covering advanced branching and merging with Subversion 1.5. The webinar follows up on the one from a month ago about branching strategies and answers many of the questions people asked. Register here.

Yesterday at the Pre-SubConf Subversion Workshop in Munich, I presented a short bit about a new Subversion 1.5 feature — WebDAV write-thru proxy support. This feature allows for a Subversion server deployment based around Apache 2.2.x and mod_proxy in which there is a single master server and associated repository, and one or more slave servers which handle read operations while passing through (proxying) write operations to that single master server. In this deployment scenario, each slave server has its own copy of the master repository which is kept in sync by some process, typically one driven by hook scripts on the master server itself.

Why might someone want to use such an arrangement? Well, users are far more likely to be doing read operations (such as checkouts, updates, status checks, log history requests, diff calculations, etc.) than write operations (such as commit, revision property changes, and lock/unlock requests). You might wish to employ several servers to share the burden of handling those read requests (a load-balancing scenario). Or perhaps you have a worldwide organization (such as CollabNet's) with offices in, say, the United States, Eastern Europe, and India. If you deploy a single centralized server across the whole organization, you will almost certainly wind up favoring some users over others in terms of the performance of the network links between them and the Subversion server. WebDAV write-thru proxies would allow you to keep geographically local slave servers for each of your global regions, universally optimizing the performance of all of your users' read requests.

For the purposes of my presentation, I whipped up a simple WebDAV write-thru proxy scenario with a single slave server, and using svnsync as the mechanism for propogating changes from the master server to the slave server. The following describes how you can do the same.

Two of the questions most asked during the webinar “Branching and Merging with Subversion 1.5” were:

• “What are the differences between Subversion 1.5's merge tracking features and svnmerge.py?”
• “Will there be a migration utility?”

I’m in Munich at the moment and today attended CollabNet’s Subversion Power Workshop that precedes SubConf, the first conference dedicated to Subversion. Answers to both questions were covered during the merge tracking sessions.

Before talking about the advantages of Subversion 1.5 merge tracking over svnmerge.py, I think it is important to say that svnmerge.py has been a very useful tool contributed and maintained by Subversion community members. We are all thankful for their contributions. That said, Subversion 1.5 does offer several advantages over svnmerge.py:

• SVN 1.5 tracks merge history at a per-path level of granularity and tracks file level merges. svnmerge.py only tracks merge info for the entire branch, it does not track finer levels (e.g.: files). This allows SVN to implement cherry picking, which svnmerge.py does not support.
• svnmerge.py doesn’t always properly merge merge-properties (like svnmerge-integrated and svnmerge-blocked), leading to property conflicts when merging across multiple branches. SVN 1.5 solves this.
• SVN 1.5 includes support for third-party tools, such as call-backs for conflict resolution. If a merge conflict arises, Subversion can call your GUI client and the client can then fire up a conflict resolution tool. This improves workflow and usability. You can see the integration between SVN 1.5 merging and GUI clients in action by trying CollabNet’s new GUI Merge Client.
• SVN’s merge tracking includes more comprehensive auditing, like querying the merge history to answer questions about what was merged when and where.

A utility to migrate svnmerge.py merge tracking history from a previous version to Subversion 1.5 was committed to the Subversion trunk just a few days ago. What can it do?

• Convert the latest merge history for multiple branches. In terms of history, information about the latest merges for each of your branches is the most important information because it allows for an easier merge the next time you merge. If you want to keep things simple and only migrate the latest merge history, you can.
• The migration utility can also scan your entire repo for merge history, or accept a series of path prefixes to look at: svnmerge-migrate-history.py /path/to/repos trunk tasks branches.

We’ve uploaded a copy of the script to the download section of the Merge Tracking Early Adopter Program. You can download it and test it out on your own repository (not for production please, Subversion 1.5 is still under development). It should work well but tell us if you run into any problems. We know of one limitation: don’t run it on a Subversion 1.5 repository that already has merge history, this has not been tested yet (then again: why would you want to do that?). Right now, only the .py script is available but the Subversion developers plan to release a Windows executable later.

One of the most common questions that occurs whenever a new release of Subversion (or any product) comes out, has to do with the considerations you need to take into account before upgrading.  I will try to cover some of those in this post.

Subversion's compatibility guidelines guarantee that any 1.x client can talk to any 1.x server.  So you do not have to upgrade the client and server at the same time.  The 1.5 client can talk to 1.0-1.4 servers and any 1.0-1.4 clients can talk to a 1.5 server.

Those same guidelines say that you never have to reload a repository when upgrading the server.  You can upgrade your server from 1.0-1.4 to 1.5 and you do not need to do anything to your repositories.  The new server will silently make any updates that are needed, when they are needed.  In the case of 1.5, a new SQLite database file will be added to the repository to store indexes that speed up the new merge tracking function.  Even though you do not have to reload your repository, there are often small efficiencies you can gain by doing a dump/load of your repository.  In the case of this release, there is a new storage mechanism for fsfs repositories.  Instead of storing all of the revisions in a single folder, we now "shard" the repository by creating a sub-directory for each 1000 revisions. This does not improve performance of Subversion, but it might improve performance of things like backup utilities or other tools you use to work with the file system on the server.  In this case, you do not have to dump/load to get this feature, there is a Python script you can run that will do the sharding for you.  This script will run relatively quickly, certainly orders of magnitude faster than a dump/load.

Some features of Subversion 1.5 will require that you have a 1.5 server and a 1.5 client to enjoy the full benefits of the feature.  For example, the merge tracking feature requires that both the client and server are running 1.5.  If one of them is running a lower version than you get the merge functionality that existed in that version of Subversion.  If you want to take advantage of merge tracking you really need to update your server and all of the clients that are doing merges.  Doing a merge with an old client will work, but will not set any of the merge tracking information when the change is committed.  This simply means that someone using a 1.5 client later will attempt to merge the same info again and get conflicts.  The svn merge --record-only command can be used to fix this after the fact.

When using the new sparse-checkouts functionality it really helps to have a 1.5 server.  The feature will work with an older server, but in most cases the server will still send the extra information (file contents) to the client and the client has to discard what it does not want.

There is an important client-side compatibility feature to take into account.  Some of the new features of 1.5 such as sparse-checkouts and changelists necessitate a new client working copy format number.  So this means as soon as you perform an operation that needs to re-write your working copy (checkout/update/switch/commit/propset etc.) then the internal format of the working copy will be upgraded to the new format for Subversion 1.5.  Once this happens, you will no longer be able to use a 1.4 client with that same working copy.  The same sort of change occurred with the 1.3 to 1.4 upgrade process.  We know that this really impacted many users that were stuck using an older client, so we are including a Python script that you can run to downgrade a working copy back to the 1.4 format.  We will distribute this as an EXE for Windows users.  Just to make this perfectly clear, you can have a 1.4 client and 1.5 client on the same system.  You just need to be careful to not mix their usage on the same working copy.  Also, pure-read operations like (status/info/ls) do not update the working copy.  So simply viewing a working copy with a new version of TortoiseSVN or SCPlugin installed does not modify the working copy format.  Conversely, however, an old version of one of these tools will not be able to read a 1.5 working copy.

Hopefully this answers the questions about upgrading to Subversion 1.5.  It is going to be a great release and I certainly hope people choose to upgrade when it comes out.  If you have additional questions that I did not cover, please post them in the comments or on the Subversion Users forum on openCollabNet.

Last week we did a webinar titled “Branching and Merging with Subversion 1.5”. This webinar focused on branching strategies, rather than the new merge tracking capabilities of Subversion 1.5. With merging becoming so much easier, many companies will want to rethink their branching strategies to better leverage the advantages of parallel development. That’s why we devoted an entire webinar to branching. If you missed this event, you can view the replay.

We received over 270 questions during the webinar. Over the next few weeks we will answer a lot of them here in this blog and we will also follow up with a webinar that elevates the story to change management. But let me start today by trying to answer the most frequently asked question (you know FAQs hardly ever really are ‘Frequently Asked Questions’, but this one is): “When will Subversion 1.5 be available?”

Should the answer come from CollabNet? Subversion is an open source project. One open source model can be quite different from another. Take MySQL, it is open source but what goes into the code is largely decided by one company, MySQL AB, which also does most of the development. The Linux kernel is a different example, there are many contributors but what makes it into the kernel goes through very few people.

CollabNet initiated Subversion and set it up as a community-driven project ruled by meritocracy (the better your contributions, the more influence you have). We like to think that contributions from our team of Subversion developers is very significant, and they often take the lead on new features (like svnsync and merge tracking), but there are many others who take initiatives and make significant contributions. The Subversion developers work as a community to make decisions around Subversion and that includes when to release a new version. CollabNet participates in this but the community rules. If you look at Subversion’s history, this process has worked very well: Subversion releases have always shown high-quality.

So, it’s not CollabNet’s role to pre-announce a new release, but of course we are very close to the community and know what is going on. Really then, when do I expect SVN 1.5?

The developers are hard at work to prep the Subversion 1.5 code for a release candidate and you can expect that candidate later this month. The release candidate is then “soaked” for at least a month, meaning that Subversion users are invited to test the new version and report bugs. If bugs are found, they are fixed and the soak cycle starts again (although this time it might be shorter).

In the end, the Subversion developers have a vote about the readiness of the release and if the verdict is “ready!”, the developers post a tarball of the code on Tigris.org and we have GA (general availability). Nobody can predict when this will exactly happen, but neither can the weatherperson tell you that it will for sure be sunny tomorrow. Here is my forecast: late December, with a chance of hail that will punch a hole in the schedule.

You can help keep the soak cycle short: start testing Subversion now! In previous posts I mentioned the Merge Tracking Early Adopter Program where you can find Subversion 1.5 pre-release binaries, a sample repository with merge history built-in, CollabNet’s new GUI clients, and a forum to discuss Subversion 1.5 and report bugs. Help the community bring the release date forward. Testing now prevents that issues are uncovered late, and it is your opportunity to give something back to the community that created the new standard in version control.

http://merge-tracking.open.collab.net/