Subversion 1.5, which is expected this summer, will include much anticipated merge tracking functionality. Auke has already blogged about this:

• Introducing Merge Tracking
• Merge Tracking in 1.5

CollabNet wants to support this important development with a Merge Tracking Early Adopter Program. The goals of this program are to assist the Subversion committers with active testing, to collect enhancement requests and to give you early access to the feature so you can plan for the future.

The home of the program is at http://merge-tracking.open.collab.net. Here you can find links to documents related to Merge Tracking (such as the design specs), beta release binaries, a test repository with merge tracking history built-in and a forum to discuss Subversion 1.5 and merge tracking in general.

We are looking for many people to join and be very active, only then can the program contribute to the development efforts. If you choose to join us, please be willing to commit to downloading and installing the software, spending significant time testing and reviewing the merge tracking feature and actively participating in the discussion forum (specially for bug reporting).

This project serves to support the Subversion open source project and we will tie into their normal processes. Potential defects will be discussed in the discussion forum and, if they are deemed real, we will forward them to the Subversion developers by logging them in Subversion's issue database or sending an email to dev@.  Also, all discussions will be publicly visible. We are working closely with the committers and have let them know about the Early Adopter Program a few weeks ago. Some of the committers will be active in the project.

If merge tracking is important to you, please join the Early Adopter Program and together with other openCollabNet members help the Subversion development team make the merge tracking feature as robust as possible.

http://merge-tracking.open.collab.net

The Subversion project is very active in publishing regular releases but does not officially endorse or maintain any binary packages of the Subversion software. Instead, you can download binaries from volunteers, including CollabNet which provides Subversion binaries for Windows, Red Hat Linux and Solaris. The advantage of this volunteer system is that Subversion is available for many platforms but the disadvantage is that for certain platforms the availability of Subversion binaries has been spotty or has not stayed in sync with Subversion releases.

On June 1st we will start a new community project on openCollabNet: SVNbinaries. The goal of this project is to create binaries for a wide range of platforms and ensure continuity through the support of a community rather than individuals. Next to creating Subversion binaries we will work on improving build scripts and we'll ensure that bindings for many languages are available for download.

We know that many people already make binaries available and it is not at all our goal to “compete” with them; being a good citizen of the global Subversion community is very important to us. The project will not be the one place for the download of Subversion binaries, we’re happy to link to other places where you can find them. The goal is to create a community that improves build scripts and ensures that a wide variety of binaries and bindings is available on a continuous basis. In fact, people who already make binaries available are warmly invited to participate in this project so others can join them to improve their build scripts, ensure bindings are available and make derivatives for different dependencies.

So, what binaries and bindings are we specifically looking for? The community should drive this. People will step forward with their work while others will ask for something. If a need is common across the Subversion community, we will find people to fill it.  CollabNet will seeded the project though; Jeremy will contribute the Mac OS X client binary he recently created.

The community is not starting with a lot of rules around participation; rules will be created over time by the community itself. We’re starting with a simple process modeled after common practices in the open source world: you contribute something, other people evaluate it and if it is deemed good and you commit to active participation then you can become a full or partial committer in the project.

If you would like to contribute your binaries and build scripts or help improve other people’s work, send us an email. You can find the project at http://svnbinaries.open.collab.net. It will go live "officially" on June 1st.
 

Jeremy Whitlock, Mark Phippard and Guido Haarmans

I must admit that I'm new to blogging and these lines have been written on plain old paper first - yes, I was a COBOL developer in the early days of my professional career and I still write a lot of things in upper case.

When I made the move to a small start-up named Atria Software back in 1996, the world of software development looked quiet different: small to mid-sized teams, typically co-located around a specially designated project server. Today's broadband connections are faster than most intranet lines then, and different sites are connected through GB-capacity leased lines? Back then, if you were lucky you had a doubled ISDN dial-up connection.

Back in these days companies started to ramp-up distributed teams to work on highly complex projects, and of course they needed an infrastructure that supported this. The answer of most commercial SCM systems was Replication and Synchronization of the repositories. You duplicate the archive, figure out the delta at the different sites from time to time, you exchange these deltas and apply it to the different replicas - done! The biggest advantage of this approach was that it worked with the thin connections. The biggest disadvantage: conflicts due to the time gap between change and sync. If the same object is changed at 2 sites, a potential conflict has to be resolved during the synchronization. You might argue that this still can be solved by mastership concepts to avoid that the same object is changed simultaneously. You are right, but this implies that you are introducing branches not because the project needs them, but just because of the replication.

Modern approaches try to solve this challenge like Raid systems with write-through technology. If you commit a change to an archive, the change is propagated immediately to the other replicas, without any mastership definition. This little time gap eliminates most of the risk - almost like a central archive!

But - just "almost". First, if the connection between the replicas is down the risk for conflict rises again. Second, it reduces the flexibility of organizations to react to change, because the setup of the infrastructure reflects the status quo (if you've ever moved replicas between different sites you know what I mean …). Third, it makes IP (Intellectual Property) Governance more complicated. I know of a ClearCase VOB that was replicated across 28(!) sites, most of them with read-only access. Imagine the cost to establish the same level of security to avoid unauthorized access at each site?

A common argument against a centralized approach in SCM is what to do if you (or your site) are offline and you need data that is not in your working copy right now. Well, in today’s world there is no longer the need for being offline (you can even go online in an airplane these days); the technology is there and it is way cheaper than bothering a large developer team with a highly complex branching strategy. And if you want to isolate work for a while before sharing it with the other project team members, Subversion (or an embedded Subversion) allows you to do that as well.

Don't get me wrong, there are still some scenarios for replication, like an extreme data volume. Then, and only then, it is the right approach. But don't let it be your silver bullet to meet the challenge of distributed development and do it because you always did it!

I must admit I have my fair share in seeding the concept of replication into organizations. But - replication as an SCM concept to support distributed teams is more than 15 years old; in our market space this is almost 2 generations of technology. It is time to try something new, although it is not that new at all because the mainframers have done it for least for 5 generations ...

This is posted by an excited NKOTB (New Kid on the Blog) coming from the pre-internet generation.

I thought I would take a break from my previous technical posts and give a general update about what has been going on in the land of CollabNet Subversion (our Subversion binaries).

We have been busy the last several weeks and the fruits of our labor are starting to see the light of day. First, a couple of weeks ago we released our Subversion 1.4.3 client and server RPM's for Red Hat Enterprise Linux. This was big news for us because it was our first 1.4.3 release and it had been bugging us that we were lagging behind. Producing the binaries is easy, but we created new packaging as well as additional scripts that we provide with the RPM. This took some extra time but now our build environment is complete and going forward it will be easier for us to stay in synch with the open source community.

Earlier this week, we released our Subversion 1.4.3 client and server packages for Solaris 10 SPARC. This was our first Solaris release and again it felt good to get it out the door. We spent some extra time making sure the build works the way we want it to but we sorted out any issues and the release sailed through our QA process. I know that I was pleased when I tested the Solaris version, installing and getting a server up and running is very easy. The extra time we spent paid off and like Linux we can now produce new releases fast.

Also this week, we helped kick-off a new area on openCollabNet that is providing "Community Binaries" for Subversion. The first binary release is for Mac OS X 10.4 PPC and Intel. Jeremy Whitlock, a contributor to this blog, deserves the credit for picking up the ball on this and running with it. He did a great job in both producing the binaries and in putting a lot of quality touches into the packaging. We will have more information on our plans and aspirations for the binaries project when we get the project launched on openCollabNet (within the next two weeks or so). For now, you OS X users, go download the binaries and let us know how it goes for you.

Finally, there is the Windows installer for CollabNet Subversion. Our current version is 1.4.2 and uses InstallAnywhere, a Java-based installer that adds a lot of heft to the download size because it includes a JRE. We are moving away from that to a Windows-specific installer. We are also adding a lot of new features to the installer such as installing the server as a Windows service as part of the install process. It is taking some time to get these features implemented correctly and QA-ed but in the end it will be worth it. We should have the final version out in a couple of weeks. Most likely it will be for Subversion 1.4.4 as that should be released between now and then and creating the binaries themselves is not the issue for us.

Note added on June 11th: we now have Subversion 1.4.4 binaries for Mac OS X, read more in Jeremy's blog post.

You love Subversion and you love your Mac. But where do you get your command line client? I had that problem too and I'm very happy to announce the availability of a new Subversion 1.4.3 binary for the Mac OS X operating system. You can download it right here from openCollabNet. Within the next few weeks it will become part of a community project that will have as its goal to provide up-to-date Subversion binaries for many operating systems.  We’ll blog about it when that project starts but let’s learn about the Mac OS X binary first.

The new OS X binary is a complete Subversion 1.4.3 command line client installation. Using the Mac graphical installer, you can install the binary on any 10.4.x OS X operating system. The binary is "universal" so it runs on both the Intel and PPC based systems. The binary contains the Subversion executables and all necessary/dependency headers and shared libraries. This means that you can compile other applications against the included libraries, for instance to compile Apache to throw a network layer onto the newly installed Subversion binary. The package also includes the JavaHL bindings for those of you using Subversion natively from a Java application, like the popular Subclipse plug-in for Eclipse. 

While this Subversion binary is fully functional, this is only the beginning. With a community project surrounding the efforts required to build this binary, we can eventually include more bindings, better packaging and possibly a Subversion server for Mac OS X. The sky is the limit and our community will be the steward of what is to come. Expect more information about the community project in the near future.

As I mentioned in my previous post, the Subversion project has identified many requirements based on input from both the open source community and from the enterprise data collected by CollabNet from its customers. The requirements as well as the functional and design specifications are available on the Subversion project site.

Subversion's upcoming 1.5 release is scheduled to deliver the core merge tracking feature with subsequent releases likely extending this. Fundamental to all merge tracking related functionality is the functionality to automatically record what change sets have been copied, or merged, where. On top of this, the 1.5 release will deliver the following functionality:

• Repeated Merge

One of the most frequent definitions of what merge tracking means is the concept of dealing with repeated merges. The requirement is to track which change sets have been applied where, so that users can repeatedly merge branch A to branch B without having to remember the last range of revisions copied. This would also track change set cherry picking by users, so that Subversion won’t accidentally re-merge change sets that are already applied.

• Cherry Picking

Cherry picking is the merging of one or more individual changes from branch A into branch B (as opposed to the merge of all previously unmerged changes). Subversion will provide a way to indicate that the change(s) have been merged into branch B. Such merging must be supported in multiple different directions, for instance if a release branch B is created by copying the trunk, both forward port changes made on branch B into the trunk and backport changes made on the trunk into branch B without confusing the merge tracking algorithm.

• Record Manual Merge

Recording manual merges allows change sets to be marked as merged without having used the ‘svn merge’ sub command to create the defined target revision. This could be to mark a situation when there is no merge tool for the file type and a user merged the changes manually. Also, it should support merge tracking of a change set which is sufficiently different when ported to a different branch that the use of ‘svn merge’ is no longer appropriate.

Example scenarios requiring this functionality include (a) the actual change you want to apply to the branch has no overlap with its incarnation on the source branch but is conceptually equivalent, (b) only a subset of a change set warrants application, and (c) the branch content has drifted far enough apart to make automatic merging impossible (for instance, because it would generate excessive merge conflicts).

• Rollback Merge

The ability to undo a merge and/or the associated tracking meta data, regardless whether the merge is the result of an automated or manual merge.

• Block/Unblock Change Set

Subversion will be able to protect revisions from accidentally being propagated elsewhere.

• Automated Merge

Subversion will have the ability to automate merges and support interfaces for resolving conflicts and other errors.

Any branching scheme requires the consideration of merging. Up until now, Subversion facilitates merges but does not record them as such in its history information. That is, Subversion does not automatically record, or track, what change sets have been merged where and what revisions were created as a result of merging.  To work around the lack of merge tracking, current best practice is to properly document merges in commit log messages.

Merge tracking is a multi-faceted set of functionality and no single definition exists that is commonly agreed upon. Most users and organizations tend to think of only the subset of features that addresses particular issues they face in their development organizations. Examples of features that are commonly part of merge tracking are to automatically record what change sets exist on what branches, prevent duplicate merges, handle bi-directional merging, support for manual recording of tracking information, block certain change sets from ever being propagated elsewhere, and various reporting capabilities.

Little over a year ago, the Subversion project started to define, design, and implement merge tracking. CollabNet contributed by gathering enterprise requirements for the open source project. In February 2006, we organized a workshop that brought select customers together with the Subversion development team to make an inventory of required functionality.  The Subversion team has taken these requirements and driven the definition, design, and implementation. CollabNet employees continue to take a key role in developing the feature and addressing open issues related to merge tracking.

Subversion’s Merge Tracking requirements, functional specification, design documents, and the current status of the project are available on the Subversion project's merge tracking pages.

In my next post, I will provide an overview of the functionality that the Subversion team intends to deliver in their upcoming 1.5 release.

The Subversion open source project has earned a reputation for paying close attention to its processes, and doing its best to find that "sweet spot" where as much information as possible is harvested from its contributors without frustrating them with overbearing rules and regulations. One area where this plays out is in the composition of log messages attached to commits and would-be commits (patches). The project has some pretty picky guidelines about the content and format of its log messages. These guidelines (which can be found at http://subversion.tigris.org/hacking.html#log-messages) are among the lengthiest bits of Subversion process documentation to be found. They require, among other things, documentation of the change made at the granularity of a single function (or symbol).

Why? Why not just use a short one-liner log message that points folks to an issue tracker artifact or an archived email thread where the full history of the background of the change is more likely to be found? Is this just an old habit from the Subversion community's CVS days that didn't die, or are there benefits?

These questions were asked of me recently, and I try here to answer them.

Recently, our founder and CTO Brian Behlendorf decided to step back a bit. He is still around but turned over the CTO cubicle to me. Quite coincidentally, but pretty much at the same time, CollabNet purchased SourceForge Enterprise Edition from VA Software, bringing the world's two premier commercial web-hosted software collaboration environments under a single roof. If all that doesn't call for a little blogging, I don't quite know what does!

So, what shall we talk about? The past? Next steps? New beginnings? How about "all of the above"?

By this point in time, I think no one can doubt that the open source software development community has accomplished some amazing things, things in some cases that the enterprise community has attempted repeatedly and failed at. Such singular achievements come to mind as sendmail and BIND, the rolling collective stewardship of the UNIX family of operating systems, the famous "LAMP stack," or any number of other dramatic achievements. But at the same time, there's no question but that the open source process has some liabilities that are really worrisome to the enterprise -- you know the list: imprecise schedules, false-start and abandoned projects, and a tendency to ignore less "sexy" details like quality, localization, and customer support. The best open-source projects can equal or better the best enterprises in all these matters, but then there are all those distressing not-so-best projects.

There's some kind of magic in the open source community process, but it needs some taming. CollabNet was founded (thanks, Brian!) to capture the magic of the open source process, civilize it just a bit, and bring it to the enterprise. They (before my time) provided a set of tools proven in the open-source arena, but didn't hesitate to help the community produce better replacements where necessary, such as by sponsorship of Subversion development to replace CVS: the magic's not in the tools, but in the processes that co-evolved with them. CollabNet added features like rich security and authorization systems, that are crucial to enterprise use but uninteresting (if not down-right antithetical) to the open-source community. Brian also brought with him, from experiences in the Apache community and elsewhere, both wisdom and contacts at the heart of the communitarian open-source magic.

Since I joined the company in 2002, we have added world-class hosting and support services, scalability, enterprise-focused features like project management and reporting, and have opened the product to partner and customer integrations through APIs. By this time, as well, surely no one has missed the significance of SourceForge. Through its public face sourceforge.net (as well as its several other properties), VA Software has served and fostered the open-source community immeasurably. Through their commercial offering, SourceForge Enterprise Edition, they've distilled that open-source magic into their own commercially successful brew. But VA Software found their business growing in other directions as well, into online media, and have discovered a need to focus their energies there.

Transferring SFEE to a more suitable home became very attractive. CollabNet is a great place for SFEE, and we're delighted to bring over this complete team, well experienced in our core field, and their solid and innovative product. So where do we go from here? Hey, give me a while to work out the details of a combined product roadmap; I'll get back to you on that. But I'm "new" around here, and we're just getting started!

Do you live in the Bay Area, or are you going to JavaOne?

CollabNet is organizing a success party for Subversion. Join members of the Subversion team at CollabNet’s headquarters in Brisbane (just south of San Francisco). It will be an informal party but we’ll have some Subversion related things to show as well, like a preview of the upcoming capabilities of Subversion 1.5.

The party will take place on May 10th starting at 4pm with cake and champagne. After that we will have a networking reception where you can meet with people like Brian Behlendorf (CollabNet and Apache co-founder), Subversion committers and many others from the Subversion community.

Find out more and register at http://www.collab.net/subversion_party_RSVP/.