How can you understand how your CollabNet site is really being used? This simple question has many implications but few answers. 

Generic web monitoring tools can do their generic best, but they don't know anything about the structure of your CollabNet site and the CollabNet product: projects, components, and actions are all key to how you use the site, but opaque to generic tools. They're not integrated.

On the other hand, tools based on inserting code snippets into your pages really demand that you already understand what's important. Too often, they can appear to confirm your theories--but only because you didn't know enough to ask the right questions. They're not exploratory.

We have a tool suite on our community site, called "CollabNet Apache Stats," or CollabaStats, that makes some attempt to be better integrated than web monitoring tools, while still preserving the exploratory power to shake you out of unexamined assumptions. CollabaStats is a log-analysis tool focused on the Apache HTTPD logs, but with extra understanding of how the CollabNet site software works.

The project website contains a wiki that guides you through setting up the data capture and insertion into a database. (Currently, CollabaStats requires a MySQL database.) Be sure to look over the Requirements information: such a database can be quite demanding, depending on your site's traffic levels and the amount of history you try to analyze.

To see the actual files of the project, you'll need to log in to the site.

The Source Code repository for the project is structured as an Eclipse BIRT workspace, to help you design reports to explore your site behavior. There are a dozen or so report templates there that you can use and modify, and pre-digested examples of their reports, so you can see some of the possibilities:

• Active Projects based on page views: template and report
• Active Projects based on Subversion use: template and report
• Most-used CollabNet components: template and report

This is a community project: you can participate. There are discussion fora where you can ask questions and (once you've gained some experience) provide answers. There's a Tracker for reporting bugs, requesting enhancements, and finding ways to contribute. And if you want to make significant code, documentation, or sample-report contributions, contact the Project Administrator (listed on the Project Home page) about becoming a project member.

It's a community: the  project, the process, and the source are open to you. Because that's the Open Source way.

As Jeffrey Hammond, of Forrester, blogged recently, Subversion is the overwhelmingly predominant version control system today. Yet if you read the open-source blogs and discussions, you see loads of growth in the "Distributed Version Control Systems" (DVCS), such as git, Mercurial, Bazaar, and the like. What's going on here?

There's an interesting even just now in the Linux kernel that sheds some light. This week, according to Greg Kroah-Hartman, some Google-contributed, Android-related code was removed from the Linux kernel. The reason: the code in question is both incomplete on its own, and specific to the Android device. Either of these properties makes it constitute a "fork," and in this case there seemed no prospect of ever healing the fork.

Now, you might think a "fork" is something that happens to the code, and certainly that's part of it. But as Kroah-Hartman points out, code-forking happens all the time. The problem here is that "no one cared about the code, so it was removed." This doesn't mean no one cares about the code that's going into Android--certainly not! It means two other things:

• No one was taking care of the code that had been contributed into the Linux kernel tree
• No one was consuming, using, extending, or applying the code for new purposes

It had been, as the phrase is, "thrown over the wall," and left.

That--abandonment--is a big problem for open-source code, both in that it's harmful and wasteful, and also in that it's common. But it's a cultural problem, not a technical one: there's no claim that this was "bad" code, just that it was a dead end.

Open-source communities are actually pretty robust against this sort of harm. This event illustrates one such defense: if it's dead, dump it. Another commonly applied defense is simply to let projects die when no one cares: the notorious numbers of open-source projects that never got anywhere are actually signs of health: the open-source community has a powerful defense called "cheap failure," that allows ideas to start out sounding good, then peter out harmlessly. One of the cool things about DVCS is that it's very easy to start a fork, very easy to reintegrate it if you choose to--and very easy to chop it off and toss it away if it ends up a dead end, as we see here.

But if we recast this whole experience into an enterprise context, it begins to look very different indeed. The essential difference is that failure within an enterprise is never "cheap." If the Linux kernel community, including Google/Android, were an enterprise, then all the work that has been spent on this fork would have come out of corporate wallet, and the decision to toss it would be a recognition of wasted resources: salaries, equipment, time, plans, marketing, process--endless kinds of costs, sunk with no payback.

The great contrast between the low cost of open-source failure, and the high cost of enterprise failure has far-reaching effects, even down to details like tool choice.

• An open-source project can usefully tell prospective contributors "clone the tree, work up your idea, and get back to us when you have something to show," because there's no cost to the project if the idea never bears fruit. An enterprise, however, does have such costs, and at the least has little incentive to allow that sort of work style.
• An open-source project can allow parallel, semi-independent, persistent forks, such as the several Linux distributions: as long as there are enough people (whether sponsored or volunteer) interested in working the fork, as long as things stay controlled enough that work can be merged back and forth, there's no problem. But an enterprise may need to match its centralized budgeting and resource management with some centralized supervision, planning, marketing, and management: making it easy to disappear from the radar is not a good thing.

And that, I think, is at least part of why the centralized Subversion model remains so strong.

An analysis of several Forrester surveys, by Forrester analyst Jeffrey Hammond, was recently published in Dr.Dobb's online (www.ddj.com). The analysis compares what developers say about their work environments with what their IT managers think is going on. Hammond calls out "seven trends that could have major implications for your IT strategy."

Looks good to me ... in fact mostly, it looks like the CollabNet strategy book:

• Rich Internet Applications, like our Eclipse, Visual Studio, and Windows desktop products
• Open Source use spreading everywhere
• Virtualization and Cloud converging into virtual private clouds (among other things), like Lab Management
• Growing Agile and agile-like development processes

One bit seems odd, though: Forrester finds that

• More than one-third of developers use Subversion for source code management; that's almost triple the share of the next most-used SCM tool, Microsoft SourceSafe.

via www.ddj.com

I might be reading that wrong, but it appears to say that Subversion is the number one version control system in their survey, by quite a wide margin. Sounds good so far. But it also sounds a bit odd: if number one covers 1/3 of devs, and number two covers 1/9, the series seems to converge with less than half of all developers using any kind of version control at all, and I don't buy that. I don't know any developer anywhere who doesn't use version control, even for personal work. In fact, I was accosted the other day by one of the baristas at my favorite coffee shop, because I was wearing a Subversion T-shirt and he wanted to tell me how to make Subversion better! Surely, that's gotta mean that Subversion use is wider than 33%?

Update: Looks like I was indeed reading that wrong: the author clarifies that the survey asked for "primary SCM." I'd expect a healthy plurality for that question. He also confirms my expectation that most developers use several SCM systems (he cites "2.1 in Europe, 1.8 in the Americas," which gets my nationalistic blood up -- come on, North- and South-Americans, we can do better! ;-). Given which, I could well imagine a lot of respondents grumbling "how am I supposed to name a 'primary' in this mish-mash?" and perhaps just checking "None" -- not "no SCM," but rather "no way to choose a so-called 'primary'."

Update number two: Author Jeffrey Hammond has blogged the SCM results in much greater depth at the Forrester Blogs. The greater depth is particularly interesting in the area of open-source SCM tools: nearly half of the respondents listed some open-source SCM tool as their "primary SCM tool." There are a lot of areas in our business where people are given to wondering why on earth anyone still "pays full price" for a commercial something-or-other, when there are such good open-source alternatives--but, in a lot of areas the fact is, people do. Looks like SCM is an area where open-source offerings really are winning!

From MASSForge, the software collaboration environment for the Commonwealth of Massachusetts, comes a great add-on to your CollabNet TeamForge site, the Calendar linked application. Better yet, this is an open-source project, hosted on our public site, ctf.open.collab.net, so if you see some improvements you'd like to make, you easily can—as well as contributing them back to the project, if they might be generally useful.

The application deploys easily into any modern Tomcat server. With that plus "Linked Application" configuration into your project, you're off and running. A new tab will appear in the project's tab-bar:

 

... and Tasks entered in the project's Tasks tool will be displayed in the calendar:

As with any good open-source project, there's a wiki with installation details, and a full suite of discussions for Announcements, Help, and any Installation Issues (none reported so far—be the first!)

You can check the project out from Subversion, or download the latest release from the File Releases area.

One of the stalwarts of Subversion, Ben Collins-Sussman (a Googler and former CollabNetizen), remarks

Not that this should shock anybody, but in case you didn’t know, now you do. The overlap between Apache and Subversion communities has always been huge since day one — with essentially identical cultures. We’ve talked about doing this for years. It means we can finally dissolve the ‘Subversion corporation’ and let ASF handle all our finances and legal needs.

via blog.red-bean.com

Observers may not realize just how true this is. This announcement is a great thing for Subversion, for ASF, for CollabNet, and all other companies who depend on Subversion for their work and products, because it means that the supporting details are in good hands, and the contributors can get back to their contributions. But in another sense, it's about as "un" an event as it well could be.

One anecdote from the press conference yesterday really drove that home for me. John Mark Walker, of OStatic, and Paul Krill, of Infoworld, asked how this change will affect Subversion developers and users. Everyone around the table leaned forward to answer, but the winner was the current President of the Apache Software Foundation and a major Subversion committer, Justin Erenkrantz, whose response began "In the Subversion community, we've always operated in the Apache way." This was echoed and reinforced by Sander Striker, past President of the ASF and another major Subversion committer, and Greg Stein, several-time past Chairman of the ASF and one of the most active Subversion committers of the moment.

So there you have it: if Presidents and Chairmen of the ASF think of themselves first as members of the Subversion community, how much change could we really be talking about?

The Internet is full of comparisons of the relative popularity of web browsers, but something these analyses miss is that "web" traffic is increasingly not about "browsers" at all. As SOAP and SaaS and Clouds expand, the web has largely become the Internet: the backbone for many services not directly displayed to human eyes.

As a case in point, consider our site Tigris.Org. I was analyzing traffic there recently, for other purposes, and came upon some interesting numbers. In a period of a few (basically randomly selected) days recently, 

• Firefox browsers hit the site 2,964,556 times, making them #3
• Internet Explorer browsers hit the site 3,350,885, making them #2 by a fairly narrow margin (13%, which is narrow compared to the 40% or more these numbers change from day to day)

But "who's number one" you ask? Subversion, with 5,724,275. Subversion traffic is roughly equal to Firefox and Internet Explorer combined! 

Chrome (501,455) is doing pretty well, for a newcomer, clocking in around #6, and Safari (302,483) is at #7. 

"Wait," you say: "there's another gap there, what happened to #4 and #5?" Web crawling spiders working for search engines come in at #5 (1,116,299), but what's #4? Java libraries of several sorts (1,798,848).

Sorted out in order:

1. Subversion: 5,724,275
2. Internet Explorer: 3,350,885
3. Firefox: 2,964,556
4. Java: 2,915,147
5. search 'bots: 1,384,917
6. Chrome: 501,455
7. Safari: 302,483

So: version control operations are overwhelmingly number 1: people are using the web to create new stuff.

Browsers are still big business, at #2 and #3 (don't short-sell your Firefox stock just yet ;-), but on the web, "automation" is "Java" is roughly equal to any human browsing.

I've just started reading Trust Agents, by Chris Brogan and Julien Smith, and if you're interested in community building and innersourcing, I think you should, too. 

If you know of the book, that might surprise you: Chris is well known in the "Social Networking" world, possibly more closely related to marketing than to software development, and I don't really do marketing. But in many ways, "Social Networking" is one of the ways in which open-source techniques are finding new life in other domains. Here's what I mean:

In "Trust, Social Capital, and Media" (the first chapter), the real money quote is:

Communities don't want to be managed: they want to be cared for.

That's a great statement! You can find it in the foundations of such open source classics as Karl Fogel's Producing Open Source  Software. Actually, I don't think Karl has any one quotable statement of that principle, but it's easily recognizable as one of his primary convictions. Consider, for example, Karl's Setting the Tone:

Choosing a mailing list address is easy; ensuring that the list's conversations
 remain on-topic and productive is another matter entirely. 

Again, Chris describes the dynamics of Trust Agent conversations:

We're just interacting. We used to do that by e-mail, which is private;
 but the fact that it's all done in public view now means that all the participants
 on the Web are creating value for each other simultaneously.

And Karl's Avoid Private Discussions section similarly shows how public discussions benefit many people, not just the original questioner, as well as creating permanent records that go on giving value long after the original conversation is complete.

Chris ties this all together like this:

Social capital is different from other kinds of capital.
 When people come together and share a meal, they not only end up fed,
 they also become tighter as a group. ...
Just think of your favorite television cop drama and how often the phrase
 "you owe me a favor" is uttered. These things are real.

In the literature of open source, this is usually referred to as a "gift culture," well-presented by Eric Raymond in his paper Homesteading the Noosphere:

If one is well known for generosity, intelligence, fair dealing, leadership ability,
 or other good qualities, it becomes much easier to persuade other people
  that they will gain by association with you.

So, what does it mean to care for a community, not just manage it? Again, Chris:

Make everything you write something that's helpful to other people;
 if those people might also be your customers, all the better.

That's the open-source inspired, communitarian spirit in which we recently started up our Community Management project (think we should change the name?).

Are you a Trust Agent? Would you like to be? Come join us, and build up a little social capital!

A while ago, I pointed out that many discussions of open-source related business models are leaving something out: infrastructure. A lot of people have asked me to explain the difference between "open core" and "open infrastructure"--don't they both mean "open-sourcing the basic stuff"? Well, to some extent they do, but the difference is real, and apparently even more significant than I had thought.

"More significant"? Looks like: Kim Weins at OpenLogic provides some survey results that indicate that customers aren't so happy with the "core vs. complement" choice. And apparently, they're not alone: the much discussed decision by the European Union, to look more closely at the Oracle-Sun merger with an eye to protect the open source MySQL product has been widely decried as a misunderstanding of open source, but may really be just another expression of this same discontent. Matt Aslett of The 451 Group has made a good case that the EU's concern strikes to the heart of the distinction I'm trying to draw. The essential concern is that the open-core approach can mean that the proprietary sponsor of the open-source project can delay, divert, or simply out-pace the community version, making it not really viable. When that happens, you're perilously close to "faux-pen source," something that pretends to be open, but is only viable commercially.

In contrast, the "open infrastructure" model identifies a viable, useful product to be created as an entirely open-source project, and then used to build something fundamentally different. The Apache community has many examples; I think this model may have been an Apache invention. To pick one specific example (and one I can speak most about, as I'm involved), let's talk about Subversion.

Subversion is an open source project. It's a completely open source project: the whole product is available under an open source license. CollabNet (yep, that's us) is, and always has been, the primary sponsor of Subversion, contributing around 25 percent of the labor, over the years, as well as a major portion of the community leadership, and the grunt work of release processes, road maps, and project management. 

I don't think there's any doubt but that there would be no Subversion, and no Subversion community, without CollabNet, because of the way the project began: Brian Behlendorf, founder and then-CTO of CollabNet, hired the team, gave them an office ... and then made them work as an open-source project, in the open, instead of an internal development project. Just as clearly, CollabNet did this for business reasons. But the Subversion product, produced by the Subversion community, is in no way "crippleware" merely intended to lure you in to the CollabNet commercial "real" product. Slipping down that slope is the great risk of the "open-core" model, where your profitability depends on a narrow feature advantage over the open source alternative. It's a risk the EU appears to be concerned about. 

CollabNet chose not to follow that path at all: Subversion is Subversion. Any Subversion is all of Subversion. Instead, CollabNet provides another, larger product: CollabNet TeamForge, a product that would be much less valuable without Subversion (which is what justifies our investment), but clearly a different product.

So what's the difference? You still don't see it? OK, maybe it's a matter of degrees. Maybe a metaphor would help. Let's say you want to be a car manufacturer. You can open-source the production of your tires, and build a car that uses those tires. That's "open infrastructure." I don't guess there's a lot of "open source tire manufacturing" going on, but I'm pretty sure that something like 100% of car manufacturers at least "out-source" their tires! It's viable, it works, everybody does it. It's a business model.

But if your automobile business model is to "open core" everything but the paint, if your value-add is only the paint ... well, then you're going to see some paint-thin profit margins, too.

Commenting on the differences between the business models of Red Hat and Acquia, Matt Asay really nails a point: there's value to be added by a company simply packaging and redistributing open-source work, but it's greatest where there's intimidating complexity to be managed. Both Red Hat and Acquia provide reliable, solid packaging of underlying open-source work (Linux and Drupal, respectively); both also provide additional value of their own, built on top of that infrastructure. But Red Hat's model relies proportionately more on the value of their packaging of the infrastructure, while Acquia weights itself more towards extensions attractive to enterprise customers.

But looking forward, he suggests that the increasing use of unpaid Linux "has the potential to disrupt Red Hat, Novell, Canonical, and other Linux vendors," and I'm not so sure of that. The trend is indeed worth watching, and the potential for disruption is real, but the value of simple, trustworthy distributions, with a legal entity answerable, remains very real for enterprises. Aside from present economic circumstances, the unpaid Linux growth is surely due at least in part to the increasingly good job of packaging and QA being done by the various distributions (not all of which are commercially backed). But mostly, I think it's the Linux-level mirror of the "adoption" benefit of more contained commercial-open-source projects. And that means much of it is use in circumstances where a commercial version wouldn't have been used anyway, and much more grows from unpaid to paid when the project at hand succeeds.

CollabNet's sponsorship of Subversion is a nice counter-example. Subversion is nowhere near as complex as Linux, or even Drupal, and Subversion is widely available in one-click installers, native libraries, and most or all of those Linux distributions just mentioned. But the CollabNet Subversion packages remain extremely popular, because they're certified by CollabNet. Many of our customers could build Subversion themselves--but that's not where they choose to spend their developer time and energy (they have their own jobs to do). Many could download some other package, or use the one provided with their platform--but CollabNet's timely updates are more important, especially for security fixes. And for enterprises, having support people, people who will answer the phone and the questions, provide the training, and provide the experience, are worth not only the trouble of using CollabNet's free downloads, but also make a support contract a worthwhile investment.

Why do people contribute to open-source or inner-source projects?  It's long been recognized that one major aspect is something usually described as "peer esteem." In Apache and the future of open-source licensing, Matt Asay makes the mistake of assuming it's the only factor, and compounds the mistake by assuming he can (re)define the term for everyone else, but he's certainly right that it's one important factor. So what does it mean, and what can we do about it?

Also this week, and totally out of left field for this blog, American Public Media's Speaking of Faith program has published an interview with neuroeconomist Paul Zak and a wealth of supporting material on the neurobiology of economics and cooperation. The research reveals some remarkable things that can be applied to this idea of "peer esteem" in the building of communities.

And (as retweeted by another author in this space, Guy Martin), Steffan Antonas has published a great discussion on How To Say Thank You On The Social Web. As he points out, "saying thank you" is more than courtesy in a community: it's the primary "currency" of the virtual gift economy—which is just the sort of economy, it turns out, that Zak is talking about.

Where these all come together is this: the kind of esteem, the kind cooperation, and the kind of thanks that each build community is, in every case, the person-to-person kind. The effective kind of esteem is not just the esteem of some faceless mass of objectively assessable "peers," it's the esteem of the peers I know. And it's not the esteem of those known people who are my peers by some scientific test, but those who I esteem as peers.  And, further, my esteem for them arises from and contributes to their esteem for me: we are mutual peers.  And such mutual esteem can only grow within, and cannot help but further feed, a strong sense of belonging to the same community. And this is just exactly the sort of community that Community Managers need to foster and feed, in order to spark the contribution and cooperation that makes open- or inner-source work so productive and powerful.

The sources I link here reveal a rich interplay of insights into what "peer esteem" is, how it's done, and what it produces. So there you are, Community Managers: you have your reading assignment for the week!