The Challenges of Remote Collaborationby Mark Murphy
As a reader of the O'Reilly Network, you're probably familiar with remote development, in the form of public collaboration around open source and free software. You are used to the notion that teams scattered around the globe build many of the tools and applications you use, from Linux and Apache to Perl and Python. For some of you, over the past few years, your boss has gotten into the act.
Remote software development is becoming increasingly important to major technology firms and the IT groups of other large firms. Collaborating in business settings resembles volunteer public collaboration, but it's not identical. It is up to you and your boss to help promote a development model and system that will be effective for everyone.
Why Remote Collaboration?
Many firms have had geographically dispersed development teams for some time now. After all, if you are like Barclays Global Investors (BGI), with offices in San Francisco, London, and Sydney, you probably need development teams in each location. Originally, these groups may have worked largely independently. More recently, to facilitate software reuse or to balance software development loads, firms have wanted such teams to work more closely together, sometimes even on the same projects.
Many firms are starting to try to involve non-developers in the software development process, to a greater degree than had occurred before. Sometimes this is tied to a new development methodology, like extreme programming. Other times it is merely a way to try to get better, faster results by reducing communication gaps between customers and the development team.
BGI Software Management Project (SMP) is a corporate-wide initiative to unite a variety of stakeholder groups, from developers to project managers and business users, on a single global software development collaboration platform. This includes combining key functions such as: source code control, version management, quality assurance, testing, and change management, into a unified framework that BGI's software development participants can access from anywhere in the world, anytime.
--from a CollabNet white paper
What's also new is the growing buzz around such offshore development, typically referring to sending some amount of software development work to groups located around the globe. This could be temporary, such as using a remote contract development shop. It could also be more permanent, such as a merger of two firms, each with its own IT department.
Already, more than 300 of the Fortune 500 firms do business with Indian IT services companies, according to Gartner. The research firm predicts that by 2004, more than 80 percent of U.S companies will have considered using offshore IT services. In addition, more than 40 percent of U.S. corporations will have completed some type of offshore IT pilot program or will be using IT services with an overseas component by that time.
--from U.S. firms move IT overseas by Ed Frauenheim, as appeared in News.com
Still other firms are pursuing remote development as a form of partnership:
Through rapid and secure third-party integration, [HP Imaging and Printing Group] and the HP partners participated in co-development of a digital Application Specific Integrated Circuit (ASIC). The design has potential for leverage and reuse with other digital ASIC teams. Although the members of the project were based in multiple locations around the world, HP team members were able to work closely and more efficiently with external partners, resulting in higher quality and fewer integration issues.
--from a CollabNet white paper
Concerns Over Remote Development
Many developers get antsy with the topic of offshore development because it conjures up visions of shipping jobs away from their home. However, there are other reasons why a firm might want to link up with a remote development partner, such as a need for specialized talent (e.g., Israeli expertise in encryption and data security).
You, as an individual developer, can also benefit from the use of remote development teams, regardless of their origin. You will get a chance to interact more closely with a wider set of peers, exposing you to other cultures and development styles. While some of that may be scary, it is frequently valuable to see how other people build software and conduct projects. You can blend ideas learned in this experience in your future endeavors.
The biggest challenge with remote development teams--no matter which side of the relationship you are on--is to maintain quality and productivity across all participants despite the physical and cultural distance. After all, it will be of little help to anyone if combining remote development teams makes these things worse.
Some problems you are used to, such as spinning up new team members on the application being built and the technologies being used. Adding a remote development team magnifies the problem.
Other issues are operational. You need to ensure that everyone is working on the same edition of the source code and has access to the same information about requirements, data models, customer issues, and the like. It is very easy to have Team A and Team B accidentally work off of different code bases, such as different versions of a common component, and wind up trying to integrate incompatible work.
Social issues also come into play. If the teams adopt an us versus them mindset, they might well wind up working at cross-purposes, with Team A being slow to adopt Team B's work due to distrust or outright disdain for who Team B is and what they are working upon. This is exacerbated by time zone and language differences that make real-time communication difficult.
Why You Should Care
If you are a developer, and you have gotten this far in the article, you probably already care about the issues with remote development. As well you should!
While executives and managers are typically the ones establishing relationships with remote development teams, it is typically up to you, the developer, to make those relationships actually work. Even better, you must do this while you and your peers are trying to crank out a new application, changes to an existing system, or some other chunk of work.
Hence, it behooves you to think about how to organize the project to make such remote collaboration go as smoothly as possible. After all, if you will be spending your time interacting with a remote development team, anything you can do up front to reduce long-term hassles will be worth your while. Besides, you and your teammates, regardless of what shore they are on, need to get your work done and be successful at it. Finger-pointing at the end of a failed project, to explain that it's their fault, may be a necessary evil, but it is still evil.
Collaborative Software Development
If you are going to collaborate with a set of remote developers, take a page from those who have done this before: open source and free software projects, plus other group activities like standards bodies and consortia. So, think about how the W3C, OASIS, Linux, Mozilla, and OpenOffice.org have all coordinated thousands of participants across the globe. In all those cases, the groups had
- Tight communications, mostly via asynchronous formats like mailing
lists, to make sure that everybody can be kept informed about everyone
else's progress, questions, and concerns.
We host some mailing lists and newsgroups at mozilla.org, to foster open communication in the developer community. We add new forums with some regularity, in response to the needs of you, the developers. What forums do you want? As new special-interest groups appear, we will create new newsgroups and mailing lists, or whatever seems most useful.
--from Mozilla.org site
- Full tracking of all collaboratively-built artifacts, from
specifications and designs to source code and QA test results, so all
participants have rapid access to the precise deliverables that everyone
is working on or are using as references.
Our developers are located all around the world. To enable them to work together on our software, we keep the source code in an Internet-accessible revision control system called CVS. Apache committers have write access to the CVS repositories, enabling them to make changes to the source code. Everyone has read access to the repositories, so you may download the most up-to-date development version of the software.
--from Apache.org site
- Some form of workflow, whether via a proscribed process or implemented
through tracking software, to make sure that everybody knows not only what
they are responsible for, but what everybody else is working on as well.
When possible, please use IssueZilla. Why use IssueZilla? Because it allows everyone to keep track of the many big and small tasks, requests, enhancements, whatever that circulate throughout an Open Source project. What is more, the issue tracker makes it easier to determine if any another user has had similar problems. And if the problem has been resolved, it can alert the relevant people to the solutions found. In short, by registering and then filing issues with IssueZilla, you become a contributing member of OpenOffice.org.
--from OpenOffice.org site
If those approaches and tools can work for thousands of widely scattered individuals, they should suffice for a few teams in a few locations. Combine that with some of the social aspects of public collaboration (give credit where credit is due, conduct continuous peer reviews, etc.), and offshore development will not seem so, well, foreign.
Gotchas and Solutions
Of course, if it were that simple, you probably would not be bothering to read this article. Development of proprietary software with remote teams has significant differences from what you see in the recent wave of public collaboration. These differences do not change the prescribed approach, but may change how you implement collaborative software development.
One difference is with security. In public collaboration, the fruits of the labor are designed to be public in the end, whether they are proposed standards or pieces of software. Hence, while security is useful, perhaps to keep things private until they are ready, remote collaboration upon a firm's proprietary software raises the ante a fair bit.
You need to establish technical means of ensuring that only the relevant development teams have the ability to view and modify the work in progress; other random individuals at either location, or Internet crackers, cannot have any way to get at the project. Depending upon the security requirements of your organization, this can be as simple as using SSH tunnels for sharing code via CVS all the way up to requiring X.509 certificate-based authentication and integration with your existing authentication database.
It is one thing to set up collaborative software development with one remote team. What happens if you start working with ten remote teams or start using the same techniques with internal, but organizationally distant, teams (e.g., IT groups for different product divisions)? Suddenly, what seemed like a simple matter of slapping together some mailing lists and an intranet turns into a management nightmare.
When setting up your collaboration environment, aim for a structure that will support more than just the one team you may be collaborating with today. For example, set up your mailing lists, issue tracking, etc. on a per-application or per-project basis, so that each distinct collaboration initiative has its own workspace. Of course, this ties back into security: you need to choose and configure tools that support multiple collaboration efforts, where distinct remote teams only have access to what they need.
Internationalization and Localization
Of course, with offshore development may come internationalization and localization issues, both with the collaborative development tools and the work products the teams are creating. It will not help your cause to mandate the use of tools that many members of one team cannot understand because of language barriers.
This gotcha transcends tools. Having localized tools helps, but they are useless if you cannot understand the other party in general. As much as anything, you need to consider this when choosing the teams to work with, or perhaps the task assignments in the teams. Putting multilingual people in key positions can help relay information to teammates who may only speak one language.
Collaborative software development--the development of software over a physical, organizational, or cultural distance--is becoming increasingly important in today's IT world. Many groups participating in public collaboration projects, such as open source software, have already honed techniques for such collaboration. What is needed today is the translation of those techniques into the enterprise, to help make successes out of the desired collaboration projects.
Return to the ONLamp.com.