Remote Pair Programming
A key ingredient in the success of EdgeCase is collaboration. We decided early on that the most effective way to develop with each other was by pair-programming. It has been fantastic.
An irritation I had before founding EdgeCase was that more companies did not try and work hard enough to figure out how to overcome the barriers of having people offsite. With the plethora of technology available, I was convinced that we could be a very effective organization without either requiring our developers to commute using a 747, or to pass up on great team members just because they didn’t live in Columbus.
The first real test came when we hired Jim Weirich. Jim lives in Cincinnati, two hours south. We had already become accustomed to working with each other in a 100% pairing environment. We made it clear that we had no intention of making him drive up to Columbus every day, nor did we expect him to move. We also had no intention of leaving him alone in his basement and throw work his way. So we decided to figure out exactly how we were going to pair with him in Cincinnati.
Our requirements were pretty straight forward. We needed:
- ability for us to work with multiple files (When in Rails, you are never in a file for very long, especially when bouncing back and forth between tests and regular files)
- a way to communicate effectively
- little or no perceived latency
Secret Sauce
After a lot of trail and error (mainly researched by Jim and Chad) here is what we came up with:
- ssh to a server that both parties can access
- screen so that both parties can share a session
- Emacs (or any headless editor that does not require a GUI environment)
- Video Skype
ssh and screen
ssh and screen allowed us to create an environment that uses up very little bandwidth. This was awesome for two reasons. One, there is very little latency (especially if you ssh in with the -C option). Two, it gives us extra bandwidth for skype.
If you have not tried it yet, you really should check out screen. It’s a really cool command line tool that most *nix environments come with now days. It allows us to look at the same shell session. So we both ssh into the box, and then join the screen session (in our case one called pairing). Then we can see what the other person is looking at. When the cursor moves, we both see it.
Some of the gotcha’s that we have encountered:
- Try and ssh in as the same person. This will keep down the amount of headaches you encounter with file permissions, etc…
- create a separate subversion user for the ‘pairing’ server (again, it helps with file permissions)
- Tweak your timeout settings. There will be lots of times where you are watching your pair type, and not actually interacting with the keyboard. ssh likes to think that you have timed out and kick you out. This is something that can be configured (I believe in /etc/sshd_config)
Video Skype
While this sounds like a luxury, it’s actually become a fantastic tool for pairing. Nothing beats seeing the facial expressions of your pair. You can pick up some great cues and it feels as if he/she is sitting right next to you. We both are using macs and have a two monitor system. Inevitably you will find yourself looking at the video image of your pair. Therefore we have elected to drag the skype window right below our camera on our laptop lids. This way, when you look at the picture, you are in essence looking into the camera, further helping the feeling of ‘being there’.
Emacs
Unfortunately some people in the company are not advanced enough to use use Emacs, and have chosen to use a remedial editor. Either way, spending time learning one of the main editors is not wasted effort. While it seems like at first you are trying to run through mud, you will be amazed at how, after a month or two, you are working faster than you ever thought possible. Years of effort have been poured into tweaking these editors and making them unbelievably extensible.
All in all, the important thing is that we do not have to launch an entire X instance simply to run our editor. TextMate is a good editor, but it’s not worth moving to something like VNC simply so we can use it (although, a subject for another post, we have actually found that TextMate is fun, and shiny, but just does not give us the support we need anyway).
Another benefit to using one of these editors is the excellent buffer support. Being able to seamlessly move from one buffer to the next, moving text around, and executing files leads to some excellent increases in productivity.
Extra Communication
Not being there does have other challenges. When you are pairing together in person, who gets the keyboard is usually pretty easy to figure out. When you are remote, you simply need to ensure you communicate more. We use phrases like tag when we are about to take over the keyboard, or yield to say you giving control to the other person.
We also do not have the luxury of a white board. So communicating ideas is a bit more difficult. This is the only place where I would say “I wish we were in person.” During our time pairing, we have become better at communicating our ideas. More times than not, we end up thinking in code. Again, it’s not great, but it does help.
Is This For Me?
This is by no means our way of saying, this is the only thing that works. What I am tyring to do is give you some more insight into how we operate, and what things we have found to be successful. While we are encouraged by tools such as Coda and SubEthaEdit they are still a little ways off. Stay tuned, as we continue to bring on more talented developers and put our recipe through it’s paces.
What has worked for you?



While I’m not convinced that Pairing is right for the way our team works… I’m glad to hear that you’ve found a lofi solution to this.
Might I suggest joe as an alternative editor that is easy to pick up for people new to the terminal editor world. I’ve been using it for about five years as my primary editor when working on servers.
heh, “lofi” solution. Name a technology we’re NOT using!
Remote pair programming has worked out very well for us I would agree. Now that leopard is out the new iChat will enable video chat and simple style whiteboarding/screen sharing.
I would also like to clarify that all the EdgeCase coders (sans 2) use VI/TextMate 100% of the time.
You will so be amazed at how, after a month or two of writing your own config file for Emacs that other editors will be a blessing.
Just curious, why not VNC? Don’t want it to take over someones entire computer? VNC in a VM?
Not to say vnc is better, just different. I spent 2 hrs last night pairing with someone over VNC and gtalk-voice. It was awesome.
Joe,
http://www.imagination3.com/LaunchPage
It’s pretty easy from playing around with it.
Michael Riley
After attending erubycon and seeing Jim Weirich’s setup with screen and emacs, I have been stuck in console ever since. I program entirely using screen and vim (the superior editor ;)) and I love it! I have messed around with ssh’ing for pair programming and it works VERY well and gets rid of all the overhead of VNC. I’ve not yet had the opportunity to use this method hard core for pair programming, but I still have found a huge productivity increase just by using screen and vim alone. Anyway, should I be linking my blog here or to edgecase? :)
Hi Joe, long time no dumb Ruby questions from me. :-)
I’m working now in QA at a company called Socialtext. Socialtext’s dev, QA, and support staff all work remotely from across the US, Canada, UK, Taiwan, and Estonia. We even have our very own curses-based wiki editor to work with: http://search.cpan.org/~lukec/Socialtext-Wikrad-0.05/lib/Socialtext/Wikrad.pm
On Nov. 14 I’ll be part of a panel on distributed working at the Boulder Ruby Group meeting. One of the people on the panel will actually be participating from California via video iChat, IRC, and the wiki, which will be made totally open and public for the duration of the panel.
It should be an interesting panel, since at least one person on the panel feels like he is not being successful working remotely.
If you’d like to join the meeting, let me know and I’ll pass on the details to you.