awareness and scaling down

I’m slightly freaked out that I have only seven months to wrap up my thesis work, so I’ve been really trying to nail down the specifics of my research in the last few weeks. The biggest challenge has been in finding a balance between doing interesting work and finishing my degree quickly.

For a while I was keen on doing a Human-Computer-Interaction-style study of a tool to support information seeking in software development. This type of research would involve recruiting a development group to study, conducting ethnographic-style interviews to uncover information needs and difficulties, building a tool to help, and evaluating it.

I think this would be really interesting — after all, real-life evaluations of such tools are rare — but I’ve now realized that the amount of time required to do a study like this puts it way out of scope for me. I was stuck in a rut trying to figure out how to maintain the flavor of the research while cutting it back to a more manageable chunk. I asked for some advice from my friend/colleague Jorge and he gave me some helpful strategies to scale things down.

Ethnographic-style interviews take up a huge amount of time in preparation and data analysis, so if I can avoid doing them, I will. One way to skip this step is to just dream up some kind of tool and see if it works. The literature is riddled with mediocre work of this style, though, and I’d really rather not. Instead, I have to ground my study based on previous work. This is tricky given that information seeking in software development is still very much in the exploratory phase, but I think there’s enough for a solid footing.

Awareness is one area of information seeking that now has some texture to it: multiple studies show that developers seek out awareness information relating to software project artifacts (what resources have changed?) and coworkers (what are my peers doing?). This seems to be the case for both collocated and distributed teams, although awareness information is easier to obtain in collocated teams. These are a few of the recent and interesting studies that I’ve come across:

• Information Needs in Collocated Software Development Teams
• Maintaining Mental Models: A Study of Developer Work Habits
• Group Awareness in Distributed Software Development
• Awareness in the Wild: Why Communication Breakdowns Occur

There are already a handful of tools that aim to support developers in maintaining awareness of artifacts and coworkers. Storey et al. provide a nice survey of these tools and a framework for describing them. With the exception of FASTDash, however, most of these tools are not evaluated at all or are evaluated based on very short, laboratory-style user studies.

I think there’s an opportunity here to do better. In addition to source code modifications, which is what FASTDash monitors, the research shows that awareness information also comes from emails, chat, and bug tracking systems. I’m already somewhat familiar with interfaces that integrate project data across multiple repositories and I’d like to see how developers make use of a tool that provides integrated awareness information in a real-world environment.

So, I’m going to head down the awareness road for a while. Any feedback is greatly appreciated.

I’m starting to make connections with software development groups that might be interested participating in my research. Obviously the details are still evolving, but if you’re reading this and you’re interested, get in touch and we’ll talk.