Aperte.org » Research

Learning from strangers

Jeremy Handcock — Fri, 06 Aug 2010 07:00:03 +0000

I'm behind in my blog reading lately and I just now came across Whitney Hess's great post My Best Advice for Conducting User Interviews. If you're new to interviewing, I highly recommend checking out her tips to see what it's all about. Her advice is applicable whether you're doing casual or empirical user research. I also highly recommend checking out the more substantial reference Learning From Strangers: The Art and Method of Qualitative Interview Studies by Robert S. Weiss. It's a wonderful book complete with methods and advice for every stage of the interviewing process including participant selection, preparation, conducting the interview, taking notes, analysis, and presentation. I bought this book when I was in graduate school to prepare for interviews with my research participants and it was immensely helpful. It's also highly approachable, even if you don't know anything about research. Conducting interviews is actually my favourite part of doing user research. I love talking to people about their work, learning about how they work, and listening to their stories. It's hard work (and as Whitney says, it can be pretty mentally exhausting) but going in prepared will make the process much more fulfilling. Enjoy these two references, and happy interviewing! Share:

Almost famous

Jeremy Handcock — Wed, 09 Jun 2010 06:52:51 +0000

It has been a whole year since I graduated in Toronto. My, how time has flown! In an overdue attempt at a victory lap, I tried to publish some of my research at the VLHCC 2010 conference. Unfortunately, I didn't make the cut. It was worth a shot though and I'm glad I made the attempt. In the spirit of openness, I've posted the paper that I submitted to the VLHCC program committee. It's my thesis boiled down to an 8-page nugget entitled Supporting Collaboration in Software Development with Discrete Event Streams. The reviewers were looking for more of a splash than I was able to provide, which is totally fair. Instead of a study examining how users work with an older (yet pervasive) idea like event-based awareness, the reviewers were clearly on the lookout for novel technology. All the same, I think there are some valuable findings in this work. New event-based collaboration tools like Google Wave, Salesforce Chatter, and Yammer have some neat ideas, but the unfortunate reality is that there is scant empirical understanding of how people use these types of tools to collaborate in a shared workspace. It seems that the new wave of tools are designed on the backs of social networking tools like Facebook and Twitter, not user research. I've tried to address some of the void in my paper and I hope it might generate some ideas in the minds of others for future work. I'm always open to chat about ideas in this area, so feel free to get in touch! Share:

Cheap data mining tricks

Jeremy Handcock — Tue, 11 May 2010 07:08:20 +0000

A few months ago, I made a jump back into industry after my brief but enjoyable stint in the world of big science research. My niftiest project at Institute for Systems Biology is described in a software article that I recently published along with Eric Deutsch and John Boyle in BMC Medical Genomics. mspecLINE: bridging knowledge of human disease with the proteome is about using a fast, simple data mining technique to find relationships between diseases and proteins and display them in a way that's useful to scientists who study proteins using mass spectrometry. The technique we used comes from a class of data mining algorithms that are really elegant: dead simple to program, easy to scale, blazing fast relative to other techniques, intuitive, and also highly effective. Given all these great qualities, I thought I'd spend some time talking about these algorithms in case they might be useful to you as well. They're derived from the Normalized Information Distance (NID), a "universal distance measure for objects of all kinds" (Vitányi et al.). NID is based on Kolmogorov complexity and is inherently incomputable, although it can be applied to real-world problems through some simple heuristics. The first of these heuristics is the Normalized Compression Distance (NCD). NCD allows you to measure the similarity between two objects (e.g., documents) using the compressed form of their string representation. The idea is very simple: objects that are similar compress similarly, while objects that are dissimilar do not. NCD has been successful in many applications: music clustering, protein classification, anomaly detection, and a number of other cool apps. The second of these heuristics---and the one we applied in our work on disease-protein association mining---is the Normalized Google Distance (NGD). NGD is a bit of a misnomer in that it doesn't necessarily have anything to do with Google, but that's what the authors first called it in the literature. NGD can be used as a measure of the semantic distance between two terms in some knowledge base. As with NCD, the basic idea is very simple: two terms that occur often in the same documents in a knowledge base are semantically related, while two terms that never occur together are unrelated. In our case, the knowledge base was MEDLINE, an index of 15 million (and growing) biomedical research articles. You could apply this technique to any knowledge base, however. Vitányi et al. describe how you could determine the semantic distance between two terms (e.g., 'horse' and 'saddle') using Google's search index. The great lure of NCD and NGD is that they're really fast. An obvious alternative technique is Latent Semantic Indexing (LSI), which simply doesn't scale for use with very large knowledge bases. NCD and NGD could be easily implemented as a MapReduce algorithm and thus they scale really easily. They're also parameter-free techniques, which is another great advantage over LSI---no need to worry about optimal dimensionality. Being general measures of similarity, NCD and NGD are pretty flexible and could be applied to many search, clustering, and classification problems. I suspect that they'll never be as good as a finely-tuned, domain-specific technique, although they're probably a good starting point and worth a look if you're shopping for something cheap and effective. Share:

Collaborating in virtual worlds

Jeremy Handcock — Fri, 19 Feb 2010 06:17:05 +0000

I recently watched a Frontline episode on avatars and virtual worlds that summarized some really interesting research going on at Standord's Virtual Human Interaction Lab (VHIL). This short clip is a good overview: The projects underway at VHIL show a lot of really exciting potential for collaboration in virtual worlds. They have a number of studies that suggest real-world social phenomena transfer into virtual worlds and that what happens in virtual worlds seems to impact real-world behaviour. In other words, users don't seem to finely distinguish their experiences in the virtual world from those in the real world. IBM is already using Second Life for remote collaboration in a big way, as are many other organizations. Meetings, conferences, you name it: it's all being done in virtual worlds. When I first heard about this form of collaboration, I thought it was a little gimmicky. It couldn't possibly replace informal, ad hoc communication in the workplace and it couldn't be any better than picking up the phone. Glancing over this research has changed my viewpoint, though. Virtual worlds are clearly more than isolated, independent realities. I'm actually pretty keen to try out a virtual world meeting. Are you on Second Life? If so, let's make up an excuse to have a meeting! (As soon as I figure out how to get more stylish clothes on my avatar.) Share:

google wave and collaborative tools

Jeremy Handcock — Tue, 02 Jun 2009 06:40:05 +0000

My initial reaction to the Google Wave demo at I/O was luke warm. C'mon, it's just a nifty UI over publish/subscribe messaging! Along with pubsub, the federated aspect of Wave---the seamless integration of users across multiple domains---is just standing on top of vanilla XMPP. From a cynical point of view, Wave is much more of an engineering feat than a shining piece of technological innovation. It's not all about recycled technology, though. The exciting part of Wave is the collaborative abstraction of XMPP that is centered around conversations rather than individual messages. As someone interested in developer tools, I see a lot of potential in Wave. The presenters at I/O actually had a demo of a Google Code issue tracker extension that allows you to create, follow, and respond to conversations in a bug report using Wave (see around the 1:02:30 mark): Very, very nice. After watching the demo, I realized that Wave-based developer tools could share a lot in common with the ideas I wrote about a few weeks ago---integrating artifact streams with work streams in a microblogging tool: In both scenarios, I think the proposed value comes from: 1. aggregating and persisting conversations in one place; and 2. enabling easy ad-hoc communication between participants. In any case, it looks like Google Wave may provide a nice framework to play with my ideas, so I'm excited to see how it shapes up. Share:

microblogging at work: workstreams and artifact streams

Jeremy Handcock — Sat, 02 May 2009 18:06:41 +0000

One of the reasons that Twitter has become so popular is that it is a successful ambient awareness tool. With each tweet from a friend or person of interest, you get a subliminal sense of what he/she is up to. Each tweet also represents an opportunity for ad hoc, informal communication between Twitterers: In Wired Magazine and then more recently in the New York Times Magazine, Clive Thompson discusses social proprioception, his term to describe the social 'sixth sense' that comes with microblogging. Proprioception is a really wonderful metaphor for describing ambient awareness and I think it can extend beyond social settings into the workplace as well. Yammer, for example, is your Twitter at work. I've never seen Yammer in action, but I think it's a really interesting idea. Just as you publish your lifestream on Twitter, you publish your workstream on Yammer---micro-level updates about what you're working on. Say you only have a once-a-day standup meeting with your team in the morning. Wouldn't it be nice to get more fine-grained details of what everyone is up to throughout the day? I could also see microblogging at work as a great way to close some of the distance in a distributed team. You get regular updates from team members afar, potentially a greater feeling of social connectedness, and opportunities for ad hoc communication. Microblogging at work is a fine idea by itself, but what about combining workstreams with artifact streams inside a microblog? The developer awareness tool that I created for my research was the artifact stream part---Aufait feeds you discrete tidbits about how your project artifacts are changing. I focused on source code, bug reports, documents, and automated builds, although the artifact could really be anything. If I put these artifact streams alongside a team's workstream, I get something like this: To me this seems like a natural fit. It augments all of the nice things about microblogging with additional awareness information that is really important for software developers. You can use a hash (ie. #sprocketdev) to publish a conversation to your team's channel and of course you can poke around other channels and people just like on Twitter. The screenshot above is Jaiku on AppEngine. Jaiku is Google's freshly open-sourced microblogging platform and it works pretty much like Twitter. Getting the aritfact streams in there is just a matter of publishing events (ie. version control commits, bug report updates) into Jaiku via its API. From there, you can get your hybrid workstream/artifact stream from Jaiku's web interface, RSS, or even using its API. I'm going to take this up as a project so I'll let you know how things come along. If you have any feedback or are interested in collaborating, feel free to get in touch! Share:

the ideal of authenticity: grad school?

Jeremy Handcock — Mon, 09 Mar 2009 18:23:08 +0000

I've been reading Michael Ignatieff's CBC Massey Lecture The Rights Revolution and I had to laugh a little when I came across this section regarding authenticity:

The central idea I absorbed then---chiefly, if not exclusively, from feminism---was that each of us has a right to choose the life we lead and that we must fight to exercise this right against all comers. This could be called the ideal of authenticity. In the name of this ideal, we all went off to find ourselves. This meant getting away from family, career, society, and going in search of the self's authentic impulses. Sometimes the results were laughable: the 1960s cult of authenticity produced dull conformity in no time. We all went in search of ourselves and ended up in graduate school.

Sounds about right! Ah well, at least grad school is more productive than bumming around a ski resort for two years. Beside that point, The Rights Revolution is a really excellent read about rights culture, civic and ethnic nationalism, and the balance between individual and collective rights in Canada. Ignatieff frames Canada as three distinct founding nations that coexist with a diverse and prominent immigrant population, and when you look at the country through that lens, it's pretty amazing that it still exists in one piece. It's even more amazing that the constitutional and legal framework has managed to satisfy---more or less---all of these different interests. There is still more to be done and although there have been bumps along the way and some very dark chapters, things could be much worse. Just read Ignatieff's Blood and Belonging. Share:

i am for hire!

Jeremy Handcock — Mon, 16 Feb 2009 06:25:24 +0000

Now that I have my master's degree wrapped up, I am officially for hire! I am primarily looking at opportunities in the Seattle area. If you are on the hunt for a creative, smart, and hard working Software Engineer, have a look over my resume and feel free to get in touch. I will be in the Seattle area in a few weeks if you want to grab a coffee and chat. Share:

visualizing mozilla lizardfeeder

Jeremy Handcock — Sun, 15 Feb 2009 23:51:06 +0000

I recently came across LizardFeeder, a "compilation of data feeds representing activity within the Mozilla community." I think it's a pretty nice tool. It shares some ideas in common with my awareness tool for developers, especially in that both tools aggregate discrete events in a software project. Obviously, the scale of our approaches is a little different; I was focused specifically on building a tool for a small team.

(LizardFeeder screenshot)

Dave Bottoms of Mozilla has published a call for designers to create "a visualization that attempts to demystify the collaborative process of making software, while celebrating [Mozilla] contributors." The goals are a little nebulous, but my take on it is they want to build on the ideas in LizardFeeder and come up with a visualization that is universally approachable, even to non-technical people. Software Engineering and Human Computer Interaction researchers have been trying to visualize software development communities for a few years. The archives of the Mining Software Repositories conference have some examples and one can also look to recent projects such as code_swarm. There have been some nice designs, for sure, and code_swarm is definitely fun to watch. At the same time, my pragmatic side is left looking for more. In practice, these are the kind of visualizations that you put up in the lobby of your building (in HiDef!) to impress visitors. Visual wow factor is a noble goal in itself---don't get me wrong---but I think we're still searching for ways to build meaningful tools in this area. The Mozilla proposal is especially welcome in that regard. I'm looking forward to seeing what comes out of it all, and given LizardFeeder's overlap with my research, some interesting ideas for awareness tools might emerge along the way. Share:

my awareness tool for developers: Aufait

Jeremy Handcock — Fri, 13 Feb 2009 21:27:20 +0000

I've put together a demo version of the tool (code name: Aufait) that I developed for my research. Feel free to try it out! If you haven't been following my work, I developed Aufait as an awareness tool for software developers. Specifically, it is designed to support awareness of peer activities and changes to shared artifacts (ie. code, bugs, documents) in a small team. From a design perspective, there are some things that I would change based on the results of my field study, but I'm interested in your off-the-cuff thoughts as well. Note that this demo is static in the sense that it won't automatically update with new data as Aufait normally would. The time interval selection features are also disabled. The demo is running on data from the Apache Derby project. I have no affiliation with Derby nor its developers; I just picked a small open source project with data that was relatively easy to obtain for the demo. If you're interested in seeing how developers use Aufait in the wild, check out my thesis paper. A quick summary of the UI follows.

The Event Timeline

The main view in Aufait is an event timeline. An event is any modification to a project artifact such as source code, a bug report, a project email, or a wiki document. The design has a lot in common with the SIMILE timeline. Each developer on the team---listed along with his/her photo on the right side of the screen---is associated with a colour. Events on the timeline are coloured according to the developer who initiated them. The idea here is to promote awareness of peer activity by clearly connecting events to individuals.

Revealing Event Content

As you can see, each event on the timeline has a short textual description. For example, an email from the project mailing list shows the subject line: You can click on nodes in the timeline to reveal more content for an event. Here you can see a bug report event that describes what the developer changed: Note that Aufait includes a link to the artifact that was modified (whenever possible). As shown above, a bug report event has a link to the bug report. The idea is that developers can get more information about a modified artifact or take responsive action by following the link.

The Details View

In addition to the event timeline, Aufait provides a more content-oriented view that is similar in design to an email or news reader. You can switch to this view by clicking on the 'Details' tab. It has a smaller version of the timeline that you can interact with in the same fashion as the larger one.

Filtering Events

You can filter events using three methods: by developer, by drag selection, and by keyword search. Clicking on any of the developer photos will filter the display to show events authored by him/her: You can also drag to select events in the timeline. The idea here is that you can select a bunch of events that might be of interest and then switch to the detail view to examine them further. Typing a keyword into the search box (and pressing enter) will filter events to show only those with content that matches. For all three filter types, you can remove a filter by activating a new filter or by clicking the 'clear filter' button.

Back-end Services

There are a few back-end services that support Aufait's UI by aggregating, storing, and publishing data from a software team's internal systems. The demo is running on static data files, but when it is running live, Aufait continuously updates itself with new events. Getting the data for the UI was a large chunk of the work in implementing this tool. I won't get into the details of it all here; have a look at my previous post Aggregating Project Events in the Wild if you're interested in how I did it. Share: