Monthly Archives: March 2012

Yo dawg, I herd you like ET

19 Monday Mar 2012

Posted by in Context, Experiences, Experiments, Hackathon, Retrospective, Testing Humor

≈ 1 Comment

I wrote out my Lab Days experience recently but didn’t get to bring you down the rabbit hole with me to experience the recursive testing goodness.

My project for Lab Days was an enhanced logging tool, but the logging is the heart of the matter, with users putting it through its paces much more stringently then the analysis functionality.

Since I usually do exploratory testing of applications at the day job and the time pressure of Lab Days left little room for formal test cases anyway, I decided to try out a new exploratory testing session logger: Rapid Reporter.

I didn’t have a lot of time to devote to learning Rapid Reporter, so I didn’t bother reading any documentation or preparing myself for how it worked, essentially exploratory testing my exploratory testing tool while exploratory testing my application under test.

It turns out this kind of recursive testing experience was just what I needed to liven things up a bit, all in the spirit of trying something new! I discovered that rapidly learning about a session logger while testing/learning a session logger, pulling log entries from an original session log, and reporting bugs via a session/chat room (HipChat) made for some perilous context-switching. More than once during the day, I had to stop what I was doing just to get my bearings.

I clearly enjoyed the experimentation because I decided to repeat the experience, though with a little less context-switching, when we upgraded our usual ET tool: Bonfire. The funniest thing about using Bonfire after working on my Lab Days project was that I realized there were tags available for log entries but the tagging indicators weren’t the same as our choice for our usability testing tool. I kept trying to use the tagging that I’d been testing all week and had to retrain myself, improving their documentation as a result of my questioning.

Still, seeing how another logging tool uses tags gave me some functionality to consider for our usability logger: how would users want to interact with tagged log entries? Clearly time to circle back with my UX designer to discuss some enhancements!

Image generated here

The status is not quo

09 Friday Mar 2012

Posted by in Context, Experiences, Experiments, Hackathon, Retrospective, Tester Merit Badges

≈ 3 Comments

Dr. Horrible http://drhorrible.com/

We tend to run “FedEx” with a fairly open format where you can do whatever you want as long as you can somehow relate it to our products.
– Atlassian

Last week, my company gave us an exciting opportunity: 5 days of work on a project related to our business.

Apparently, they’ve done something like this before, long before my time, so you’d have to ask some of the more tenured folks at Daxko about it.

I worked with the same folks who volunteered with me at the WebVisions Hackathon earlier this year and we kept in mind what my colleague Will said about that experience: “The short time box and no feature constraints necessitated a laser-sharp focus on one thing.”

So we noodled over several viable candidates and finally settled on building a better mousetrap – or, in this case, UsabLog.

A clarification on terminology from my UX colleague:
“Logging” in this context doesn’t mean “system logging of events.” It means human capture of what the user said, what the user did in the app (e.g., where user clicked), and any additional comments to provide context. The point of logging is to provide us with a record of what went down so we have an accurate recollection for later analysis.

I had the good fortune to be a user of the original UsabLog application over the course of many usability sessions as a session logger, so I was rather familiar with its strengths and weaknesses. I was able to contribute some bug reports and feature suggestions for consideration during our lunchtime planning discussions, but my Scrum team’s UX designer was our team’s sponsor. She compiled an experiment plan that identified our purpose and detailed the problems we considered in the pre-existing Usablog and the opportunities we had to satisfy those needs.

Our usability sessions up to this point involved an interview led by the facilitator (i.e. UX designer) and logged by another team member (e.g. me) via the free, open source, web application Usablog, which then exported logs to CSV for use in a program such as Excel and which we in turn manually fed into a mindmap program such as FreeMind. While this process did work for us, the export and manual copy-paste was rather tedious and laborious, or as she put it “it would directly contribute to user research process efficiencies.” We knew there could be a better way.

Goals of the experiment:

  • Rapidly capture rich user feedback during research interviews and usability tests through logging of user events and comments
  • Organize logs from multiple sessions into one study for ease of access and visibility
  • Use log entries to synthesize findings
  • Quickly jump to a spot in the session’s video by clicking on the associated log entry

In particular, we wanted these features:

  • Multi-session logging.
  • Log entries are timestamped when the logger starts typing for video synchronization.
  • Custom tags.
  • Multi-logger logging.
  • One tool for logging and post-session analysis.

We established a definition of done and recognized our dependencies since any impediments would have serious impact on our progress during the limited time of the competition.

I would love to tell you that we were entirely successful in meeting our goals and implementing all of our features, and then going on to take first prize in the competition. Alas, this was not to be. We only accomplished some of our goals and features and awesome projects from other teams placed above us.

However, the experiment was a roaring success in many ways:

  • I had first-hand experience with paired UX design under the tutelage of my UX designer colleague. She suggested that I man the helm and she steered me back on course when I went astray. I won’t claim that my first UI mockups were beauties, but the process and conversation certainly were.
  • I made my first commit to a Github open-source repository and thereby qualify for the Open Source Nerd Merit Badge (which happens to feature the Github mascot Octocat) which I had been hankering to do ever since I discovered its existence. Also, this was the first time I fixed a bug in the source code, so even though my changes were minor it was thrilling.
  • Exploratory testing based on Github commit notifications in the HipChat chat room we used for the team. Rather than pursuing session-based test management, I tried a looser structure based around the latest and greatest changes instead of setting charters and time-boxing exploration around the stated goal.
  • Real-time bug reporting of issues found during exploratory testing via HipChat messages and screenshot attachments was new and interesting. This is the lowest overhead asynchronous bug management approach I’ve tried and it was effective. Granted, we didn’t come out with a backlog of known issues written down somewhere, but we rectified the most critical problems before they had a chance to fester.
  • We didn’t let a little thing like heading home for the day stop us from collaborating remotely when we got back to business after hours. Being able to work at odd hours put some of my insomnia to good use. I also learned a bit about .NET and model/view/controller architecture, which turned out to be good preparation for the following – and last – day.
  • When one of our programmer teammates fell ill, I paired with our remaining developer to push on toward the goal. Although I think I spent more time asking questions to help think through the implementation than actually contributing code, it was a fruitful day, wrapping up an important feature a mere 30 minutes before the Big Reveal.
  • I used the resulting product to real-time log the presentations during the Big Reveal. Oh so meta, but also hopefully illustrative of the capabilities of the application for future use. If nothing else, it gave our sick friend a way to catch up on the excitement as he recovered over the weekend.
  • We accomplished only some of our goals and features but they were the most essential. Our product is usable as-is, though with some known bugs that do not inhibit happy-path use.
  • Why do they call it FedEx days? Because you have to ship! Our resulting application is ready for use – or enhancement if you’re feeling ambitious!
  • And last, but certainly not least, victory lunch! Nothing so sweet as celebrating effective teamwork.

Image source

February results for Tester Merit Badge

09 Friday Mar 2012

Posted by in Approaches, Experiments, Techwell, Tester Merit Badges

≈ Leave a Comment

Tester Merit Badge - Explorer

My February was pretty crazy, so I didn’t pursue all of the parts of the Explorer Tester Merit Badge this month. I skipped these parts due to time constraints:

3. How Long and How Far
4. Walk the Distance
8. Bus and Train Maps

No sweat. I’ll certainly be coming back to this one since exploratory testing (ET) is meat-and-potatoes testing for me.

Read the rest on the Techwell blog and tell me about your results!

Three heads and a tiara

08 Thursday Mar 2012

Posted by in Experiences, Soft Skills, Testing Humor

≈ 2 Comments

(Trying out a shorter and more casual post style, so let me know if you like it!)

Tiara

The other day I went into our office building’s bathroom down the hall. When I went to wash my hands, I noticed a white box sitting on the counter by the sink and saw that it contained a tiara, of all improbable things.

Since it was getting to be the end of the workday on a Friday, I thought someone surely would need this for some social event over the weekend. Having only one female coworker – and she’s not a pageant contestant – I hoped it would be a lady in one of the neighboring offices and went knocking on doors. Unfortunately, the two people who answered were males who looked at me strangely.

Fortunately, one of them, a man whose first language was likely Russian, allowed for the possibility that something might just be lost in translation and took me to his HR colleague. When I explained the same thing to her, she asked the other woman in their break room about it and then followed me back down the hall to see for herself. I appreciated that she allowed for the possibility that I might be reporting something factual.

Having reported the strange observation, I left the situation in her capable hands and hoped that she found a resolution since the tiara was not still on the counter when I returned on Monday morning.

While I would like to say that I didn’t recognize the you-so-crazy looks these level-headed people were giving me, I have had enough odd bug reports to discuss with developers that I know well the look that developers favor when I approach them with a bug report: I clearly had sprouted two more heads. That had to explain the strange things I was spouting. And I certainly do appreciate the ones who willingly suspend their disbelief – or indulge their curiosity – long enough to investigate my strange claims.

Of all the people who could have found the tiara and reported its presence for claiming, at least I knew from day-to-day work experience what it was like to have three heads.

Image source

The sultry sound of testing

07 Wednesday Mar 2012

Posted by in Experiences, Experiments, Podcast, Publications, TWiST

≈ Leave a Comment

Mike Wazowski with mic

Follow the sultry sound of my voice – Mike Wazowski

… for some great testing conversation!

I have made an appearance on the TWiST podcast, which you can stream online or download after registering for the Software Test Professionals’ website.

I discuss differentiating yourself from other testing job applicants with Matt Heusser, Michael Larsen, Wade Wachs, and Ben Yaroch.

For your enjoyment, here are the direct links (once you log in):

Getting Hired as a Tester, Part 1

Getting Hired as a Tester, Part 2

Getting Hired as a Tester, Part 3

Image source

Of Paths and Cycles

06 Tuesday Mar 2012

Posted by in Context, Experiences, Experiments

≈ 1 Comment

virtual ride

I joined the YMCA last summer and have been trying out different ways of being more active. (My active lifestyle resolution wasn’t just aimed at professional development.) Recently, I came to the conclusion that having more structure seems to work for me in learning testing and so might be helpful around my fitness progress as well.

To that end, I made a Coach Approach appointment and met with my coach to learn how the program works. She and I talked for nearly an hour about what my goals would be (set up along the SMART guidelines) and how I could work my current interests into a structured plan. She suggested that I try out some of the exercise equipment that they have and talked over different ways to cope with the boredom that creeps in and deters people from continuing their progress.

One of the machines she suggested for me was an exercise bike with a computerized screen. Today, I decided to give it a go. I dressed out, filled my water bottle, and found an available bike. Since I’m interested in taking up cycling, this seems like a nice way to build up my stamina until the weather warms a bit. I plugged in my headphones and turned my attention to the log in menu on the screen. Realizing this was not a touchscreen application, I observed that there were a variety of buttons to interact with the system.

Since I wasn’t sure I was going to stick with this workout method, I selected guest log in just to try out the system. I selected a beginner course, put my hands on the handlebars, and began pumping the pedals. I immediately noticed that the handlebars and foot pedals provided information to the system as did the buttons on the panel below the screen. I found some good music to keep my ears busy and started observing the software.

Happy Path

Normally, I bemoan working out on exercise equipment as “the race to nowhere,” finding myself immediately unsatisfied with the experience of running in place and staring straight ahead in a gym environment. However, having an application to test while burning calories certainly was a welcome change. I don’t think my coach realized just how easily I could avoid boredom with some software to occupy my attention!

I tried a couple of different paths, or courses as they called them, each with a different scenery motif and points of visual interest. I was amused to discover that steering with the handlebars was entirely unnecessary since the program forced me to stay on the path and stopped displaying any virtual cyclist I ran down. At first, I was a bit disconcerted when virtual cyclists would pass through me from behind and appear to pop out of my chest. Backpedaling served only as an indication that I wasn’t moving forward, as though I had stopped pedaling completely, and so didn’t help me to put more space between my virtual handlebars and the virtual chest-burster cyclists. I thought one of these virtual cyclists represented the “pacer” that appeared on my progress bar, but I eventually figured out that the pacer didn’t have a manifestation on the course, only in the ride-in-progress statistics reporting areas.

Push it real good

However, I noticed some issues during my first ride:

  • Objects in the scenery were drawn with perspective and would update with a jerk when they entered the middle of the field of vision.
  • A bush on the edge of the path happened to overhang the path enough that my virtual handlebars passed through it.
  • A virtual cyclist was stranded on the side of the path oriented sideways rather than in the direction of travel, as all of the other virtual cyclists were.
  • Another representation of a rider (ghost?*) appeared on the path oriented sideways but didn’t seem to be animated.
  • After I completed the ride, the screen showed my ride’s statistics as a modal dialogue, but I could see the heart rate monitor, RPM, speed, and ride timer were still updating on the disabled screen.
  • One of the post-ride statistics was the local facility’s leaderboard for that course and although my time ranked higher than the last person on the board my time was not displayed.

*I wasn’t clear on what the system meant by a ghost rider who could appear on the course, so this may have been correct software behavior.

Integration, schmintegration

After a trip home and a well-earned shower, I settled in with my laptop to check out the website that interfaces with the on-site system. The site proclaimed that their system engages your mind and I certainly found that to be true, perhaps in a way they hadn’t anticipated.

Although I had created an account through the log in screen of the exercise bike, the website prompted me to complete my profile online before I could access the information. Though I usually think of this as an annoyance, few required fields and a humorous selection of security questions made it a pleasant experience.

The News informed me that I could share my ghost through Facebook or Twitter, though I still had little idea of how that would be used, having not seen it in action. I declined to use the social media hook, deferring it until I have an opportunity for more investigation. I was happy to see that my first workout records and awards were available online, though I didn’t “post a ghost” through email or printable QR code. When I found the Ghost Selection options, I could see that a ghost was something like a pacer but more personalized or specific.

I noticed several issues online:

  • I was hopeful that the online system would show my ranking since the on-site exercise bike had not, but both the global leader boards and boards for my fitness facility omitted me.
  • The first attempt to view leaderboards for my fitness facility showed data from a location in some other state although subsequent refresh seemed to correct the problem.
  • I also encountered the server’s technical difficulties page.
  • Some header graphics failed to display, though sometimes page refresh corrected this.
  • Leaderboard page breadcrumbs did not always correspond to the displayed page (e.g. inaccurate, current page omitted).
  • Firebug showed me at least one typo in the Javascript that caused an error and one page’s source included comments in the code, which I have read is discouraged.
Game on

Although I was happy to know that my workout data was preserved and available online in some form, the leaderboards could use some work. While the software product team may not have been concerned with real-time updates to leaderboards, as a first-time user I really wanted to see how my performance stacked up against the more seasoned players, which is an important part of the gamification angle that this product leverages to defeat boredom and keep users involved in exercise. I’ll certainly try this system again and hope that I can ride out both the bugs and the boredom.

Image source