Category Archives: Experiences

A House Divided

12 Thursday Apr 2012

Posted by in Context, Experiences

≈ Leave a Comment

Amenities

With the recent The Hunger Games movie release came several fanboy and fangirl friends banding together to attend a performance on opening weekend. For good or ill, I ended up purchasing the tickets for the group. I made it to the box office days in advance and the transaction was happily mundane and successful.

The day before the show, one of the friends decided to bring a plus one along for the fun. However, she was very concerned that she couldn’t purchase a ticket online and didn’t want to cancel the date due to technical difficulties. She asked whether the theatre could add a ticket to my transaction and I agreed to look into it.

Give me something I can work with

Visiting the theatre’s website, I confirmed that the online information made the date seem doubtful, showing only 2 performances at that time with one of them in sold out status. When I tried the performance that wasn’t sold out, I noticed that the list of available tickets was rather short and only included reserved seating types. I hesitated to buy a single reserved ticket since the rest of us had general admission tickets.

(For those of you outside the U.S., we don’t have many movie theatres with reserved, or assigned, seating here. Almost every showing I’ve ever attended has been general admission. I only recently started patronizing a theatre that provided reserved seating – for a premium. For my American readers, one of the considerations for movie theatre software sold to international chains is the need to provide support for reserved seating as well as intermission.)

May the odds be ever in your favor

I tried calling the theatre to no avail, so I resolved to head over there after work to see about that ticket. When I spoke to the cashier, she explained that what had seemed like 2 separate performances online were really just 1 showtime.

Some programmer’s technical solution to the split house for a single performance came through to the web interface in a confusing form. In my experience, an auditorium, or house, had always been either general admission or reserved seating. And, although I tested movie theatre software for over 5 years, I had not encountered this feature request: splitting a single auditorium into 2 classes of ticketing.

Fortunately for my friend’s date, movie theatre software has a sold out threshold greater than zero, allowing for eventualities like broken seats, roof leaks, or other unexpected customer service issues. Knowing that, I confidently requested another ticket and easily obtained it. Granted, we ended up sitting in the front row craning our necks a bit as the pack of tween girls next to us excitedly discussed the movie play-by-play, but for once my testing savvy turned up a solution rather than a problem, averting a star-crossed lovers situation.

Testing Bliss

03 Tuesday Apr 2012

Posted by in #testchat, Context, Experiences, Experiments, Soft Skills, Techwell

≈ 5 Comments

It’s no secret: I adore testing software. It’s my weapon of choice, despite having happened upon it by chance many moons ago. (What other career transforms forgetfulness and clumsiness into strengths since they result in unexpected, non-happy path usage? Ultimately, I think it’s the variety that keeps me coming back for more on a daily basis.)

Given my feelings about testing, it came as no surprise to me that others would agree and rate this profession highly, whether on CareerBliss or elsewhere, as reported by Forbes. (I’ll also admit to having been a bit of an I/O Psych nerd back in the day, so this survey appeals to me in various ways.) I can’t seem to leave my curiosity at the door, so I had to go see for myself what questions were used as the basis of this data. (Yes, HR folks, that’s my story and I’m sticking to it.)

With categories like Company Culture, Work-Life Balance, The Place You Work, The People You Work For, The People You Work With, It’s Party Time!, Work Freedom, and Growth Opportunities, it almost felt like attending a company meeting at my current employer. (Did I mention we’re hiring a developer for my team?)

I was curious to see whether other testers had the same reaction to the questions used to generate the data that CareerBliss analyzed, so I culled out 5 questions of at-most-140-characters designed to find out.

  • Q1) Which people at work most affect your happiness: co-workers, boss, CEO?
  • Q2) How does the level of challenge in your work influence your feelings about your testing job?
  • Q3) Is there a job-provided perk/reward/tool that keeps you happy as a tester?
  • Q4) As a tester, do you have a good balance of freedom and growth?
  • Q5) How does the support at work make testing a great career?

Check out the storify-ed version of our #testchat on Twitter.

Not everyone has the same experience of software testing and my experience has certainly changed over time. I wanted to take a moment to consider the various aspects of software testing that the article identified:

  • requirements gathering – been there, done that both before and after implementation
  • documentation – frequent contributor, sometimes sole author
  • source code control – only for my automation code, but I didn’t set it up myself
  • code review – if you consider pairing with a developer on code during a sprint, then I’ve tried it and with some success
  • change management – not so much, though we did have a composition book in the testing lab to log all hardware changes to a system I worked on; sometimes it was more like a log of who I should hunt down to get the hardware back…
  • release management – the closest I get to this is being able to deploy to my cloud test environment and boy am I happy about that
  • actual testing of the software – bread and butter for me

I love having been involved in the entire software development process at various times during my career. (I’ve even prototyped some UI ideas, though I wouldn’t call that an area of strength or concentration. Glad to have those UXers on board these days!) I do feel that I’m an integral part of the job being done at the company. I am quite happy that my job involves frequently working with people.

However, I do take issue with this being presented as a positive aspect of the job:

software quality assurance engineers feel rewarded at work, as they are typically the last stop before software goes live

Doesn’t that smack of Gatekeepers to Quality to you? I don’t ever want to set up an adversarial relationship with my developers that says I need to defend the users against their disregard, and I don’t want to be involved only at the end as a last stop before kicking a product out the door. I know that happens at times but it’s not my preference. Positive personal interactions and preventative measures certainly contribute to my testing bliss.

Take the survey yourself at CareerBliss and let me know how your experience compares!

I’ll be analyzing the tagged responses from Twitter over on Techwell soon!

Here is some related reading that has come up in recent days:

Q3) Is there a job-provided perk/reward/tool that keeps you happy as a tester?

Jon Bach on tools for testing

Ajay Balamurugadas on tools for testing

Q5) How does the support at work make testing a great career?

Horizontal careers: “each of us will need to overcome our personal assumptions about moving up the career ladder, and think more about how we add value across.”

Scott Barber disagrees

Image source

Yo dawg, I herd you like ET

19 Monday Mar 2012

Posted by in Context, Experiences, Experiments, Hackathon, Retrospective, Testing Humor

≈ 1 Comment

I wrote out my Lab Days experience recently but didn’t get to bring you down the rabbit hole with me to experience the recursive testing goodness.

My project for Lab Days was an enhanced logging tool, but the logging is the heart of the matter, with users putting it through its paces much more stringently then the analysis functionality.

Since I usually do exploratory testing of applications at the day job and the time pressure of Lab Days left little room for formal test cases anyway, I decided to try out a new exploratory testing session logger: Rapid Reporter.

I didn’t have a lot of time to devote to learning Rapid Reporter, so I didn’t bother reading any documentation or preparing myself for how it worked, essentially exploratory testing my exploratory testing tool while exploratory testing my application under test.

It turns out this kind of recursive testing experience was just what I needed to liven things up a bit, all in the spirit of trying something new! I discovered that rapidly learning about a session logger while testing/learning a session logger, pulling log entries from an original session log, and reporting bugs via a session/chat room (HipChat) made for some perilous context-switching. More than once during the day, I had to stop what I was doing just to get my bearings.

I clearly enjoyed the experimentation because I decided to repeat the experience, though with a little less context-switching, when we upgraded our usual ET tool: Bonfire. The funniest thing about using Bonfire after working on my Lab Days project was that I realized there were tags available for log entries but the tagging indicators weren’t the same as our choice for our usability testing tool. I kept trying to use the tagging that I’d been testing all week and had to retrain myself, improving their documentation as a result of my questioning.

Still, seeing how another logging tool uses tags gave me some functionality to consider for our usability logger: how would users want to interact with tagged log entries? Clearly time to circle back with my UX designer to discuss some enhancements!

Image generated here

The status is not quo

09 Friday Mar 2012

Posted by in Context, Experiences, Experiments, Hackathon, Retrospective, Tester Merit Badges

≈ 3 Comments

Dr. Horrible http://drhorrible.com/

We tend to run “FedEx” with a fairly open format where you can do whatever you want as long as you can somehow relate it to our products.
– Atlassian

Last week, my company gave us an exciting opportunity: 5 days of work on a project related to our business.

Apparently, they’ve done something like this before, long before my time, so you’d have to ask some of the more tenured folks at Daxko about it.

I worked with the same folks who volunteered with me at the WebVisions Hackathon earlier this year and we kept in mind what my colleague Will said about that experience: “The short time box and no feature constraints necessitated a laser-sharp focus on one thing.”

So we noodled over several viable candidates and finally settled on building a better mousetrap – or, in this case, UsabLog.

A clarification on terminology from my UX colleague:
“Logging” in this context doesn’t mean “system logging of events.” It means human capture of what the user said, what the user did in the app (e.g., where user clicked), and any additional comments to provide context. The point of logging is to provide us with a record of what went down so we have an accurate recollection for later analysis.

I had the good fortune to be a user of the original UsabLog application over the course of many usability sessions as a session logger, so I was rather familiar with its strengths and weaknesses. I was able to contribute some bug reports and feature suggestions for consideration during our lunchtime planning discussions, but my Scrum team’s UX designer was our team’s sponsor. She compiled an experiment plan that identified our purpose and detailed the problems we considered in the pre-existing Usablog and the opportunities we had to satisfy those needs.

Our usability sessions up to this point involved an interview led by the facilitator (i.e. UX designer) and logged by another team member (e.g. me) via the free, open source, web application Usablog, which then exported logs to CSV for use in a program such as Excel and which we in turn manually fed into a mindmap program such as FreeMind. While this process did work for us, the export and manual copy-paste was rather tedious and laborious, or as she put it “it would directly contribute to user research process efficiencies.” We knew there could be a better way.

Goals of the experiment:

  • Rapidly capture rich user feedback during research interviews and usability tests through logging of user events and comments
  • Organize logs from multiple sessions into one study for ease of access and visibility
  • Use log entries to synthesize findings
  • Quickly jump to a spot in the session’s video by clicking on the associated log entry

In particular, we wanted these features:

  • Multi-session logging.
  • Log entries are timestamped when the logger starts typing for video synchronization.
  • Custom tags.
  • Multi-logger logging.
  • One tool for logging and post-session analysis.

We established a definition of done and recognized our dependencies since any impediments would have serious impact on our progress during the limited time of the competition.

I would love to tell you that we were entirely successful in meeting our goals and implementing all of our features, and then going on to take first prize in the competition. Alas, this was not to be. We only accomplished some of our goals and features and awesome projects from other teams placed above us.

However, the experiment was a roaring success in many ways:

  • I had first-hand experience with paired UX design under the tutelage of my UX designer colleague. She suggested that I man the helm and she steered me back on course when I went astray. I won’t claim that my first UI mockups were beauties, but the process and conversation certainly were.
  • I made my first commit to a Github open-source repository and thereby qualify for the Open Source Nerd Merit Badge (which happens to feature the Github mascot Octocat) which I had been hankering to do ever since I discovered its existence. Also, this was the first time I fixed a bug in the source code, so even though my changes were minor it was thrilling.
  • Exploratory testing based on Github commit notifications in the HipChat chat room we used for the team. Rather than pursuing session-based test management, I tried a looser structure based around the latest and greatest changes instead of setting charters and time-boxing exploration around the stated goal.
  • Real-time bug reporting of issues found during exploratory testing via HipChat messages and screenshot attachments was new and interesting. This is the lowest overhead asynchronous bug management approach I’ve tried and it was effective. Granted, we didn’t come out with a backlog of known issues written down somewhere, but we rectified the most critical problems before they had a chance to fester.
  • We didn’t let a little thing like heading home for the day stop us from collaborating remotely when we got back to business after hours. Being able to work at odd hours put some of my insomnia to good use. I also learned a bit about .NET and model/view/controller architecture, which turned out to be good preparation for the following – and last – day.
  • When one of our programmer teammates fell ill, I paired with our remaining developer to push on toward the goal. Although I think I spent more time asking questions to help think through the implementation than actually contributing code, it was a fruitful day, wrapping up an important feature a mere 30 minutes before the Big Reveal.
  • I used the resulting product to real-time log the presentations during the Big Reveal. Oh so meta, but also hopefully illustrative of the capabilities of the application for future use. If nothing else, it gave our sick friend a way to catch up on the excitement as he recovered over the weekend.
  • We accomplished only some of our goals and features but they were the most essential. Our product is usable as-is, though with some known bugs that do not inhibit happy-path use.
  • Why do they call it FedEx days? Because you have to ship! Our resulting application is ready for use – or enhancement if you’re feeling ambitious!
  • And last, but certainly not least, victory lunch! Nothing so sweet as celebrating effective teamwork.

Image source

Three heads and a tiara

08 Thursday Mar 2012

Posted by in Experiences, Soft Skills, Testing Humor

≈ 2 Comments

(Trying out a shorter and more casual post style, so let me know if you like it!)

Tiara

The other day I went into our office building’s bathroom down the hall. When I went to wash my hands, I noticed a white box sitting on the counter by the sink and saw that it contained a tiara, of all improbable things.

Since it was getting to be the end of the workday on a Friday, I thought someone surely would need this for some social event over the weekend. Having only one female coworker – and she’s not a pageant contestant – I hoped it would be a lady in one of the neighboring offices and went knocking on doors. Unfortunately, the two people who answered were males who looked at me strangely.

Fortunately, one of them, a man whose first language was likely Russian, allowed for the possibility that something might just be lost in translation and took me to his HR colleague. When I explained the same thing to her, she asked the other woman in their break room about it and then followed me back down the hall to see for herself. I appreciated that she allowed for the possibility that I might be reporting something factual.

Having reported the strange observation, I left the situation in her capable hands and hoped that she found a resolution since the tiara was not still on the counter when I returned on Monday morning.

While I would like to say that I didn’t recognize the you-so-crazy looks these level-headed people were giving me, I have had enough odd bug reports to discuss with developers that I know well the look that developers favor when I approach them with a bug report: I clearly had sprouted two more heads. That had to explain the strange things I was spouting. And I certainly do appreciate the ones who willingly suspend their disbelief – or indulge their curiosity – long enough to investigate my strange claims.

Of all the people who could have found the tiara and reported its presence for claiming, at least I knew from day-to-day work experience what it was like to have three heads.

Image source

The sultry sound of testing

07 Wednesday Mar 2012

Posted by in Experiences, Experiments, Podcast, Publications, TWiST

≈ Leave a Comment

Mike Wazowski with mic

Follow the sultry sound of my voice – Mike Wazowski

… for some great testing conversation!

I have made an appearance on the TWiST podcast, which you can stream online or download after registering for the Software Test Professionals’ website.

I discuss differentiating yourself from other testing job applicants with Matt Heusser, Michael Larsen, Wade Wachs, and Ben Yaroch.

For your enjoyment, here are the direct links (once you log in):

Getting Hired as a Tester, Part 1

Getting Hired as a Tester, Part 2

Getting Hired as a Tester, Part 3

Image source

Of Paths and Cycles

06 Tuesday Mar 2012

Posted by in Context, Experiences, Experiments

≈ 1 Comment

virtual ride

I joined the YMCA last summer and have been trying out different ways of being more active. (My active lifestyle resolution wasn’t just aimed at professional development.) Recently, I came to the conclusion that having more structure seems to work for me in learning testing and so might be helpful around my fitness progress as well.

To that end, I made a Coach Approach appointment and met with my coach to learn how the program works. She and I talked for nearly an hour about what my goals would be (set up along the SMART guidelines) and how I could work my current interests into a structured plan. She suggested that I try out some of the exercise equipment that they have and talked over different ways to cope with the boredom that creeps in and deters people from continuing their progress.

One of the machines she suggested for me was an exercise bike with a computerized screen. Today, I decided to give it a go. I dressed out, filled my water bottle, and found an available bike. Since I’m interested in taking up cycling, this seems like a nice way to build up my stamina until the weather warms a bit. I plugged in my headphones and turned my attention to the log in menu on the screen. Realizing this was not a touchscreen application, I observed that there were a variety of buttons to interact with the system.

Since I wasn’t sure I was going to stick with this workout method, I selected guest log in just to try out the system. I selected a beginner course, put my hands on the handlebars, and began pumping the pedals. I immediately noticed that the handlebars and foot pedals provided information to the system as did the buttons on the panel below the screen. I found some good music to keep my ears busy and started observing the software.

Happy Path

Normally, I bemoan working out on exercise equipment as “the race to nowhere,” finding myself immediately unsatisfied with the experience of running in place and staring straight ahead in a gym environment. However, having an application to test while burning calories certainly was a welcome change. I don’t think my coach realized just how easily I could avoid boredom with some software to occupy my attention!

I tried a couple of different paths, or courses as they called them, each with a different scenery motif and points of visual interest. I was amused to discover that steering with the handlebars was entirely unnecessary since the program forced me to stay on the path and stopped displaying any virtual cyclist I ran down. At first, I was a bit disconcerted when virtual cyclists would pass through me from behind and appear to pop out of my chest. Backpedaling served only as an indication that I wasn’t moving forward, as though I had stopped pedaling completely, and so didn’t help me to put more space between my virtual handlebars and the virtual chest-burster cyclists. I thought one of these virtual cyclists represented the “pacer” that appeared on my progress bar, but I eventually figured out that the pacer didn’t have a manifestation on the course, only in the ride-in-progress statistics reporting areas.

Push it real good

However, I noticed some issues during my first ride:

  • Objects in the scenery were drawn with perspective and would update with a jerk when they entered the middle of the field of vision.
  • A bush on the edge of the path happened to overhang the path enough that my virtual handlebars passed through it.
  • A virtual cyclist was stranded on the side of the path oriented sideways rather than in the direction of travel, as all of the other virtual cyclists were.
  • Another representation of a rider (ghost?*) appeared on the path oriented sideways but didn’t seem to be animated.
  • After I completed the ride, the screen showed my ride’s statistics as a modal dialogue, but I could see the heart rate monitor, RPM, speed, and ride timer were still updating on the disabled screen.
  • One of the post-ride statistics was the local facility’s leaderboard for that course and although my time ranked higher than the last person on the board my time was not displayed.

*I wasn’t clear on what the system meant by a ghost rider who could appear on the course, so this may have been correct software behavior.

Integration, schmintegration

After a trip home and a well-earned shower, I settled in with my laptop to check out the website that interfaces with the on-site system. The site proclaimed that their system engages your mind and I certainly found that to be true, perhaps in a way they hadn’t anticipated.

Although I had created an account through the log in screen of the exercise bike, the website prompted me to complete my profile online before I could access the information. Though I usually think of this as an annoyance, few required fields and a humorous selection of security questions made it a pleasant experience.

The News informed me that I could share my ghost through Facebook or Twitter, though I still had little idea of how that would be used, having not seen it in action. I declined to use the social media hook, deferring it until I have an opportunity for more investigation. I was happy to see that my first workout records and awards were available online, though I didn’t “post a ghost” through email or printable QR code. When I found the Ghost Selection options, I could see that a ghost was something like a pacer but more personalized or specific.

I noticed several issues online:

  • I was hopeful that the online system would show my ranking since the on-site exercise bike had not, but both the global leader boards and boards for my fitness facility omitted me.
  • The first attempt to view leaderboards for my fitness facility showed data from a location in some other state although subsequent refresh seemed to correct the problem.
  • I also encountered the server’s technical difficulties page.
  • Some header graphics failed to display, though sometimes page refresh corrected this.
  • Leaderboard page breadcrumbs did not always correspond to the displayed page (e.g. inaccurate, current page omitted).
  • Firebug showed me at least one typo in the Javascript that caused an error and one page’s source included comments in the code, which I have read is discouraged.
Game on

Although I was happy to know that my workout data was preserved and available online in some form, the leaderboards could use some work. While the software product team may not have been concerned with real-time updates to leaderboards, as a first-time user I really wanted to see how my performance stacked up against the more seasoned players, which is an important part of the gamification angle that this product leverages to defeat boredom and keep users involved in exercise. I’ll certainly try this system again and hope that I can ride out both the bugs and the boredom.

Image source

Quality Stands Out

17 Monday Oct 2011

Posted by in Experiences

≈ 1 Comment

My fishie

One of the nice things about going to a science-fiction convention is that you blend into the crowd in your obscure-reference costume. You can go about your nerdy business without anyone stopping you every 5 seconds to ask for a photo op. At Dragon*Con this year, I spent much more time wandering the halls to take in the experience than following the programmed tracks of activities. One advantage of this was the premium people-watching. Some people who are passionate costumers never appear at any of the costuming panels or track sessions. Their costumes might not even fit through the doors of the track’s room!

When you’re wandering around strangely attired in public with your 40,000 closest friends, you will inevitably encounter someone else costumed as the same character. There is a moment of recognition that offers the chance for geeky high-fives and kudos for sharing your interest. The one problem with meeting geeks who get the references is they know their subject matter deeply and can spot inaccuracies in your garb. If you are attempting to replicate an iconic image of a character, they’ll spot deviations immediately. This reminds me of something Mike Lee said in his Making Apps That Don’t Suck talk: “There’s a good chance what you think is wrong with the product, no one else notices or cares about. … Your users are probably not nerds, unless you make software for people who make software and then only God can help you.” When faced with fanboys, you cannot slack off.

On the flip side, when your costume is high quality, people may not care about recognizing the nerdy reference and stop you every 5 seconds just to admire your workmanship. The design is so well-executed or intricate that they don’t care about the subject matter and just want to stare.

“If you want to be remembered, be memorable. If you want to stand out in the crowd, it helps to come up with something other than just looking like everybody else.” — Mike Lee

The real geek gold is in a high-quality obscure-reference ensemble that gets you both kinds of attention. [And if you can actually work popular culture into this mix, you’re golden.]

Big Fish in a Big Pond

I attacked the costuming problem in the same way I attack my testing: with the goal of having the best execution. I know that the users of my software are the nerds of their genre (niche market), much more intricately familiar with the nuances of their business than I. I know that missteps in the vital functions will not go unnoticed or unreported. The software must satisfy the production quality its highly specialized market demands. For niche markets, “the final product quality … is associated more with the specific needs that the product is aimed at satisfy.” (Wikipedia) I studied my source material, in this case Neil Gaiman’s Sandman graphic novels, and noted all the little tell-tale character attributes that must be preserved to be faithful to the design, or in this case the many designs.

However, I know that a faithful reproduction is not what I want to deliver. I want an unexpected element in my ensemble that would transform a good idea to great. I was tempted to purchase a fish balloon and carry that around the con, but I was much happier when I discovered a navigable fish blimp as the perfect accessory for my Delirium. Similarly, knowing what people (human oracles) say they want in their software is only the first step in satisfying their needs, so we cannot limit our testing to only the scenarios they state they want to execute but instead we must explore beyond the known. We can be advance scouts reporting back the plausibility of satisfying those unstated needs. “The essential value of any test case lies in its ability to provide information (i.e. to reduce uncertainty).” – Cem Kaner & James Bach

Then we can take a shot at that surprise and delight that Mike Lee advocates and really wow the crowd.