Here are the slides from my full length Big Visible Testing talk, presented at Agile2013 in Nashville, TN on August 6, 2013.
My experience report paper will be published by the Agile Alliance under the conferences archive as part of the proceedings of the Agile2013 conference. You can also download the PDF here: ClaireMoss-BigVisibleTesting-Agile2013
During the year and a half of experimentation that included the big visible charts that are included in this slide deck, I read over the following resources, only some of which would easily fit into the IEEE format. This is the full bibliography of my research, as far as I have been able to track down my sources. (At the time, I wasn’t expecting to cite them for anyone else, so I probably didn’t bookmark everything I read.) I hope the following links will prove helpful to you in developing your own big visible charts. Let me know how it goes! And please share any sources that you find helpful. I’m always looking for new inspiration.
My first dev team was an XP dev team that dogfooded our own digital signage product to display success/failure for the thousands of unit tests in the suite (i.e. single flag for whole suite red/green).
Other eXtreme Programming big visible charts
Heatmaps (from code analysis)
Paul Holland’s Exploratory Testing charter Kanban board
Lanette Creamer and Matt Barcomb gave a presentation that included ET charter management in big visible charts; podcast preview of their session
Visualizing above the product team
Including faces of people/profiles in the big visible charts
I can’t remember whether I’d see this one at the time or not… it might have been something I discovered after my time on the team mentioned in my presentation: Visual management for agile teams
Although the above resources were all I knew at the time I began my experiments, as I prepared my IEEE paper for the Agile2013 conference proceedings, I was tracking down my sources and came across these other relevant pages & posts that have given me some great ideas of things to try next!
Other cool extreme feedback devices:
After some discussion in my session about suggesting solutions for distributed teams, I was looking for some digital implementations of big visible charts, but I don’t know how these would work out for you.
With the First Day of School quickly approaching, it’s time for:
What I Did For My Summer Vacation – Part 1
I get really excited about hanging out with people, especially friends, especially a combination of new and old friends. So it was with great happiness that I set about organizing a geek gal weekend.
Our first conversations centered around budget (fixed), deadline (fixed), and features (flex). We started talking over the various activities that different destinations could provide to entertain us. Then, I paused the conversation to bring the focus back to value: when we looked back on this weekend, how did we want to remember it? how would we feel about the way we spent that time together? would features even feature in these stories we would tell? Instead of features, we realized that functions (what the product was going to accomplish) and attributes (characteristics desired by the clients) mattered more. (Why yes I was reading Weinberg this morning. How could you tell?)
We wanted to relish the simplicity of being together, whether wearing goofy vintage clothes (gold lamé for the win!), cooking our own meals together, telling silly stories, or engaging in feminine activities with a geek spin (magnetic nail polish was not as simple as expected) that would be low-key and more about togetherness than busyness. We wanted to craft something lasting (collaborative artwork – packing the craft supplies was a must not a want!) and reinforce durable friendships that appreciated our differences.
With this clear focus in mind, suddenly the locale was much less important than inclusivity to maximize togetherness.
Planned For Sand
So we made an informal backlog of tasks to tackle researching options (beach vs mountains), reviewing results, and prioritizing options (beach!) before presenting our findings to the group for dot voting. (Typical agilists, I’m tellin’ ya.) Fortunately, we found a viable approach and went forward with making arrangements to execute this solution (road trip!), adjusting as we went to accommodate discovered needs.
How did it all turn out? Stay tuned for scintillating tales of laughter and danger in What I Did On My Summer Vacation – Part 2!
As Ash, I want to defend myself against deadites (undead creatures) so that I can retrieve pages from the Book of the Dead.
– defend from a distance
– defend at close range
Two distinct pieces of value, huh? Clearly, we need a story split here!
As Ash, I want to repurpose the stump of my right arm into a fearsome weapon so that I can defend myself against undead creatures at close range.
– well supported, weight-balanced
– hands-free operation
– use available materials
– holds up under stress
– close-range fighting
– leather harness
– chainsaw mounted on handcuff
– chainsaw pull operated via bracket on harness
As Ash, I want another weapon for my left hand so that I can defend myself against undead creatures at a distance.
– easy storage
– one-handed operation
– uses available materials
– distance fighting
– sawed-off shotgun
– uses right-hand-mounted chainsaw to saw off shotgun (story dependency or taking advantage of existing features?)
– convenient back holster
Bonus feature/discovered value:
– clever shorthand terminology: “Boomstick”
By now, the distinction between testable user story acceptance criteria focused on user value and the resulting technical implementation should be painfully clear. Groovy?
For more than a year now, I’ve been shopping around for a hairdresser who could provide the ideal haircut. The first two attempts were incomplete, poor likenesses of the beauty I had in mind. I had a clear vision of the intended result, but I lacked the vocabulary to communicate that vision to professionals who could implement the solution. I had fallen victim to one of the classic blunders! (No, it’s not never go in against a Sicilian when death is on the line.) I knew it when I saw it but I couldn’t articulate it. So frustrating!
So I turned to the internet for solace. I perused numerous galleries of smiling women with hair of various lengths, shapes, and hues. In order to find the images I needed for my initial point-and-grunt interface – a printout of images pasted into a document for my first two appointments – I had to first identify the search terms that would produce optimal matches. I quickly cycled through searches from the generic “short haircuts” to the slightly more specific “bob hairstyles,” feeding my learning back into my process. As I progressed toward an exemplar of the captivating coiffure, I began to build a jargon file – and a Pinterest board.
Natural language is ambiguous and context dependent, so any requirements described in natural language are rarely complete. … This is especially problematic when something seems obvious but we need domain expertise or knowledge of a particular jargon to understand it fully. – Gojko Adzic
- short haircuts
- cropped hair
- bob hairstyles
- asymmetrical bob
- graduated bob
- stacked bob
- angled bob
- long bob
- layered bob
- inverted bob
- severely angled stacked bob
I don’t know whether those terms produce crystal clear images in your head, but I could see that these terms had a wide range of interpretations even among fashionistas.
I have heard it said that social media is a time suck, with Pinterest often held up as the mother of time sucks. However, I disagree. For my purposes, Pinterest was a fabulous tool for collecting all these visual bookmarks in one place, building a virtual gallery of hairstyle models as a communication tool.
When I booked my appointment online, I had included only a link to the first image I had found that was a rough approximation, leading her to ask upon my arrival whether I was the one who had sent her the Rihanna photo. (Of course not! That was Nicki Minaj!) That early draft of my request submitted in advance had given her time to mull over the idea.
I am pleased to say the result was exactly what I had hoped for and I will be visiting the Madam LV Salon again. Ultimately, being able to show this gallery to my hair craftswoman convinced her that my request was not a lark, that I had done my homework, and that my Pinterest board was in fact a specification by example.
So last year I joined the YMCA. My employer works in this space and they supplement our memberships … on the condition that we attend with a minimal frequency. Nothing to understand your customers quite like becoming their customer! However, working out isn’t really my thing. The “race to nowhere” has no appeal for me. But I went anyway, determined to learn something. Despite my stubbornness on that point, the inertia of years of study was hard to overcome. I needed backup.
Joining the coach approach program was explicitly about wanting to make improvements. The Y coaches promote “adopting healthy habits and changing the way [the participants] live their daily lives.” I knew I wanted to make a change, but I also knew that I didn’t want it badly enough to go it alone quite yet. Having never had a personal coach, I wasn’t quite sure what to expect. What I encountered resonated with my recent experience learning about the role of ScrumMaster.
In particular the sprint activity of retrospective is “an opportunity to learn how to improve.” Defining success in this particular context was the first step. My ScrumMaster watched the process and guided it, making it okay to talk about uncomfortable topics, but it was up to me to do the work. The first big step was being able to establish a safe environment to talk with a more experienced and professional person about a potentially sensitive topic.
In the case of my workout routine, this was my minimal compliance rather than wholehearted adoption of lifestyle change. My Coach Approach coach helped me to develop a vision for the future that would be better than the past. We focused on setting goals while recognizing that the plan had to fit into my work/life balance with the loose structure of frequent check-ins rather than plugging my height, weight, and weight loss goal into a one-size-fits-all spreadsheet.
I was surprised to find that discussion about my health could be fun when my counselor was so friendly and supportive. I would have expected an intervention to be really uncomfortable. Retros can be that way sometimes. But they can also be a welcome change of pace. Roughly every 2 weeks – after we catch up on socializing and the excitement we’ve had since our last chat – my coach and I looked at the artifacts of my progress, paying attention to the time line of events going on in the background and how that influenced the results. Keeping this cadence allowed us to build a healthy relationship that encouraged risk-taking and speaking from the heart. So when my coach suggested adding a weights routine to my cardio, I felt fine with scoffing openly and she felt fine with reminding me of my goals, not allowing my emotions to derail the discussion but remaining fully present and focused.
As our meetings progressed, she offered appreciation of the progress I made, while encouraging me to try new approaches that could yield better results. Even when I felt like I was backsliding, she found a way to put more emphasis on understanding what I had accomplished and focused on encouraging me to keep going. We talked about what parts of the routine were working well, what lessons I learned (like when I hated the treadmill but loved the AMT – hey, participation in individual exercises is optional!), what I could do differently next time, and what might need more scrutiny. We tried to analyze the problems and propose solutions to the boredom, considering a variety of alternatives. It was honest but not accusatory. (Hey, eveybody gets bored with the routine.)
So I’ll admit she’s done me some good. I agree with another participant who said, “My personality is better, my production has gone up, my mental clarity has improved, and my energy level has increased dramatically.” Granted, I just have a lot of energy in general, so I wasn’t likely to sit back and passively take it in – well, as passive as you can be while sweating profusely. I started to recognize my excuses as just excuses, feeling more empowered to modify the situation, learning to manage that impulse to excuse myself from the hard work of changing. Accepting that I actually knew something about working out and lifting weights and could be responsible for designing more of the workout and analyzing my progress on the path to wellness? Yeah, last week was weird.
One ScrumMaster wrote, “At the end of a successful project, everybody says, ‘Gee, I wish we could do it again.’ Using this definition, was the project a success?” Well, I can’t say that I’ve enjoyed every moment of it, but figuring out that I could test software and sweat profusely at the same time? Priceless! But seriously folks, having my coach express sincere and significant appreciation for the care and work I put into making progress sent the message that she cared about and me personally, not just reducing the failure rate of some anonymous gym member. And that’s where the magic happens.
(Special thanks to my dev James who pointed out that coach approach is workout retro!)
Presented as an Emerging Topic at CAST 2012
This was my talk proposal:
I have always thought of myself as an agile tester. After all, my development teams have always delivered features in 2 week sprints. My testing activities included reviewing requirements or stories before the planning meetings to assemble a list of questions and test ideas that I would use to approach the work proposed. I participated in a review before code completion that allowed for some exploratory testing, brief and informal though it may have been at times. In the past couple of years, I also planned and coded test automation.
However, over the past year, I have been transforming from a pseudo-agile tester to a true agile tester. Rather than sitting apart from the software developers in my own quality engineering department, I am now seated in the same room as the other employees from a mix of disciplines who are on my product delivery team. Rather than testing in a silo, I have been gradually increasing the visibility of testing activities through exploratory test charter management, defect backlog organization, and paired exploratory testing with both testers and non-testers. The feedback loops have shortened and the abbreviated time between activities necessitated adjusting how I provide information.
Testers are in the information business. We take the interests and concerns of the business as communicated through the product owner – or in my case the product owner team – and combine those with the needs of the customer as expressed in the story and further augment those with our experience using and analyzing software for deficiencies, abberations, and oddities. We draw upon a variety of resources including the experience and perspectives of fellow testers, heuristics, and product history to approach the goal of delivering a product the customer values, focusing especially on the quality aspects of that value.
Now that the audience for my testing comprises a mix of disciplines and the work environment has shifted from a heavier process to transparent, quick information access, I have been experimenting with different ways to execute testing and to represent the outcomes of that testing activity so that the information consumers understand it in ways that best suit each of their perspectives.
In my brief presentation, we will examine 3 different agile team member personas and their implications for presenting and maintaining testing information as well as the inherent tensions between their distinct and various needs. I will trace my learning curve of adjusting to their needs through the various experiments I have completed in this context, though these lessons extend beyond a purely cross-functional agile product development team.
Other testers will come away with a fresh perspective on viewing their product team members and focus on the value testing artifacts provide to a software development team.
Last year, I was haunting the interwebs watching, Tweeting, and chatting. This year, I’ll be coming to you live through the magic of technology. (This is the first reason I’ve had to crack open PowerPoint so it should be entertaining!)
Catch my agile software testing emerging topic talk Big Visible Testing at 10 AM PDT today!
Again, here’s the link to watch me:
Update: Recording uploaded to YouTube
I recently tweeted that I was feeling frustrated with the medical billing industry. At that moment, I was particularly bothered by the seemingly endless wait of navigating the phone tree to get to a human being only to be shunted to voicemail. However, miracle of miracles, I did in fact get a return call from a person to discuss a recent explanation of benefits (EOB) I had received from my insurance company.
While my family had benefited from the services of this provider over the course of the last couple of years, we decided to move on and I was looking to settle accounts. I couldn’t help noticing that the EOB included Dates of Service that occurred after we had changed providers. Not wanting to immediately proclaim insurance fraud, I called up the provider to discuss what I imagined to be a data entry mistake.
The helpful billing staff member at the other end of the phone call fielded my questions calmly and clearly had heard this complaint before. She explained to me that sometimes there have been data entry mistakes related to the intake of a patient. Since recurring billing is scheduled based on this intake date, she had encountered situations where all of the billing they ever submitted for a patient had incorrect Date of Service values. She said that in this case she would have to make a note of the error, remember that it had occurred, and continue to submit that note with any problematic records since she wasn’t permitted to alter the medical record. I don’t think she meant that the software provided the option to enter an explanation:
If possible, explain why the earlier note was incorrect, the reason for the error, and the reason the error was noticed.
— Medical Economics
Rather, I think she had some separate, manual workaround for note-taking related to someone’s file.
Apparently there was an update/edit capability in the software but she was instructed not to use this due to legal concerns around altering a medical record. Presumably, their software did not provide differentiation between original and updated values with appropriate timestamps:
With electronic medical records, the computer program must show the dates of the original notes and the dates of any changes or new entries.
— Medical Economics
For the user, the result was effectively having no way to edit data. What a difference that context makes!
However, this was not our problem. She also elucidated a software bug in the system that involved a gradual creep of date ranges. For example, a monthly shipment of supplies would be billed over the course of 4 weeks after each ship date. After 2 years of shipments, the weekly billing dates had a lag of 11 days, which happened to put them after the date they discharged us from their care, making the incorrect billing appear to be fraudulent.
Despite having full CRUD functionality to allow correction of incorrect data entry, the staff would not correct records and in addition to that had to live with the pain of a creeping inaccuracy, leading to enough friction for the billing staff user that she had a canned answer for my concerns, which she had clearly addressed many times with other patients and their families. What a knot of confusion for the staff to untangle over and over again.
I can only hope that my new medical provider has a different medical record software provider and that the creeping crud doesn’t prove to be highly contagious.
With the recent The Hunger Games movie release came several fanboy and fangirl friends banding together to attend a performance on opening weekend. For good or ill, I ended up purchasing the tickets for the group. I made it to the box office days in advance and the transaction was happily mundane and successful.
The day before the show, one of the friends decided to bring a plus one along for the fun. However, she was very concerned that she couldn’t purchase a ticket online and didn’t want to cancel the date due to technical difficulties. She asked whether the theatre could add a ticket to my transaction and I agreed to look into it.
Give me something I can work with
Visiting the theatre’s website, I confirmed that the online information made the date seem doubtful, showing only 2 performances at that time with one of them in sold out status. When I tried the performance that wasn’t sold out, I noticed that the list of available tickets was rather short and only included reserved seating types. I hesitated to buy a single reserved ticket since the rest of us had general admission tickets.
(For those of you outside the U.S., we don’t have many movie theatres with reserved, or assigned, seating here. Almost every showing I’ve ever attended has been general admission. I only recently started patronizing a theatre that provided reserved seating – for a premium. For my American readers, one of the considerations for movie theatre software sold to international chains is the need to provide support for reserved seating as well as intermission.)
May the odds be ever in your favor
I tried calling the theatre to no avail, so I resolved to head over there after work to see about that ticket. When I spoke to the cashier, she explained that what had seemed like 2 separate performances online were really just 1 showtime.
Some programmer’s technical solution to the split house for a single performance came through to the web interface in a confusing form. In my experience, an auditorium, or house, had always been either general admission or reserved seating. And, although I tested movie theatre software for over 5 years, I had not encountered this feature request: splitting a single auditorium into 2 classes of ticketing.
Fortunately for my friend’s date, movie theatre software has a sold out threshold greater than zero, allowing for eventualities like broken seats, roof leaks, or other unexpected customer service issues. Knowing that, I confidently requested another ticket and easily obtained it. Granted, we ended up sitting in the front row craning our necks a bit as the pack of tween girls next to us excitedly discussed the movie play-by-play, but for once my testing savvy turned up a solution rather than a problem, averting a star-crossed lovers situation.