Abstract:
Think manual testing is waste? Think again! If you’re not learning when you’re testing, you’re doing it wrong! People exploring systems can be your best defense against unknown problems and your greatest way of finding unexpected opportunities.
While automation is well adapted for repeating the same thing over and over again, human beings are great at doing things differently.
Doing is not enough! We need to think during our review and examination processes to improve outcomes. How do we design manual exploration to provide value in today’s fast-moving development culture?
Come to this workshop for hands-on experience with the full lifecycle of exploratory testing charters.
Learning Outcomes:
Structuring manual exploratory testing for transparency
Leo: Bob, there is a ground-breaking new book, that has just come out. Now, not everything in this book, of course, applies to you, but I’m sure that you can see when you see the title, exactly how it could help.
Bob: Baby Steps?
Leo: It means setting small reasonable goals for yourself one day at a time. One tiny step at a time.
My friend is having her first babies. She shared her wonderful plans for the nursery with us and I saw an opportunity to create something special and new for the occasion.
I’ve blogged before about my crafting habits but not about my design process. Given the reference point of the nursery that inspired my mom-to-be friend, I immediately reached out to a more experienced collaborator, a friend who frequently scrapbooks with me.
We riffed on ideas until we landed on one that intrigued us, and we started to develop it more through discussion. However, given our short timeline, since we intended to have the gift ready for the baby shower, I started to create initial prototypes of the components we planned to combine for our project. Using scrap paper, I cut out the shape that had inspired us the most as a reference point. I made several variations that preserved the color palette we wanted to use, so that even those early attempts would provide better information about the final product.
I sent pictures of these prototypes to my friend so that we could evaluate them together before I moved on to the next small piece of work. She had great ideas for coordinating next steps, so I continued to design and construct independent components, evaluating each as I went.
Once I had gathered several together, I called my husband over to provide a second opinion since he is very familiar with the intended recipients of the gift. He liked what he saw and offered suggestions for additional enhancements that I loved – but I hesitated. While I was in love with my design, would our friends like it?
Having invested this much time and effort into the design, initial construction, and overall style, I was loathe to give up any part of my vision. Then, I reminded myself that while I was spending joyful hours creating this work of art, our friends would spend years in the nursery with their children. No matter what I thought of my design, I had to be ready to kill my darlings. I picked up the phone and made the call.
Our friends agreed to take a look at the in-progress photos, to confer privately, and then to get back in touch with us. To my delight, they loved what they saw! Their vision for the nursery matched our vision for the gift. I breathed a sigh of relief.
Armed with this early feedback, I felt more confident about moving on to additional design and implementation. However, an unexpected illness kept my friend from being able to collaborate, and our work fell behind schedule. Not wanting to show up empty-handed to the party, despite knowing how welcome and appreciated I would be, I put together a smaller sample of our project as well as the latest work-in-progress photos of the whole.
At the party, I revealed one of the most recent developments to the excited couple. Other guests brought lovely gifts, from necessary supplies to handmade blankets. We enjoyed the serendipity of another decor gift perfectly coordinating with our project! The nursery is coming together, one baby step at a time.
While we weren’t able to deliver everything we hoped at the time we intended, we delivered something valuable as early as possible with the knowledge that the mom’s “delivery date” milestone is a bit farther down the road – the only delivery in our project that won’t be early and often.
A funny thing happened today at work. I found out that some of my colleagues literally see things differently. Many of us found ourselves surprised by what others perceived to be true about something as simple as an image. We were swept up in #dressgate: a raging internet controversy about a photo of a dress and its colors.
I’m on Team Blue and Black. However, I wanted to see how the other half lives. I tried various ways to see white and gold: viewing the image on different devices, changing screen brightness, angling the screen, walking around in different ambient light. The various experiments all produced the same results. Trusting my perceptions, I could not give any credence to the perspective that the dress was a different pair of colors, despite seeing many online posts to that effect.
I mentioned this to my team at work, only to discover that there were others who had no idea anyone disagreed with them. As a member of Team White and Gold, my team’s designer was surprised to hear there was a Team Blue and Black – as surprised as I was. 🙂 I couldn’t help wondering whether she was expecting a covert camera to emerge as part of some elaborate prank.
Fortunately, working with designers means having deeper organizational knowledge about colors. By the time lunch rolled around, another colleague had created an online tool for experimentation with the image to see for ourselves how image manipulation would change perception. Another designer mentioned that he had sampled the original image to identify the colors and then created swatches of the colors perceived by others to overlay the image in order to show both positions contrasted with each other, explaining about the impact of shadows and subtle colorblindness.
Designers FTW!
Then, he suggested another avenue of investigation: flash blindness. In flash blindness, a bright light bleaches (oversaturates) the retinal pigment resulting in sudden vision loss that doesn’t immediately return to normal, but it usually wears off gradually. So my team devised an experiment to expose our designer’s eyes to a bright white lightsource: a blank page on a screen. When she quickly switched from the bright white background to the original dress image, she was able to see blue and black coloration. However, after a few moments, when she glanced at the dress image again, her retinas had recovered and she saw the original white and gold pigments. This was consistent with reports from other online posters who mentioned scrolling down the page and then being able to see different colors. This transient state seemed to be a source of great consternation and some panic.
While this was a fun way to spend our lunch hour, it was also a great opportunity to practice some of the problem-solving skills I learned at last year’s Problem Solving Leadership workshop:
Experimenting to gather information – Although I was not able to see the white and gold version of the dress without manipulating the image, I learned new ways that didn’t work.
Perceptions, What’s true for you – I felt quite certain about the stability my own perceptions after looking at them from various angles
Watch how other people are behaving – While I thought it was quite surprising that many others had such completely different perceptions, I did not assume they were wrong just because I couldn’t observe the same things.
Be cautious about not noticing – I gave others the benefit of the doubt knowing that I can bias myself to ignore information sometimes.
How to take in info – I looked for a variety of sources of information about the disparate points of view to obtain a balanced set of data.
Resisting information – I paid attention to reports of heated arguments between people from the different viewpoints, noticing the emotion involved in what seemed like a purely factual question.
Motives (test interpretation, seek intent) – I asked two observers from Team White and Gold questions since they could see what I could not
Reading minds – I tired not to assume that anyone was punking me or simply being ornery but instead was open to the possibility of being wrong.
Style vs intent (make more congruent) – Rather than trying to convince anyone of my point of view, I listened to their experiences and observed their learning process.
Social structures – It was interesting to see that even within the design group there were opposing assessments of the information. I also saw how team members collaborated rather than confronted each other when trying to understand where each was coming from.
How do you get people to recognize what you saw? – I waited for an opportunity for them to experience it directly and shared the information that I had so the other team members could judge for themselves, now that they had more to work data
Show you care by speaking up – I could have ignored people who didn’t agree with me, dismissing their viewpoint as simply wrong. However, engaging in dialogue was a great team-building experience and helped to establish more common understanding.
Reactions – By giving myself a charter of observing others’ behavior, thought processes, and evidence, I was better able to empathize with what was a shocking experience from their point of view.
Eyes open! Use your senses – I took suggestions from the designers about resources for assessing color perception and did not assume that I could gather unbiased information. In the end, I know more about myself than I did when this silly discussion started.
Learn from others – I certainly know more about color, perception, troubleshooting, experimentation, and these particular colleagues than I did before I posted the question “What color is this dress?” so I call today a win. 🙂
Aaaaand I couldn’t help trolling just a little bit by “wearing the colors” today…
