Friday, November 18, 2016

Session 5: Integrating User Testing/User Research capability into cross-functional Agile development teams

Initiator: Daniel Hettrick
Note taker:

Discussion / Key Concepts / Quotes / Revelations / Highlights

  • Development Team is responsible for executing product vision
    • starts with high-level design (designer's mental model of how product/service will be used/experienced by end-user)
    • may involve creating "personas" which are representative of core demographic
    • Problem: traditionally, users are not engaged until late in development life-cycle
      • e.g. tweaks before launch OR A/B testing of final UI design
      • this is more a validation approach than an iterative design and development approach
  • User Testing/User Research domain encompasses multiple concepts and numerous types of testing including (but not limited to):
    • Usability Testing (identifies inconsistencies in UI design, differences in designers and users mental models, etc.)
      • Is critically important to do this as early as possible in development cycle
      • Can be done with small batches of users (~6), expert or heuristic evaluation
      • Seeking to quantitatively identify "Gulfs or Execution" or "Gulfs of Evaluation" where the user either did not know how to perform an action they wanted or could not understand what they were supposed to do next
    • "Playtesting" for entertainment products/experiences is similar to Usability Testing, but focuses on subjective engagement (how did the user feel) rather than the more quantitative focus of Usability Testing 
    • User Acceptance Testing (UAT) defines threshold criteria for success of key interactions
      • Using a development approach like ATDD (Acceptance Test Driven Development) may help to ensure team's focus remains on critical user interactions
      • This type of testing may be requirement for items being considered "DONE"

Recommendation / Next Steps / Actions
  • Advocate for creating shorter feedback loops between users/customers and development teams
    • Feedback early and often!
  • Embed User Testing skills within team (hire them or train some team members in User Testing design and facilitation)
  • Utilize resources such as
    • Online resources:
      • Usability Hub
    • Consider engaging 3rd party companies such as WAC Research and Key Lime Interactive in FL to help structure, design, and facilitate user testing
  • Consider cultivating a database of testing candidates across multiple demographics to help ensure your tests get fresh insight (do not rely on convenience samples) 
  • Consider adopting ATDD or Behavior Driven Development (BDD) / Specification by Example to align the team's focus with system response to key customer interactions

1 comment:

  1. Additional training resource discussed was Nielsen Norman Group ( which is a wealth of information for Discount Usability Testing and other UX topics.
    They now have a course in Lean UX and Agile -
    Thanks again to all who participated in the session.