Tec's soup

So What About Synchronous Remote Usability Testing?

The previous week, I posted part One of this article.

In order to integrate the user into the design process at the earliest possible date, it is customary to develop a low-fi prototype which will allow initial evaluation of the overall design. We are however faced with a problem if we want to combine the qualitative feedback of synchronous usability testing with the advantages of non-biased remote testing. How can both approaches be combined?

Yet again, technology should hold the answer to this question:

Requirements for Synchronous Usability Testing:

• test-user and moderator need to be connected via the Internet
• a shared screen needs to be in place for both to view the prototype
• a live audio connection should be in place for synchronous communication
• the data of the test-session needs to be recorded (audio, video recording of all mouse movements)
• recorded data needs to be stored for instant retrieval

Optional requirements which may increase the quality of the session:

• information and tasks should be visible to the test-user
• moderator’s ability to integrate questionnaires which the user can fill in his own time
• the moderator can change parts of the prototype whilst conducting the test
• comments and annotations can be included on-the-fly
• test-user and moderator can be connected via a live video-feed

Usability Test: Methodologies

The overall test should allow the following methodologies:

• Thinking Aloud (test-user must think aloud and express what he thinks or misses in the prototype)
• Wizard of Oz Prototyping (the prototype can be changed on-the-fly for instant ratifications)
• Team Observation: (the rest of the production team follows the usability test, make notes and come up with alternatives)
• Formal and Informal Tests
• Click-path Analysis
• Use of Questionnaires

Next week, part THREE of this article will follow.

The article will be posted in 3 parts -which should be released a week after another.

Developing a software application is a creative and complex process. It becomes specially challenging when trying to design an interface which suits the end-user. Designers, project managers and programmers all have different views on how to approach their common goal: a successful, stable and user-friendly design.

Now let’s be honest: how many of you have met their targets on time and within budget? How many of you have spent sleepless nights because of unforeseen problems which came up in the very last minute?

I bet there are a few of you and one of them is me!

Most problems come up at a stage where a lot of design and programming work has already been invested. These problems become apparent when the test-users start getting their fingers dirty and as soon as they uncover flaws with your work, a lot of heartache will follow. User-tests are necessary yet are often deployed when it is ‘nearly too late’. Don’t get me wrong, I’m a keen supporter of usability tests but I believe that the traditional ways of running such tests are outdated.

The user should be included in the production process as soon as possible. The sooner this happens, the less time and money you will spend on conducting usability tests since you will be able to spot the flaws at an early stage. Makes sense, right?

So why do we keep insisting on setting up expensive test labs and pay thousands of dollars for expensive equipment and spend time and money in finding the right test-users? I want to present you a different approach to user-testing, a way which may very well change your attitude towards the established forms and also a way which can save you from those troubles mentioned earlier.

In order to gain the feedback you need, it always helps to gather qualitative data of users which can test your product in the right environment. Having housewifes, students or seniors sitting in your office and trying out that new e-commerce website for your client isn’t the way forward, or is it? Well, i think it is defenitely more safe and reliable to let them stay at home and do it from their own desk. Why ruin that new test-rig if you can get your test-users to work remotely?

Remote Usability Testing

Remote Usability Tests are conducted in the natural environment of the user (e.g. at home).

The fact that it is conducted remotely and not in a lab environment has a lot of benefits:

• test-user can work in his natural and familiar surrounding
• limits biased feedback since user is not ‘under pressure’
• saves money for travel & set-up costs
• creates sustainable usability testing for a greener world

Synchronous Usability Testing

Synchronous Usability Testing is a well established approach for running qualitative Usability Tests.

As the name suggests, synchronous testing is done on a one-to-one basis and in real time. The test-user will be running tasks set by the moderator which then can be closely monitored and recorded. The moderator’s difficult task is to interpret and evaluate the test-user’s problems with the product. This is not an easy task and it takes a lot of experience and well refined ability to use the feedback in the most effective way possible. This is why synchronous Usability testing is often done with low-fi prototypes – therefore early testing is possible.

The advantages of Synchronous Usability Testing are:

• allows moderator to guide the user
• produces qualitative feedback
• questions can be solved on the spot
• UX flaws can be spotted at an early stage

Next week, part TWO of this article will follow.

The previous week, part TWO of this post was on the benefits which digital, web-based prototyping will make exportation and presentation easier. This FINAL part will show you how iterations can be made easy and that using digital solutions will in fact help against the fight of global warming.

Quick Iteration:

One of the main benefits of using the digital solution is the possibility to make us of quicker, easier and cost-effective iterations. But this will only happen if the prototype can be adjusted to new requirements in an instant. Digital paper prototypes can be re-used and won’t have to be created all over again once a new iteration starts. You simply have to adjust specific elements once and then apply those changes to the rest of the prototype, that’s it.

Sustainability:

The average usability testing project leaves a footprint of approximately 250 kilograms, or 0.25 a tonne of CO2. That may not seem much but that is close to amount of CO2 emission as a 3 hour flight. Usability testing is universally seen as the best way to improve a system’s ease and satisfaction of use. If one usability test itself emits the equivalent of a 3 hour flight, there clearly are considerable gains to be made! In an ordinary usability test, someone travels from his/her location to a laboratory or office where they interact with a test facilitator. Normally this takes about an hour and the process is repeated with 7 to 10 people.

The carbon emission for a usability testing project is based on an average of 10 participants, with each participant traveling 20 kilometers return to get to the test and spending 1 hour with the test facilitator.
More on: http://www.prnewswire.com

Conclusion:

By making use of the newest technologies it is now possible to make the shift from paper-based prototyping to digital or web-based prototyping. User-centered design, sharing of ideas, iterative work-flow, collection of feedback and collaborative work are all aspects which speak in favor of implementing such process. Developers, designers, clients and test-users alike can benefit from working on digital prototypes which engage them from the very first idea. Unnecessary iterations which often confuse and hinder continuous work-flow can be a thing of the past since everyone will be up-to-date. Test-users can work in their natural surrounding whilst the design team can make changes on-the-fly.

Overall, web-based prototyping can only be beneficial for all parties involved. Also, since using excessive sheets of paper can be a thing of the past, it will be a make our planet a little greener.

The previous week, part One of this post was dealing with some of the core ideas of why classic paper prototyping is no longer sufficient. This week, I will talk about why versioning of prototypes and the ability to acces, export and present the results are a necessity.

The Need for Digital Prototyping:

That big round table to which everyone gathers around can never be big enough! The bigger the table becomes, the more sketches and papers are on the whiteboard, the bigger the chance of losing out on some detail. A reasonable alternative would be making use of digital prototyping and to have that table digitized (including the papers, sketches and the words being said). Why keep working with the whiteboard, stacks of paper which have to be carried around the office? Digitizing the work and all what comes with it would make the chaotic meetings a thing of the past. Using a digital solution is a  way which allows collaborative work-flow to be fully recorded, ammended, shared and viewed at at given time.

Team Work Challenge:

One of the requirements for successful team work is to administrate various versions of the same prototype. Versioning of prototypes has two meanings within this context:

(1) the different stages in development process which can be accessed at any time
(2) different versions of the same prototype

Once several people work on the same prototype the need for digital versioning quickly becomes apparent. While one is already working on the CSS, the other is still in the process of developing the menu bar; now can you see what may go wrong? Using a tool which gathers all the different process and work-flows of the various co-designers will help to unite the project into one single application and help the collaborative flow. Real-time collaboration will ensure that misunderstandings and miss-communication are reduced to a minimum. A team which works from different locations and on different elements within the same project are in dear need of such a tool!

Presenting the Results:

Once a prototype is finished, the real work starts. The prototype will be used for extensive usability tests, will be used for presentations and is part of the developers specification. All the various players need access to the prototype which again needs to be in various formats. Since it is pivotal to have a prototype ready for presentation, viewing and export, it should be digitized and ready-accessible on the Web. Well, not accessible to all, just to the one’s involved in the process of course!

Usability tests need to be run and an interactive and/or clickable prototype must be easily accessible to the test users. Furthermore, if qualitative feedback is what is needed, a moderator also would need to access the prototype without a problem. In addition, the developers and the rest of the design team must be able to quickly ‘click through’ and be able to make some minor changes if needed. This is why a prototype needs to be open to all the people involved, needs to be updated in real-time and be exportable to any format and at any time!

Great post on UCD and the common belief systems. By Thomas Petersen

“In broad terms, user-centered design (UCD) is a design philosophy and a process in which the needs, wants, and limitations of end users of an interface or document are given extensive attention at each stage of the design process. User-centered design can be characterized as a multi-stage problem solving process that not only requires designers to analyze and foresee how users are likely to use an interface, but also to test the validity of their assumptions with regards to user behaviour in real world tests with actual users. Such testing is necessary as it is often very difficult for the designers of an interface to understand intuitively what a first-time user of their design experiences, and what each user’s learning curve may look like.

The chief difference from other interface design philosophies is that user-centered design tries to optimize the user interface around how people can, want, or need to work, rather than forcing the users to change how they work to accommodate the software developers approach.

So as you can see a typical UCD process to define it in terms of the hammer test, is based on testing the drawing, the cutout and the Styrofoam hammer.

Not the actual hammer.So why is that? How comes something that seems to be an obvious problematic implementation of the goal of UCD, have become the norm?

This way, users have become customers and you can suddenly start to test where it matters with valuable feedback.

This will no doubt mean that many have to re-educate themselves and rethink how they approach design whether it be UX, IA, UI or GUI. It is none the less as stated, necessary to stay relevant for the future. A pivotal part of this will also be to re-educate clients and help them understand that they will need to look at at product design a little different.

Design is a decision, not a democracy. If you are serious about using design strategically then courage is the strategic advantage you should be looking for. And with the ability to quickly change wrong assumptions it’s not really dangerous, just common sense.

Jakob Nielsen’s article on Agile User Experience Projects suggests that good implementation of usability in agile projects can be vanguarded by:

“Separate design and development, and have the user interface team progress one step ahead of the implementation team. That way, when it comes time to build something, it’s already been designed and tested. (And yes, you can do both in a week or two by using paper prototypes and discount user testing.)”

Out of our own experience,  small software and design companies do not have the necessary funds to run a dedicated UX-team and by using the above mentioned approach, good design and usability can be achieved.

This article which will deal with the various benefits which digital web-based prototyping can bring to your production cycle. The main idea of this article is to promote the digital implementation from the very start of the production work. Many  design agencies still work with pen and paper, a method which has been used for a long time but which in today’s world should be seen a thing from the past. It is time to explore the possibilities which technology can offer us!

The article will be posted in 3 parts -which should be released a week after another. Part ONE will offer a little review on classic prototyping and talk a little about the general structures within the team. The following articles will go more in-depth and talk a little about the ambitions, requirements and solutions for making the product development a little easier and more fun. I hope that you will enjoy this article and hopefully be able to take something useful out of it. Thanks!

Introduction:

User-centered software development is a dynamic and creative process. In the prototyping phase and in the evaluation, one can see it’s benefits and new challenges arise for the production team. The conceivability of the clients must be fully understood, ideas of the heterogeneous design team as well as the gathered feedback of the target audience must be included into the design. A challenging task indeed!

Due to global requirements of today’s digitally connected world, ‘classic paper prototyping‘ often is no longer sufficient. A bunch of draft papers are easily misunderstood, mixed up or lost in the super information highway and the need for an all-encompassing, digital and rapid solution is becoming more of a demand.

Therefore, web-based rapid paper prototyping has been growing into a well known topic for designers, developers and clients. But what is it what makes digital sketching so appealing to the usability community? Are design agencies simply too lazy to do the manual work with pen and paper or do they simply want to be more Eco-friendly and want to stop the deforestation of the amazon by refusing to use paper-based prototypes? This article will touch some of the core points and issues within the field and will list the justifications of this development.

Paper Prototyping:

The easy way to create low-fi prototypes, to gather the design team around the big table or the whiteboard and a chance for everyone to jot down their ideas on the project. This technique supports the main ideas of rapid paper prototyping. Everyone can make changes and also view the changes already made by others. Everyone knows what is going on.

However, there are a few problems with this method. Increasing the complexity of a prototype whilst keeping a full overview on the project as a whole can become a bit of a challenge. Once a change has been made, it it difficult to be undone. Of course you can simply delete or throw away an error prone design but it is not as easy as a simple ‘CTRL+Z’.
To add to this, imagine that the team works from different locations and with different tasks to manage … it can become a mess in no time! Therefore, paper prototyping is no longer sufficient for the demands of the modern design agency.

Prototyping Roles:

The main roles in the prototyping stage are of course the users, the design team, the developers and the client. In order to finish up with a satisfying product, the different needs of those interest groups must be met and considered. Gathering those people around the same table, meeting their requirements, dealing with time pressure, budget limitations and the different locations of the various key players are a tough one to call.

The Design Team:

The design team is a collective of some very smart and able professionals. The interaction designer, visual designer, information architect, human researcher, usability experts and prototype developers are just some of the many roles one can find in a team. In order to express their varied ideas on how the end product should be like, collaborative prototyping is the way to meet this ambition. This method will enable them to discuss and clarify the multitude of requirements.

Next week part TWO of this article will follow. Subjects are the challenges of a team, collaborative working and requirements for digital prototyping.