UI testing for v2.5?

Testing for OpenLP (rightly) focuses on verification and validation of the program's functionality. This ensures it is built right; testing for bugs, and checking it all works as expected.

How much has testing been utilised in the design phases? User testing can be particularly helpful in making UI decisions. It is one thing for devs to identify features that the program needs, but it is another to assume that since a particular UI setup works for them, and they've never had any issues with it, then that UI will work fine for all users.

See this topic for example discussion of a UI decision: http://openlp.org/en/forums/development/feature_requests/clear_distinction_between_verses_in_preview_and_live_view.html

Don't get me wrong, I think OpenLP has a great UI, and I don't want to change anything for v2.0! I'm just wondering how much scope there is to incorporate user testing in the future design process.

Useful testing could include: testing of different levels of users on the existing interface, on possible UI mockups, and on alternative programs (eg OpenSong, EasyWorship) - and analysing the positives and negatives of each interface for different users.

The big problem with UI design "testing" is that everyone has their own opinion of what a good UI looks like. Have you seen the rather different UIs of each of the commercial offerings? And how each of the applications boasts that it has the most user-friendly interface?

Now, I'm not saying that OpenLP has the best interface out there, but in the feedback we've received from folks over the last 7 years, many have remarked that OpenLP has an easy-to-use interface. In developing OpenLP 2.0 and having the opportunity to rewrite the UI as well, we decided to rather stick with what is "tried and true". The current layout seems to work quite well, and we didn't want to completely alienate our version 1.2.x users when they move to OpenLP 2.0.

We are open to ideas for smaller improvements in the UI (have you seen the changes we've made to the song and Bible searching interfaces over the last few releases?) where they make sense, and I'm always willing to listen to well thought out reasons for particular UI ideas.

We can't improve OpenLP if we don't know where it needs improvement, so if you have ideas let us know, but please don't feel offended if we don't like them.

Yeah the different commercial packages do have some pretty different ideas! I think OpenLP should pride itself on having an interface that is powerful and pretty efficient and yet not too complex for the beginner user  In my personal opinion, some elements of the UI hit the spot really nicely, while some could be tuned a little more.

My main point in the post above is to do with UI usability testing. I know everyone, user and dev alike, has their own opinion of what the UI should and shouldn't be!

But what I'm wondering is, once an idea has been tabled (or implemented), would there be value in doing proper usability testing on how well the UI change works?

I'm talking about setting a user a task they need to achieve, and then observing (directly / via video recording / via user 'think-aloud' / via post-test interview) how they go about it, what difficulties they have, how long it takes them, etc. This can be done on a prototype or a working system, on both old and new versions of the UI. The aim is to gather quantiative data on whether a UI change is better or worse, for both novice and experienced users, and use that data to make more informed UI design decisions.

Has this sort of usability testing been done for OpenLP? With any success?

UI design then becomes an iterative process of redesigning the UI on the basis of usability testing, and then submitting it for more testing, and so on.

PS I'm glad to discover a dev culture that is open to ideas, and honest in it's assessment of them!

The main reason why this has not been done before is the lack of testers. While being around in an open source project long enough, everybody tends to end up as a developer, so watch out.

I am sure developers would love to hear your ideas about what needs improvement and how you would like it done, if these are presented in reasonably short manner and do not sound too demanding. And if the changes are easy enough, these may be used. Your best change doing some lobby work to get a developer interested, is on IRC.