Designing The Band: Chapter Five

Quality Assurance and Internet Explorer

The first thing I did last week was to sort out my display issues. These seemed minimal and were restricted to IE, but in the end turned out to be a bit of a pain.

I’ll not bore you with the details. Let’s just say I resorted to hacks and it added a few hours to my total development time. The main thing to note here is that even today, in 2004, when you are building a site from scratch (let alone working with legacy code) you will still run into browser and platform issues.

Once I had those solved I tackled my validation errors, none of which were to hard.

Cafe (Bar) Testing With Real People

I got/stole this idea from an article over at Goto Media.

Cafe testing isn’t really “testing” at all. At least not how I did it, but I think this idea would be good for smaller sites and sites of this sort. Whereas “real testing” might be a bit much.

The Beauideal site would not be big enough for full blown testing and I doubt there would be any budget for it anyway. Things like this cafe test I did can be a cheap, easy way to get some user feedback without resorting to formal, more involved, user testing.

What I did was head down to a small tavern here in Seattle University district with my laptop, my mock-ups and a sign that basically read:

Are you a music fan? Want a free beer? All you need to do is sit down here and give me your time and your honest opinion.”

I then had them look at my mock-ups and answer some of the following questions:

• Do you use the Web?
• Do you visit music/band sites?
• If so, what for?
• If not, why?
• What is your first impression of the site?
• Can you describe it?
• Does it seem easy to understand?
• Do the navigation labels make sense?
• What info would you find in each section?
• Where would you look for “x” information?
• Does anything seem missing?

I talked to eleven people over about a two hour period. Lots of people came up and talked to me, but not everyone wanted to sit down and look at my mockups. A few people thought it seemed like a cool idea and were very interested in why I chose a bar, etc. It was an interesting experience to say the least.

I didn’t get useful feedback from everyone, so I’m not going to go over what everyone said and I’ll try to keep it to short overviews followed by my general “take” of the whole thing. These may be a bit rough — I didn’t have time to smooth out my notes — I’m warning you now.

#1 “Jan”

He doesn’t use the Web much. He’s got access at work, but doesn’t have much time. He’s never intentionally visited a band site, but he’s pretty sure he has seen some in the past. He mentioned he hates “fancy intros” and has a hard time finding what he’s looking for all the time on the Web.

He’s much less Web savvy than most of Beauideal main user base would be I imagine. I got the impression that “Google” is synonymous with the Web for him.

When I showed him the site his first impression was “oh cool.” When asked to describe it he mentioned he liked the colors and the photo. He preferred the first version, as the second didn’t look “hardcore” enough. Overall he thought it was a pretty cool looking site — better than “that animated clip art stuff” he sees all the time.

When asked about the section labels he was a bit confused by the “Extras” label and didn’t have any idea what would be in there. The rest seemed ok to him.

#2 “Alan”

“Alan” is a Web designer! Uh oh! He obviously uses the Web quite a bit, does visit band sites and is usually disappointed in them, even though he is a big fan of Flash for these sites. He figured an all Flash site suits the audience most times and that the kind of users these have would be motivated to get past any problems. (He’s wrong!) When pressed a bit he did concede that there are times when Flash can be frustrating, but with “entertainment” sites this shouldn’t be a problem.

I showed him the site and his first impression was that it had a “different look” to many sites of its kind. He liked it, but thought it looked a bit unpolished. He did think it had potential though and when asked to describe it he said, “grungy and rough — but in a good way.”

He liked the second version a bit more, as it was a bit more polished in his opinion. He did think either would be easier to use than many band sites as they were “simple and more Web site-like” than most music sites. He didn’t seem to think this was necessarily a good thing.

He didn’t have any problems with the labels and what each section would contain. He did ask where the search box was and wondered why we didn’t have one in there.

#3 “Sally”

She is a student and a big music fan. She was a bit older but seemed to be a good representative of Beauideal’s user base. She uses the Web a bit and does look at band sites, mostly for lyrics and other basic info.

She really liked the first design and described it as “classy” and “interesting”. She said she liked how it was a bit different from other sites and that the second one was much “plainer” in comparison.

She thought it would be easy to use, but wondered where she would find lyrics. When I asked here where she would look, she said “Extras” — good because that is where I’d have put them! She also mentioned she really liked having the “news” on the front page as that is sometimes hard to find on sites.

#4 “Lei”

“Lei” was “Sally’s” friend. She doesn’t use the Web much, but she knows here way around pretty well. She doesn’t look at band sites really, not that she could remember. She just checks e-mail, sometimes looks up info on things (she didn’t specify for what, at least not that I can remember, and I didn’t write it down) and uses e-bay.

She really liked the site. Thought it was “pretty” and “really interesting”. She liked the first version as well. The labels seemed to work ok for her. She wanted to know if there was going to be any “fun” stuff like games and such. She never plays games or looks at animations, but she thought a site like this would have them.

#5 “Ray”, #6 “Kevin” and #7 “Katie”

These folkswere there after work together. I talked to them all at once. They all use the Web all the time and did visit band sites pretty often. They seemed pretty Web savvy and hit band sites for all kinds of reasons ranging from entertainment to information.

“Ray” and “Katie” liked the site, but thought it was a bit boring. They wanted to see something a bit more striking, maybe with animation or something “more cool.” They both liked the first version, but again thought it was rough and kind of plain. “Kevin” thought the site was very cool, and liked the fact that it was “designed” but “plain” at the same time. He thinks many designs are too complicated and “busy.”

None of these three had problems with the labels and all mentioned that site seemed like it would be really easy to get around.

General Observations

I think in general people liked it quite a bit and they definitely preferred the first layout over the second. I got the idea that everyone found the site unique in many ways, but not overwhelmingly so. I took that as a good sign. I didn’t see any major problems to fix and I don’t think any major changes would be made based on this feedback.

Overall I found the information I gathered from this exercise helpful, and probably worth $50 and two or so hours of my time, but not amazingly so. I got quite a bit of the same comments from folks and I think that because of the way I did it people seemed to want to give good feedback. I mean I was buying them a drink in most cases and this probably skewed their view even though I told them to be honest.

Also, I feel this would work MUCH better with a real, live site or more fleshed out prototype. I found these people had a hard time getting into the unfinished version.

Still, it was pretty fun and as I said, I did get something out of it! We’re actually planning on a more involved Cafe test for the hospital in the near future. I’ll make sure and let you all know how it goes.

Next Steps: Design Details, Finishing Up, and A Post Mortem

I plan on spending this next week working out some design issues and polishing my templates. If I have time I’ll build out the pages and some of the design elements there. I realize this is something I had planned to do, but I just don’t know if I can.

I also plan to write up detailed “final verdict” for the site. I want to talk about how long it would have taken, and how much it would have cost in the real world. I’d like to estimate the project in real terms and hopefully show value from my process.

It may seem like this hasn’t progressed as well as planned and that it’s taken a long time. But I went to extra effort on many things. These posts, for example, wouldn’t have been a deliverable for any design project and they we’re no cake walk. I’m still pretty sure when the numbers are tailed-up it’ll look pretty good.

See y’all next week for the final chapter in this little experiment.