PatternMusic Blog

When I worked at Apple and Claris we had sophisticated usability labs with multiple cameras and one-way mirrors to record every subtle aspect of a user’s interactions. We built and tested prototypes to validated our software interface designs.

As a solo independent app developer I knew the value of these techniques, but I also knew that with my limited resources I couldn’t practice them the same way. What I needed to do was recreate as much of the iterative testing and development model as possible but make it practical for developing my app PatternMusic.

A product specification was needed to manage a project with the scope of PatternMusic. Usually a specification’s primary purpose is to communicate between team members. That wasn’t a problem I had. But I also knew from experience that such a document was going to be extremely valuable six months down the line when the objectives of the product’s design would no longer be fresh in my mind.

Usability is a huge part of the iPhone and mobile experience. And I wanted PatternMusic to reflect the accessibility of the iPhone. Don’t confuse that with trying to make music making easy. I believe that’s a fallacy.

PatternMusic is an attempt to do something else entirely. What I’ve tried to do is use the flexibility and power of the iPhone device to lower barriers and foster success among music enthusiasts.

I see usability failures all over the place right now. There are a lot of very hard usability problems in both music software and mobile applications. Unless you address usability you are going to end up with a lot of junk or go through many tries before you end up with something people want to use.

The other thing you see a lot of right now is apps that assume their user base is made up entirely of experts. That’s fine in and of itself, but that was not the design goal of PatternMusic. Fortunately, experts benefit from enhanced usability just as beginners do.

So one of the challenges with PatternMusic was to figure out how as an independent developer to incorporate usability engineering into my garage shop software development craft. Obviously intuition is a big part of where you start. You go with your gut. Find patterns. Use cookbooks. User interface issues are often known problems, and many problems have been solved with varying degrees of success. You do market research, and you look at what’s been done before. You find out what works and what doesn’t. But use critical skills, and you don’t just mimic. Figure out where the usability traps are.

Those are the basics for coming up with your design specification. But then you need to have a way to validate your designs during the implementation phases of development. Most formal design processes would be completely impractical or impossible for an independent software developer. But what I tried to do was tailor usability techniques into my development by capturing user reactions through frequent and often casual testing iterations. The cost was that it resulted in significantly extending the development cycles compared to developing to a hard specification. And I ended up doing usability tweaks all the way up through final beta testing.

As an example, just the before the app was completed I determined that the pop-up tips that had been added to clarify the navigation controls would work even better if they were actual buttons rather than just label balloons. So in this case the interaction model had gone full circle. I had started with buttons. They were replaced with a pop-out tab-button mechanism to maximize screen real estate. But it soon became clear that, while effective with experienced users, the pop-out tabs were causing trouble for beginners. Beginners needed traditional buttons. So the shipping app ended up with both mechanisms. The highly efficient pop-out mechanism is available for users who are up the learning curve and the very accessible standard buttons are there for the novice. The two systems ended up gracefully complementing each other.

Looking at the many aspects of PatternMusic’s design that went through multiple design iterations, one area that stands out was the instrument note pattern editor. I knew from the start that the pattern editor had to integrate music note feedback. You can’t just expect users to set notes based only on note names or the position on a piano keyboard and hope they will know how that is going to sound. PatternMusic had to have musical feedback as you touched. The app had to be an interactive musical instrument simultaneously with being a sequence editor — all that working through the same interface and sound engine. That was fundamental.

PatternMusic uses a touch-hold interface — you touch the button and it sounds the note; you hold the button and you set the note. It seems obvious, and it proves to be a very intuitive mechanism that’s easy for the user to discover. One thing that was tricky was coming up with the timing where touch-hold became easily discoverable but still gave the user a sense of being in control. In the end I added a setting to allow advanced users to fine-tune that sensitivity.

In addition to interactive musical feedback in response to touches I knew the pattern editor was going to require visual feedback. Initial designs used a small circle to communicate where you were touching on the grid. And if you held the touch the actual dot underneath would get filled in solid. I discovered very early on that some people have very big fingers, and some people just couldn’t see the circle because of the way they hold their fingers. I had to make the circle bigger so it wasn’t obscured by the actual touch. So the circle got bigger… but then it became harder and harder to see precisely where you were touching. I animated the circle to make it contract around your touch. That worked well — an added benefit was that the animation gave users an indication of how long they needed to hold the touch. That worked better too, but it was still hard for people to tell precisely where they were touching. I added a pop-up label over the piano keyboard to indicate what note you were hitting, but people still had trouble.  Finally adding cross-hairs in addition to the touch ring solved the problem. People could see and hear what note they were setting and what the pitch was. And if they didn’t want that note they could intuitively move their finger around. It was a success…  most of the time.

Even with all those touch feedback mechanisms a significant group of testers still had trouble with precision. Apple recommends 44-pixel-wide buttons on the iPhone. On a 320×480 display that doesn’t give you room for a lot of notes. I had to go to about half the recommended size in order to get a 12 note scale in one dimension and 16 notes in the other. Because PatternMusic has visual and audible feedback it’s actually more precise than that seems, but still it wasn’t working for some testers.  So I created a zoom mechanism to provide larger note targets. It’s called “big boxes”. The big boxes mode gives users a note target that is 50% larger than the default. As a consequence PatternMusic can only display an 8 note scale vertically at one time when displaying “big boxes”.  Users were losing the ability to view the bigger picture view in exchange for greater precision. It was clear that the mechanism to toggle the view from small to big notes needed to be right there on the editor screen — not hidden away off in the Settings App. The “big boxes” mode toggle button became an application-wide preference right there on the transport controls of the editor. But testing showed it worked. Users understood the mechanism, and once it was implemented they expected it to behave the way it does.

“I’ve got blisters on my fingers!”

Another pattern editor usability issue showed up with a tester who is a long-time guitar and mandolin player. The tips of his fingers are neither flat nor pointed. He actually has several points on the tips of his fingers because of the calluses that have formed with 30 years of daily guitar playing. He found it difficult to make a single touch to set a single note — always inadvertently getting accidental multiple touches and setting multiple notes. Here was an example where multi-touch (which is used in the pattern editor in order to allow you to create chords or set notes very quickly) was getting in the way. I added an advanced option to turn off multi-touch and force the pattern editor to only respond to a single touch. The user interface scales back to single-touch use without any issues except the loss of chording. Problem solved.

These examples of incremental usability improvements were rolled into the pattern editor section of PatternMusic through an iterative testing and design process. But that’s only half of the story. I won’t go into detail about the features that were removed from the app because they tested poorly, or it became clear that the feature was going to require an inordinate amount of time to refine.  Some of these almost-ran features as implemented might have worked satisfactorily in a one-off or D.I.Y. project like a Reaktor or Max patch, but they weren’t going to work across a user-base with vastly varying levels of experience, patience and skill. That’s been an important part of what PatternMusic is about from the very beginning of its design.

Usability engineering is a long process. It requires a lot more than just saying, “I’m going to built it this way.” When faced with a usability problem a developer’s gut reaction is often to say the users must be wrong, “I’m not going to change it just because they can’t figure it out.” It’s very common. But I set some very specific goals for usability in PatternMusic, and that’s the path I tried to follow. From the beginning I had rule on the design of the product, that if I was going to put a feature in, that it had to really work well. If it wasn’t going to be good enough it wasn’t going in the app.

I hope that this attention to detail shows every time you use the app. PatternMusic is designed to be environment for music exploration so I designed it for return visits.

Check out the PatternMusic Pattern Editor Tutorial to see how it turned out.