The document has moved .

Marcus Perryman's WebLog

• Let’s Talk About Touch (Part2)
  

  In part 1 I introduced several of the key pieces in Windows Mobile 6.5 gesture story, including the basics of what a touch gesture is and how the system identifies and delivers gesture messages to your app. I finished up by introducing you to the Physics Engine, so let’s continue from there.

  Natural Physical Interaction

  I talked about the new Physics Engine component briefly in part 1 but I skipped over a couple of important bits, so let me cover them off here.

  Panning on a list of data is relatively simple to implement because the expected behaviour is fairly obvious, especially when the input device is a finger or thumb – it should work just like sliding a piece of paper around on a slippery desktop surface. However modelling the scroll gesture at the end of a pan sequence is a little harder to get right because the expectations are not so clear cut: what speed should the data move at, does the speed decay and if so at what rate?

  The touch team did quite a bit of research into what the ‘right’ i.e. most natural response should be and captured the corresponding deceleration algorithm in the Physics Engine. We then built apps around that and presented users with the results so we could fine tune the algorithm parameters.

  It was evident from this research that the human eye is very sensitive to the movement and deceleration used in the scroll animation, and it’s important to ensure the response to a scroll gesture is always predictable and the feedback is instantaneous – this helped us focus our performance tuning efforts when working on our controls in the OS.

  The Physics Engine is used to map the animation of several other aspects such as snapping to an item in a list (more later) and rubber band where the velocity of scroll gesture would take the display position outside the visible content region.

  The key point is when you are implementing UI that moves content in response to a scroll gesture, use the Physics Engine to drive the animation to ensure the user experience is consistent and predictable for the user.

  What do you call the space that’s not client and not ‘non-client’?

  Extending the concept of touchable content gives rise to some scenarios new to Windows Mobile. Imagine an application showing a list of items with a nice scroll bar to indicate the relative position of the visible content. Through gesture support the user can freely navigate up and down the list by direct manipulation (i.e. without touching the scrollbar), but what happens at the limit of the content? Extending the idea of direct manipulating means we really want the top and bottom of the content to be visually flexible so the user can pan the list beyond its limit and see a clear indication of the top of the document –showing a physical document border for example - and then on release see a smooth animation back to a edge of the document.

  We’ve always had the concept of client space where data is drawn, and non client space like the menu, title bar, border etc. But this is something new! The space uncovered by going beyond the list limit doesn’t have a name – it’s not really client space because it’s beyond the client limit (and beyond the scroll bar range!) but it’s definitely not non-client because it’s drawn in the client area. In 6.5 we didn’t formally define this space however when updating the controls and applications to become touch aware we had to solve the problem of what to draw in this space - which was not as straightforward as it sounds when considering that most of our controls can be owner draw / support background themes and were originally designed to work well with a scrollbar.

  If you are planning to support direct manipulation via gestures and make use of the Physics Engine then make sure your controls can support the concept of this new window space. There is an option to disable rubber banding in the Physics Engine and this will stop your window from uncovering this new space – but be aware doing this is likely to degrade the user experience.

  Item Snapping

  While we’re on the topic of the Physics Engine, another feature it exposes is item snapping. Somewhere in Window’s long ago past scrollbars were introduced to support a smaller display area than the actual data needed. And they have served us well for many years. Scrollbars allow the data space and the view space dimensions to be provided while allowing some of the physical display details to remain hidden, such as the real number of pixels needed to draw a piece of the screen.

  There are lots of list style controls out there that use the scrollbar is this way because its very convenient: specifying the scroll range as the number of items, the current scroll position representing the item number and the page size described in number of items that can be visually displayed. The built in listbox and listview controls work in this way.

  Unfortunately this approach causes some problems when introducing direct manipulation, specifically the pan gesture. The user wants to see pixel by pixel movement in response to the touch input but there aren’t actually any valid scrollbar positions available to represent each pixel position because the scroll range is in items and each item is more than one pixel. That may not be such a problem while the gesture is in progress – round the scroll position to the nearest item, after all it’s just an indicator at this point – but when the session ends the user is unlikely to have aligned the item exactly on the pixel boundary required to match the scrollbar.

  The Physics Engine supports the concept of item snapping to help solve this problem. The item height / width can be specified when creating the Physics Engine, and then all animations are guaranteed to end exactly on an item boundary. In the case of a scroll the end point of the scroll is either stretched or shrunk to the nearest boundary and the animation adjusted to make this look smooth and predicted. The Physics Engine can also be used with a zero initial velocity to provide an item snap animation for when the user ends a pan session without a flick.

  Here is a snippet from the Physics Engine sample in the DTK showing how to use item snapping:

  {

      PHYSICSENGINEINIT initState = {sizeof(initState)};

  ...