{"id":9739,"date":"2013-08-16T13:08:00","date_gmt":"2013-08-16T13:08:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/visualstudioalm\/2013\/08\/16\/coded-ui-test-gesture-support-in-visual-studio-2013\/"},"modified":"2019-02-14T17:54:24","modified_gmt":"2019-02-15T01:54:24","slug":"coded-ui-test-gesture-support-in-visual-studio-2013-2","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/devops\/coded-ui-test-gesture-support-in-visual-studio-2013-2\/","title":{"rendered":"Coded UI Test Gesture support in Visual Studio 2013"},"content":{"rendered":"<p><span style=\"font-size: 12px\">As part of Coded UI Test&rsquo;s support for windows store apps introduced in Microsoft Visual Studio 2012 as detailed in this <\/span><a style=\"font-size: 12px\" href=\"http:\/\/blogs.msdn.com\/b\/visualstudioalm\/archive\/2013\/06\/27\/using-coded-ui-to-test-xaml-based-windows-store-apps.aspx\">blog<\/a><span style=\"font-size: 12px\"> , there is now playback support for touch gestures on Windows Store Apps. These gestures are only a simulation and hence one would not need to have a touch monitor to use these API&rsquo;s. Currently these API&rsquo;s can only be used under the CUIT for Windows store project.<\/span><\/p>\n<p>All the members in this family of Coded UI Test gesture API&rsquo;s have two modes of operation so to speak:<\/p>\n<ul>\n<li>A gesture can be performed on a control<\/li>\n<li>A gesture can be performed on an absolute screen coordinates. We term this a <strong>Control-less<\/strong> &nbsp;<strong>Gesture <\/strong>which will be detailed in a separate blog post. However when using this mode of operation the gesture action is to be framed such that it works the same across resolutions.<\/li>\n<\/ul>\n<p>In the former case the way a gesture searches for a control is similar to how a CodedUITest Mouse action does today. In the section that follows, we walkthrough each of the gesture API&rsquo;s provided with examples. Please refer to the end of the blog post for a sample project demonstrating this support.<\/p>\n<p><span style=\"font-size: large;color: #3366ff\">Gesture classification and walkthrough<\/span><\/p>\n<p>The list of touch API&rsquo;s can be broadly classified into two categories:<\/p>\n<ol>\n<li>Static Gestures &#8211; Static interactions triggered after an interaction is complete.<br \/> Eg: Tap, Double Tap and Press-and-Hold.<\/li>\n<li>Manipulation Gestures &ndash; Involves direct manipulation of the object.<br \/>Eg: Slide, Swipe, Flick, Pinch, Stretch and Turn.<\/li>\n<\/ol>\n<p>Before we proceed with the different gesture API&rsquo;s provided and their usage it is important to understand the parameters that can be passed to them and their units.<\/p>\n<ul>\n<li>Length &nbsp;in pixels<\/li>\n<li>Duration in milliseconds<\/li>\n<li>Direction in degrees<\/li>\n<li>Point coordinates in pixels again.<\/li>\n<\/ul>\n<p> <span style=\"color: #3366ff;font-size: medium\">Static Gestures:<\/span><\/p>\n<p><span style=\"font-size: small\"><strong>Tap \/ Double Tap:<\/strong><\/span><\/p>\n<p>A tap simulates a left mouse button click.<\/p>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008000\"> \/\/ A simple tap on a button control.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Tap(button);<br \/><span style=\"color: #008000\"> \/\/ A tap at the bottom left part of the screen to bring up the start screen. A control-less gesture.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Tap(<span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(5, <span style=\"color: #008080\">UITestControl<\/span>.Desktop.BoundingRectangle.Height - 10));<br \/> <\/code><\/pre>\n<p>&nbsp;A double tap on the other hand simulates a double click which can come into play in cases where one would need to select a word or depending on the app, magnify content.<\/p>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008000\">\/\/ A double tap at a point inside a textbox control to highlight a word.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.DoubleTap(textbox, <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(10,10));<br \/> <\/code><\/pre>\n<p><span style=\"font-size: small\"><strong>Press and Hold:<\/strong><\/span><\/p>\n<p>&nbsp;A press and hold gesture involves a single finger tap which extends for a given duration, the effect of which is similar to a right mouse click. This gesture is mostly used to bring up tool tips or context menus. For instance the following code brings up a context menu on an image control.&nbsp;<\/p>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008000\">\/\/ A press and hold on an image control for a default duration of 1200 ms.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.PressAndHold(imageControl);<br \/>&nbsp;<\/code><\/pre>\n<pre class=\"scroll\"><a href=\"https:\/\/devblogs.microsoft.com\/00\/00\/00\/45\/92\/2211.PressAndHold.png\"><img decoding=\"async\" style=\"margin: 5px\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2013\/08\/2211.PressAndHold-1.png\" alt=\"Press and Hold\" \/><\/a>&nbsp;<\/pre>\n<p><span style=\"color: #3366ff;font-size: medium\">Manipulation Gestures:<\/span><\/p>\n<p><span style=\"font-size: small\"><strong>Slide:<\/strong><\/span><\/p>\n<p>A slide is used primarily for panning interactions but can also be used for drawing, moving or writing.<\/p>\n<p>Scenarios include:<\/p>\n<ul>\n<li>Pan a scrollable container control to bring hidden content into view.<\/li>\n<li>&nbsp;Draw using the slide API in a painting App.<\/li>\n<\/ul>\n<p>Parameters:<\/p>\n<ul>\n<li>Start Point<\/li>\n<li>End Point<\/li>\n<li>Duration&nbsp; &#8211; Default value being 500ms.<\/li>\n<\/ul>\n<p>The values of the start and end points correspond to the coordinates on the screen or relative to a specified control. For instance a panning interaction from (200,300) to (400,300) will result in a slide from (200,300) for 200 pixels. In the following code sample the two slide actions have the same effect but each represents a mode of operation &ndash; Control and Control-less.&nbsp;<\/p>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008080\"> Point<\/span> absoluteStartPoint = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(control.BoundingRectangle.Right -10, control.BoundingRectangle.Top + control.BoundingRectangle.Height\/2);<br \/> <span style=\"color: #008080\">Point<\/span> absoluteEndPoint = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(control.BoundingRectangle.Left + 10, control.BoundingRectangle.Top + control.BoundingRectangle.Height\/2);<br \/> <br \/> <span style=\"color: #008080\">Point<\/span> relativeStartPoint = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(control.BoundingRectangle.Width - 10, control.BoundingRectangle.Height \/ 2);<br \/> <span style=\"color: #008080\">Point<\/span> relativeEndPoint = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(10, control.BoundingRectangle.Height \/ 2);<br \/> <br \/> <span style=\"color: #008000\">\/\/ A slide on the control for the default duration.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Slide(control, relativeStartPoint, relativeEndPoint);<br \/> <br \/> <span style=\"color: #008000\">\/\/ A slide on absolute screen coordinates for the default duration (Control-less mode)<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Slide(absoluteStartPoint, absoluteEndPoint);<br \/> <\/code><\/pre>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/00\/00\/00\/45\/92\/8686.Slide.png\"><img decoding=\"async\" style=\"margin: 5px\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2013\/08\/8686.Slide_.png\" alt=\"Slide\" \/><\/a><\/p>\n<p><strong><span style=\"font-size: small\">Swipe:<\/span><\/strong><\/p>\n<p>A swipe is a short slide with the primary intention of selection. Swipe is always associated with a control and hence this is the only gesture that cannot be operated in a control-less mode.<\/p>\n<p>Scenarios include:<\/p>\n<ul>\n<li>Swipe down on a tile to select it<\/li>\n<li>Swipe across an email in you inbox to select it.<\/li>\n<\/ul>\n<p>Parameters:<\/p>\n<ul>\n<li>Direction &ndash; Specified either by the UITestGestureDirection enum or an angle in degrees.<\/li>\n<li>Length &ndash; Default value being 50 pixels.<\/li>\n<\/ul>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008000\"> \/\/ A swipe downwards on the control for a length of 40 pixels.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Swipe(selectableImageControl, <span style=\"color: #008080\">UITestGestureDirection<\/span>.Down, 40);<\/code><\/pre>\n<pre class=\"scroll\"><a href=\"https:\/\/devblogs.microsoft.com\/00\/00\/00\/45\/92\/7331.swipe.png\"><img decoding=\"async\" style=\"margin: 5px\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2013\/08\/7331.swipe_-1.png\" alt=\"Swipe\" \/><\/a><\/pre>\n<p><strong><span style=\"font-size: small\">Flick:<\/span><\/strong><\/p>\n<p>A flick gesture can be associated with a rapid slide in a scenario where one would quickly want to go through a long list of items. This gesture is characterised by its high speed.<\/p>\n<p>Parameters:<\/p>\n<ul>\n<li>Start Point<\/li>\n<li>Length<\/li>\n<li>Duration<\/li>\n<li>Direction &ndash; Specified either by the UITestGestureDirection enum or an angle in degrees.<\/li>\n<\/ul>\n<p><strong>Note:&nbsp; <\/strong>Although flick, slide and swipe seem similar they differ in their purpose of usage and are hence designed to be comfortably used for their appropriate objective.<\/p>\n<p>Flick is fast if the gesture is to cover a large distance in a short duration and is slow if it is to cover a short distance for a long period of time. The below code snippet illustrates this on the start screen. Notice that as the duration of the gesture increase flick approaches slide.<\/p>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008080\"> Point<\/span> relativeStartPoint = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(startScreen.BoundingRectangle.Width-100,startScreen.BoundingRectangle.Height\/2);<br \/> <br \/> <span style=\"color: #008000\">\/\/ A fast flick on the start screen.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Flick(startScreen, relativeStartPoint, (<span style=\"color: #0000ff\">uint<\/span>)startScreen.BoundingRectangle.Width \/ 2, <span style=\"color: #008080\">UITestGestureDirection<\/span>.Left, <span style=\"color: #ff0000\"><strong>2<\/strong><\/span>);<br \/><span style=\"color: #008000\"> \/\/ A relatively slower flick.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Flick(startScreen, relativeStartPoint, (<span style=\"color: #0000ff\">uint<\/span>)startScreen.BoundingRectangle.Width \/ 2, <span style=\"color: #008080\">UITestGestureDirection<\/span>.Left, <span style=\"color: #ff0000\"><strong>15<\/strong><\/span>);<\/code><\/pre>\n<p><strong><span style=\"font-size: small\">Pinch\/Stretch:<\/span><\/strong><\/p>\n<p>Pinch\/Stretch is essentially used for one of the following types of interactions: optical zoom, resizing or semantic zoom.<\/p>\n<p>Scenarios included:<\/p>\n<ul>\n<li>Zoom in a map<\/li>\n<li>Zoom in a semantic zoom control.<\/li>\n<\/ul>\n<p>Parameters:<\/p>\n<ul>\n<li>Point1 &ndash; Indicating where one touch contact is placed initially<\/li>\n<li>Point2 &#8211; Indicating where the other touch contact is placed initially<\/li>\n<li>Distance each finger moves<\/li>\n<\/ul>\n<p><strong>Note:<\/strong> Pinch never overshoots beyond the point of contact of the two touch points.<\/p>\n<p>As an example the pinch gesture below performs a simple zoom out on an image control.<\/p>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008080\"> Point<\/span> relativePoint1 = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(imageToZoom.BoundingRectangle.Width\/4, imageToZoom.BoundingRectangle.Height\/4); <br \/> <span style=\"color: #008080\">Point<\/span> relativePoint2 = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(3 * imageToZoom.BoundingRectangle.Width\/4, 3* imageToZoom.BoundingRectangle.Height\/4);<br \/> <br \/><span style=\"color: #008000\"> \/\/ A pinch on a control with each touch contact moving a 100 pixels towards each other.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Pinch(imageToZoom, relativePoint1, relativePoint2,100);<br \/> <\/code><\/pre>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/00\/00\/00\/45\/92\/0714.pinch.png\"><img decoding=\"async\" style=\"margin: 5px\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2013\/08\/0714.pinch_.png\" alt=\"Pinch\" \/><\/a>&nbsp;<\/p>\n<p><strong><span style=\"font-size: small\">Turn:<\/span><\/strong><\/p>\n<p>This gesture rotates a control in either clockwise or counter-clockwise direction.<\/p>\n<p>Parameters:<\/p>\n<ul>\n<li>Point1 &ndash; Indicating the focal point for rotation.<\/li>\n<li>Point2 &ndash; Indicating the point on the circumference of the turn.<\/li>\n<li>Angle of Turn.<\/li>\n<\/ul>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008080\"> Point<\/span> relativePoint = <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(imageToTurn.BoundingRectangle.Width - 10, 10);<br \/> <br \/><span style=\"color: #008000\"> \/\/Rotate an image about a focal point for 90 degrees clockwise.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Turn(imageToTurn, <span style=\"color: #0000ff\">new<\/span> <span style=\"color: #008080\">Point<\/span>(10, 10), relativePoint, 90);<br \/> <\/code><\/pre>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/00\/00\/00\/45\/92\/2021.Turn1.png\"><img decoding=\"async\" style=\"margin: 5px\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2013\/08\/2021.Turn1_.png\" alt=\"\" \/><\/a><\/p>\n<p><strong>Note:<\/strong>&nbsp; For a more generic scenario where one would want to rotate a control around its mid-point only the control and the angle of turn needs to be specified and the gesture logic takes care of the rest.<\/p>\n<pre class=\"scroll\"><code class=\"csharp\"><span style=\"color: #008000\">\/\/ Rotate the control around its midpoint for 180 degrees clockwise.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Turn(imageToTurn, 180);<br \/> <\/code><\/pre>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/00\/00\/00\/45\/92\/1643.Turn2.png\"><img decoding=\"async\" style=\"margin: 5px\" src=\"https:\/\/devblogs.microsoft.com\/devops\/wp-content\/uploads\/sites\/6\/2013\/08\/1643.Turn2_.png\" alt=\"Turn\" \/><\/a><br \/> <strong>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <\/strong><strong>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;<\/strong><\/p>\n<p><span style=\"color: #3366ff;font-size: large\">UITestGestureDirection<\/span><\/p>\n<p>This enumeration helps in easily specifying the 4 basic direction in which a gesture can be performed and has the following members:<\/p>\n<ul>\n<li><strong>Right<\/strong> equivalent to 0 degrees from the horizontal.<\/li>\n<li><strong>Down <\/strong>equivalent to 90 degrees from the horizontal.<strong><\/strong><\/li>\n<li><strong>Left <\/strong>equivalent to 180 degrees from the horizontal.<strong><\/strong><\/li>\n<li><strong>Up <\/strong>equivalent to 270 degrees from the horizontal.<strong><\/strong><\/li>\n<\/ul>\n<p>Flick and swipe have overloads which accept this enumeration as a direction.<\/p>\n<p><span style=\"font-size: large;color: #3366ff\">Limitations<\/span><\/p>\n<ul>\n<li>Currently there is no support for creating a custom gesture through Coded UI Test either by sending raw touch input or by coalescing two or more gestures.<\/li>\n<li>UI Synchronization is not enabled for gestures which means that a gesture call will not block until the app responds to it.<\/li>\n<li>The input injection libraries that our gestures implement have a limitation that prevents them from working correctly to activate the edges in a multi-monitor config. Hence, we recommended using Mouse.Click() or Keyboard.Sendkeys() alternatives in place of edge gestures&nbsp; in these scenarios. For instance, in case an app bar needs to be brought up, you can use a Mouse.Click(MouseButtons.Right) or Keyboard.SendKeys(&ldquo;Z&rdquo;,ModifierKeys.Windows). For a Charms bar you can use Keyboard.SendKeys(&ldquo;C&rdquo;, ModifierKeys.Windows). However usage of gestures for edge actions would work fine in a single monitor scenario. You can use a slide or a flick to bring up either the charms or the app bar like below:\n<pre class=\"scroll\"><code class=\"cplusplus\"><span style=\"color: #008000\"> \/\/ To bring up the charms bar.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Slide(<span style=\"color: #0000ff\">new <\/span><span style=\"color: #008080\">Point<\/span>(<span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Width - 1, <span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Height \/ 2),<br \/> new <span style=\"color: #008080\">Point<\/span>(<span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Width - <span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Width \/ 4, <span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Height \/ 2));<br \/> <br \/> <span style=\"color: #008000\">\/\/ To bring up the app bar.<\/span><br \/> <span style=\"color: #008080\">Gesture<\/span>.Flick(<span style=\"color: #0000ff\">new <\/span><span style=\"color: #008080\">Point<\/span>(<span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Width \/ 2, <span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Height - 1),<br \/> (<span style=\"color: #0000ff\">uint<\/span>)<span style=\"color: #008080\">Screen<\/span>.PrimaryScreen.Bounds.Height \/ 4, <span style=\"color: #008080\">UITestGestureDirection<\/span>.Up, 5);<br \/> <\/code><\/pre>\n<p>&nbsp;<\/p>\n<\/li>\n<\/ul>\n<p><span style=\"font-size: large;color: #3366ff\">Additional Resources<\/span><\/p>\n<ul>\n<li>MSDN documentation on touch gestures:&nbsp;<a href=\"http:\/\/msdn.microsoft.com\/en-us\/library\/windows\/apps\/hh761498.aspx\">http:\/\/msdn.microsoft.com\/en-us\/library\/windows\/apps\/hh761498.aspx<\/a>&nbsp;.<\/li>\n<li>Find attached a sample CodedUITest project demonstrating an E2E scenario using touch gestures. Please download and deploy the C# version of the&nbsp;<a href=\"http:\/\/code.msdn.microsoft.com\/windowsapps\/Input-Windows-8-gestures-e0bb367d\">sample app<\/a>&nbsp;before running the test.<\/li>\n<\/ul>\n<p><a href=\"https:\/\/devblogs.microsoft.com\/wp-content\/uploads\/sites\/6\/2019\/02\/CUITGesturesSample.zip\">CUITGesturesSample.zip<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>As part of Coded UI Test&rsquo;s support for windows store apps introduced in Microsoft Visual Studio 2012 as detailed in this blog , there is now playback support for touch gestures on Windows Store Apps. These gestures are only a simulation and hence one would not need to have a touch monitor to use these [&hellip;]<\/p>\n","protected":false},"author":79,"featured_media":45953,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1,252],"tags":[],"class_list":["post-9739","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devops","category-testing"],"acf":[],"blog_post_summary":"<p>As part of Coded UI Test&rsquo;s support for windows store apps introduced in Microsoft Visual Studio 2012 as detailed in this blog , there is now playback support for touch gestures on Windows Store Apps. These gestures are only a simulation and hence one would not need to have a touch monitor to use these [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/9739","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/users\/79"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/comments?post=9739"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/posts\/9739\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media\/45953"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/media?parent=9739"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/categories?post=9739"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/devops\/wp-json\/wp\/v2\/tags?post=9739"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}