Reading the word under the cursor via UI Automation

Raymond Chen

A while back, I showed how to use UI Automation to extract the text under the mouse cursor. Let’s refine the program so it can extract the word under the cursor.

The ability to extract subunits of text is provided by the Text­Pattern class. You can ask for the text around a specific point by asking Range­From­Point to create a zero-length range directly under the cursor, and then adjust the range to grab the text you want.

Start with the first Little Program and make these changes:

using System;
using System.Windows;
using System.Windows.Forms;
using System.Windows.Automation;
using System.Windows.Automation.Text;

class Program
{
 static Point MousePos {
  get { var pos = Control.MousePosition;
        return new Point(pos.X, pos.Y); }
 }

 public static void Main()
 {
  for (;;) {
   AutomationElement e = AutomationElement.FromPoint(MousePos);
   if (e != null) {
    object o;
    if (e.TryGetCurrentPattern(TextPattern.Pattern, out o)) {
     var textPattern = (TextPattern)o;
     var range = textPattern.RangeFromPoint(MousePos);
     range.ExpandToEnclosingUnit(TextUnit.Word);
     Console.WriteLine(range.GetText(-1));
    }
   }
   System.Threading.Thread.Sleep(1000);
  }
 }
}

We grab the element under the mouse and see if it supports Text­Pattern. If so, we ask the text pattern for a Text­Pattern­Range that corresponds to the position of the cursor. It’s where the insertion point would go if you clicked there. Since the insertion point always goes between characters, the initial range has zero length. We expand it to the word enclosing the imaginary insertion point, then extract the text, then print it.

Note that the Text pattern is typically supported only by editable text controls, so you will need to practice with Notepad or some other text editor.

Just to show how annoying you can push things, let’s secretly select the text under the cursor.

using System;
using System.Windows;
using System.Windows.Forms;
using System.Windows.Automation;
using System.Windows.Automation.Text;

class Program
{
 static Point MousePos {
  get { var pos = Control.MousePosition;
        return new Point(pos.X, pos.Y); }
 }

 public static void Main()
 {
  for (;;) {
   AutomationElement e = AutomationElement.FromPoint(MousePos);
   if (e != null) {
    object o;
    if (e.TryGetCurrentPattern(TextPattern.Pattern, out o)) {
     var textPattern = (TextPattern)o;
     var range = textPattern.RangeFromPoint(MousePos);
     range.ExpandToEnclosingUnit(TextUnit.Word);
     // Console.WriteLine(range.GetText(-1));
     range.Select();
    }
   }
   System.Threading.Thread.Sleep(1000);
  }
 }
}

Once a second, the program autoselects the word under the cursor. This gets really annoying fast, but it’s just a demonstration.

Raymond Chen

Follow