{"id":237,"date":"2021-03-29T11:18:21","date_gmt":"2021-03-29T18:18:21","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/math-in-office\/?p=237"},"modified":"2021-04-18T14:39:37","modified_gmt":"2021-04-18T21:39:37","slug":"math-accessibility-trees","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/math-in-office\/math-accessibility-trees\/","title":{"rendered":"Math Accessibility Trees"},"content":{"rendered":"

This post discusses aspects of making mathematical equations accessible to blind people. Equations that are simple typographically, such as \ud835\udc38 = \ud835\udc5a\ud835\udc50\u00b2, are accessible with the use of standard left and right arrow key navigation and with each variable and two-dimensional construct being spoken<\/a> or felt when the insertion point is moved to them. At an insertion point, the user can edit the equation using the regular keyboard input methods, perhaps based on UnicodeMath<\/a> or LaTeX<\/a>, or using a refreshable braille display using Nemeth Braille<\/a>. But it can be hard to visualize a more typographically complex equation, let alone edit it. Instead, the user needs to be able to navigate<\/a> a complex equation using a mathematical tree of the equation.<\/p>\n

More than one kind of tree is possible and this post compares two trees for the equation\n\"Image<\/a><\/p>\n

Each tree node is labeled with its math text in UnicodeMath along with the type of node. UnicodeMath lends itself to being spoken especially if processed a bit to speak things like \ud835\udc4e\u00b2 as \u201ca squared\u201d in the current natural language as described in Speaking of math\u2026<\/a>. The first kind of tree corresponds to the traditional math layout used in documents, while the second kind corresponds to the mathematical semantics. Accordingly we call the first kind a display tree<\/em> and the second a semantic tree<\/em><\/a>.<\/p>\n

More specifically, the display tree corresponds to the way TeX and OfficeMath<\/a> display mathematical text and approximates the way Presentation MathML<\/a> represents mathematical text. Mathematical layout entities such as fractions, integrals, roots, subscripts and superscripts are represented by nodes in trees. Binary and relational operators that don\u2019t require special typography other than appropriate spacing<\/a> are included in text nodes. The display tree for the equation above is<\/p>\n

\u2514\u2500Math zone\r\n  \u2514\u2500 \u201c1\/2\u03c0 \u222b_0^2\u03c0 \u2146\ud835\udf03\/(\ud835\udc4e+\ud835\udc4f sin\u2061 \ud835\udf03) = 1\/\u221a(\ud835\udc4e\u00b2\u2212\ud835\udc4f\u00b2)\u201d\r\n    \u251c\u2500 \u201c1\/2\u03c0\u201d fraction\r\n    \u2502\u00a0\u00a0\u251c\u2500 \u201c1\u201d numerator\r\n    \u2502\u00a0\u00a0\u2514\u2500 \u201c2\u03c0\u201d denominator\r\n    \u251c\u2500 \u201c\u222b_0^2\u03c0 \u2146\u03b8\/(\ud835\udc4e+\ud835\udc4f sin\u2061 \ud835\udf03)\u201d integral\r\n    \u2502\u00a0\u00a0\u251c\u2500 \u201c0\u201d lower limit\r\n    \u2502\u00a0\u00a0\u251c\u2500 \u201c2\u03c0\u201d upper limit\r\n    \u2502\u00a0\u00a0\u2514\u2500 \u201c\u2146\u03b8\/(\ud835\udc4e+\ud835\udc4f sin\u2061 \ud835\udf03)\u201d integrand\r\n    \u2502\u00a0\u00a0\u00a0\u00a0\u00a0\u2514\u2500 \u201c\u2146\u03b8\/(\ud835\udc4e+\ud835\udc4f sin\u2061 \ud835\udf03)\u201d fraction\r\n    \u2502\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u251c\u2500 \u201c\u2146\u03b8\u201d numerator\r\n    \u2502\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u2514\u2500 \u201c\ud835\udc4e+\ud835\udc4f sin\u2061\u03b8\u201d denominator\r\n    \u2502\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u251c\u2500 \u201c\ud835\udc4e+\ud835\udc4f\u201d text\r\n    \u2502\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u2514\u2500 \u201csin\u2061\ud835\udf03\u201d function apply\r\n    \u2502\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u251c\u2500 \u201csin\u201d function name\r\n    \u2502\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u2514\u2500 \ud835\udf03\u201d argument\r\n    \u251c\u2500 \u201c=\u201d text\r\n    \u2514\u2500 \u201c1\/\u221a(\ud835\udc4e\u00b2\u2212\ud835\udc4f\u00b2)\u201d fraction\r\n       \u251c\u2500 \u201c1\u201d numerator\r\n       \u2514\u2500 \u201c\u221a(\ud835\udc4e\u00b2\u2212\ud835\udc4f\u00b2)\u201d denominator\r\n          \u2514\u2500 \u201c\u221a(\ud835\udc4e\u00b2\u2212\ud835\udc4f\u00b2)\u201d radical\r\n             \u251c\u2500 \u201c\u2b1a\u201d degree\r\n             \u2514\u2500 \u201c\ud835\udc4e\u00b2\u2212\ud835\udc4f\u00b2\u201d radicand\r\n                \u251c\u2500 \u201c\ud835\udc4e\u00b2\u201d superscript\r\n                \u2502\u00a0 \u251c\u2500 \u201c\ud835\udc4e\u201d base\r\n \u00a0\u00a0\u00a0\u00a0\u00a0          |  \u2514\u2500 \u201c2\u201d script\r\n                \u251c\u2500 \u201c\u2212\u201d text\r\n                \u2514\u2500 \u201c\ud835\udc4f\u00b2\u201d superscript\r\n                   \u251c\u2500 \u201c\ud835\udc4f\u201d base\r\n                   \u2514\u2500 \u201c2\u201d script\r\n<\/pre>\n
Note that the invisible times implicit between the leading fraction and the integral isn\u2019t displayed and the expression \ud835\udc4e + \ud835\udc4f sin\u03b8<\/em> is displayed as a text node \ud835\udc4e + \ud835\udc4f followed by a function-apply node sin\u03b8<\/em>, without explicit nodes for the + and an implied invisible times.<\/p>\n
To navigate through the \ud835\udc4e + \ud835\udc4f and into the fractions and integral, one can use the usual text left and right arrow keys or their braille equivalents. In OfficeMath<\/a>, one can navigate through the whole equation with these arrow keys, but it\u2019s helpful also to have coarser grained navigation keys to go between sibling nodes and up to parent nodes. For the sake of discussion, let\u2019s suppose the tree navigation hot keys are those defined in the table<\/p>\n\n\n\n\n\n\n
Ctrl+\u2192<\/td>\nGo to next sibling<\/td>\n<\/tr>\n
Ctrl+\u2190<\/td>\nGo to previous sibling<\/td>\n<\/tr>\n
Home<\/td>\nGo to parent ahead of current child<\/td>\n<\/tr>\n
End<\/td>\nGo to parent after current child<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
For example starting at the beginning of the equation, Ctrl+\u2192 moves past the leading fraction to the integral, whereas \u2192 moves to the start of the numerator of the leading fraction. Starting at the beginning of the upper limit, Home goes to the insertion point between the leading fraction and the integral, while End goes to the insertion point in front of the equal sign. Ctrl+\u2192 and Ctrl+\u2190 allow a user to scan an equation rapidly at any level in the hierarchy. After one of these hot keys is pressed, the linear format for the object at the new position can be spoken in a fashion quite similar to ClearSpeak<\/a>. When the user finds a position of interest, s\/he can use the usual input methods to delete and\/or insert new math text.<\/p>\n
Now consider the semantic tree, which allocates nodes to all binary and relational operators as well as to fractions, integrals, etc.<\/p>\n
\u2514\u2500Math zone\r\n  \u2514\u2500 \u201c1\/2\ud835\udf0b \u222b_0^2\ud835\udf0b \u2146\ud835\udf03\/(\ud835\udc4e+\ud835\udc4f sin\u2061\ud835\udf03)=1\/\u221a(\ud835\udc4e\u00b2\u2212 \ud835\udc4f\u00b2)\u201d\r\n     \u2514\u2500 \u201c=\u201d text\r\n        \u251c\u2500 \u201c\u22a0\u2062\u2062\u201d implied times\r\n        \u2502  \u251c\u2500 \u201c1\/2\ud835\udf0b\u201d fraction\r\n        \u2502  \u2502  \u251c\u2500 \u201c1\u201d numerator\t\r\n        \u2502  \u2502  \u2514\u2500 \u201c2\u03c0\u201d denominator\t\r\n        \u2502  \u2514\u2500 \u201c\u222b_0^2\ud835\udf0b \u2146\ud835\udf03\/(\ud835\udc4e+\ud835\udc4f sin\u2061\ud835\udf03)\u201d integral\r\n        \u2502     \u251c\u2500 \u201c0\u201d lower limit\t\r\n        \u2502     \u251c\u2500 \u201c2\u03c0\u201d upper limit\t\r\n        \u2502     \u2514\u2500 \u201c\u2146\ud835\udf03\/(\ud835\udc4e+\ud835\udc4f sin\u2061\ud835\udf03)\u201d integrand\t\r\n        \u2502\t \u2514\u2500 \u201c\u2146\ud835\udf03\/(\ud835\udc4e+\ud835\udc4f sin\u2061\ud835\udf03)\u201d fraction\r\n        \u2502\t    \u251c\u2500 \u201c\u2146\ud835\udf03\u201d numerator\t\r\n        \u2502\t    \u2502\t\u2514\u2500 \u201c\u2062\u22a0\u201d implied times\r\n        \u2502\t    \u2502\t   \u251c\u2500 \u201c\u2146\u201d text\r\n        \u2502\t    \u2502\t   \u2514\u2500 \u201c\ud835\udf03\u201d text\r\n        \u2502\t    \u2514\u2500 \u201c\ud835\udc4e+\ud835\udc4f sin\u2061\ud835\udf03\u201d denominator\t\r\n        \u2502\t       \u2514\u2500 \u201c+\u201d text\r\n        \u2502\t\t  \u251c\u2500 \u201c\ud835\udc4e\u201d text\r\n        \u2502\t\t  \u2514\u2500 \u201c\ud835\udc4f sin\u2061\ud835\udf03\u201d function apply\r\n        \u2502\t\t     \u2514\u2500 \u201c\u2062\u22a0\u201d implied times\r\n        \u2502\t\t\t\u251c\u2500 \u201c\ud835\udc4f\u201d text\r\n        \u2502\t\t\t\u2514\u2500 \u201csin\u2061\ud835\udf03\u201d function\r\n        \u2502\t\t\t   \u2514\u2500 \u201c\u2061\u201d function apply\r\n        \u2502\t\t\t      \u251c\u2500 \u201csin\u201d function name\r\n        \u2502 \t\t\t      \u2514\u2500 \u201c\ud835\udf03\u201d argument\t\r\n        \u2514\u2500 \u201c1\/\u221a(\ud835\udc4e\u00b2\u2212 \ud835\udc4f\u00b2)\u201d fraction\r\n\t   \u251c\u2500 \u201c1\u201d numerator\t\r\n\t   \u2514\u2500 \u201c\u221a(\ud835\udc4e\u00b2\u2212 \ud835\udc4f\u00b2)\u201d denominator\r\n\t      \u2514\u2500 \u201c\u221a(\ud835\udc4e\u00b2\u2212 \ud835\udc4f\u00b2)\u201d radical\r\n  \t\t \u251c\u2500 \u201c\u2b1a\u201d degree\t\r\n\t\t \u2514\u2500 \u201c\ud835\udc4e\u00b2\u2212 \ud835\udc4f\u00b2\u201d radicand\r\n\t\t    \u2514\u2500 \u201c\u2212\u201d text\t\r\n\t\t       \u251c\u2500 \u201c\ud835\udc4e\u00b2\u201d superscript\t\r\n\t\t       \u2502  \u251c\u2500 \u201c\ud835\udc4e\u201d base\r\n\t\t       \u2502  \u2514\u2500 \u201c2\u201d script\t\r\n\t\t       \u2514\u2500 \u201c\ud835\udc4f\u00b2\u201d superscript\t\r\n\t\t\t  \u251c\u2500 \u201c\ud835\udc4f\u201d base\r\n\t\t\t  \u2514\u2500 \u201c2\u201d script\t\r\n<\/pre>\n
The semantic tree corresponds to Content MathML<\/a>. It has drawbacks: 1) it\u2019s bigger and requires more key strokes to navigate, 2) it doesn’t correspond to speech order, and 3) it requires a Polish-prefix mentality. Some people have developed such a mentality, perhaps having used HP calculators, and prefer it. But it\u2019s definitely an acquired taste and it doesn\u2019t correspond to the way that mathematics is conventionally displayed, edited, and spoken. Accordingly the first kind of tree seems significantly better for speech and editing, at least for the math encountered in grades K-12.<\/p>\n
The choice for higher-level math is complicated by the fact that the usual meanings for superscripts, vertical bars, and other notation may be incorrect. For example, exponents are usually powers and it\u2019s appropriate to speak \ud835\udc4e\u00b2 as \u201ca squared\u201d. But in tensor analysis, exponents can be indices and saying them as powers is incorrect. One way around this is to say \ud835\udc4e\u00b2 as \u201ca superscript 2\u201d or \u201ca sup 2\u201d, but it would be better to know the author\u2019s intent and generate more descriptive speech. Another example is |\ud835\udc65|. In math up through calculus, this is the absolute value of \ud835\udc65. However, in higher-level math it could mean the cardinality of the set \ud835\udc65, or something else. In these cases and many others in advanced math, the semantic tree might reveal the author\u2019s intent better than the display tree.<\/p>\n
The MathML working group is studying ways to make Presentation MathML support accurate speech for ambiguous mathematical notations.<\/p>\n
Both kinds of trees include nodes defined by the OMML<\/a> entities listed in the following table along with the corresponding MathML<\/a> entities<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Built-up Office Math Object<\/td>\nOMML tag<\/td>\nMathML<\/td>\n<\/tr>\n
Accent\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/strong><\/td>\nacc<\/td>\nmover\/munder<\/td>\n<\/tr>\n
Bar<\/strong><\/td>\nbar<\/td>\nmover\/munder<\/td>\n<\/tr>\n
Box\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/strong><\/td>\nbox<\/td>\nmenclose (approx)<\/td>\n<\/tr>\n
BoxedFormula\u00a0\u00a0\u00a0\u00a0<\/strong><\/td>\nborderBox<\/td>\nmenclose<\/td>\n<\/tr>\n
Delimiters<\/strong><\/td>\nd<\/td>\nmfenced or mrow with mo\u2019s<\/td>\n<\/tr>\n
EquationArray<\/strong><\/td>\neqArr<\/td>\nmtable (with alignment groups)<\/td>\n<\/tr>\n
Fraction<\/strong><\/td>\nf<\/td>\nmfrac<\/td>\n<\/tr>\n
FunctionApply<\/strong><\/td>\nfunc<\/td>\nmrow with &FunctionApply;<\/td>\n<\/tr>\n
LeftSubSup<\/strong><\/td>\nsPre<\/td>\nmmultiscripts (special case of)<\/td>\n<\/tr>\n
LowerLimit<\/strong><\/td>\nlimLow<\/td>\nmunder<\/td>\n<\/tr>\n
Matrix<\/strong><\/td>\nm<\/td>\nmtable<\/td>\n<\/tr>\n
Nary<\/strong><\/td>\nnary<\/td>\nmrow followed by msubsup w n-ary mo<\/td>\n<\/tr>\n
Phantom<\/strong><\/td>\nphant<\/td>\nmphantom and\/or mpadded<\/td>\n<\/tr>\n
Radical<\/strong><\/td>\nrad<\/td>\nmsqrt\/mroot<\/td>\n<\/tr>\n
GroupChar<\/strong><\/td>\ngroupChr<\/td>\nmover\/munder<\/td>\n<\/tr>\n
Subscript<\/strong><\/td>\nsSub<\/td>\nmsub<\/td>\n<\/tr>\n
SubSup<\/strong><\/td>\nsSubSup<\/td>\nmsubsup<\/td>\n<\/tr>\n
Superscript<\/strong><\/td>\nsSup<\/td>\nmsup<\/td>\n<\/tr>\n
UpperLimit<\/strong><\/td>\nlimUpp<\/td>\nmover<\/td>\n<\/tr>\n
Ordinary text<\/strong><\/td>\nr<\/td>\nmtext<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
MathML has additional nodes, some of which involve infix parsing<\/a> to recognize, e.g., integrals. The OMML entities were defined for typographic reasons since they require special display handling. Interestingly the OMML entities also include useful semantics, such as identifying integrals and trigonometric functions without special parsing.<\/p>\n
In summary, math zones can be made accessible using display trees for which the node contents are spoken in the localized linear format and navigation is accomplished using simple arrow keys, Ctrl arrow keys, and the Home and End keys, or their Braille equivalents. Arriving at any particular insertion point, the user can hear or feel the math text and can edit the text in standard ways.<\/p>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
This post discusses aspects of making mathematical equations accessible to blind people. Equations that are simple typographically, such as \ud835\udc38 = \ud835\udc5a\ud835\udc50\u00b2, are accessible with the use of standard left and right arrow key navigation and with each variable and two-dimensional construct being spoken or felt when the insertion point is moved to them. At […]<\/p>\n","protected":false},"author":40611,"featured_media":55,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-237","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-math-in-office"],"acf":[],"blog_post_summary":"
This post discusses aspects of making mathematical equations accessible to blind people. Equations that are simple typographically, such as \ud835\udc38 = \ud835\udc5a\ud835\udc50\u00b2, are accessible with the use of standard left and right arrow key navigation and with each variable and two-dimensional construct being spoken or felt when the insertion point is moved to them. At […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/posts\/237","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/users\/40611"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/comments?post=237"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/posts\/237\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/media\/55"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/media?parent=237"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/categories?post=237"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/tags?post=237"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}