{"id":633,"date":"2023-04-05T10:22:42","date_gmt":"2023-04-05T17:22:42","guid":{"rendered":"https:\/\/devblogs.microsoft.com\/math-in-office\/?p=633"},"modified":"2023-04-05T15:14:49","modified_gmt":"2023-04-05T22:14:49","slug":"math-dictation","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/math-in-office\/math-dictation\/","title":{"rendered":"Math Dictation"},"content":{"rendered":"

You can dictate \\(a^2+b^2=c^2\\) faster than you can write or type it, so math dictation can be handy for anyone working with math, notably on mobile devices. It can also make math more accessible. Math speech is similar to UnicodeMath<\/a>, which you can use to enter equations into Word, PowerPoint, and other apps. Accordingly, we translate English math speech as recorded via Office dictation into UnicodeMath and build it up into OfficeMath<\/a>. Currently we can dictate equations in English from algebra, trigonometry, and calculus into OneNote and PowerPoint.<\/p>\n

Examples of math dictation and the resulting OfficeMath are<\/p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Math Dictation<\/strong><\/td>\nResulting OfficeMath<\/strong><\/td>\n<\/tr>\n
A squared plus b squared equals c squared<\/td>\n\n

\\(a^2+b^2=c^2\\)<\/p>\n<\/td>\n<\/tr>\n

Integral from minus infinity to infinity of e to the minus x squared dx equals square root of pi<\/td>\n\n

\\(\\displaystyle\\int_{-\\infty}^\\infty e^{-x^2} dx=\\sqrt\\pi\\)<\/p>\n<\/td>\n<\/tr>\n

limit as N goes to infinity of left paren 1 + 1 over n right paren to the N equals e<\/td>\n\n

\\(\\displaystyle\\lim_{n\\rightarrow\\infty}{\\left(1+\\frac{1}{n}\\right)^n}=e\\)<\/p>\n<\/td>\n<\/tr>\n

sine squared x plus cosine squared x equals one<\/td>\n\n

\\(\\sin^2{x}+\\cos^2{x}=1\\)<\/p>\n<\/td>\n<\/tr>\n

derivative of f of x with respect to x = second derivative of f of x with respect to x = second partial derivative of f of x,y with respect to x = 0<\/td>\n\n

\\(\\displaystyle\\frac{df\\left(x\\right)}{dx}=\\frac{d^2f\\left(x\\right)}{dx^2}=\\frac{\\partial^2f\\left(x,y\\right)}{\\partial x^2}=0\\)<\/p>\n<\/td>\n<\/tr>\n

One over two pi space integral from zero to 2\u03c0 of D theta over begin a + b sine theta end equals one over square root of begin a squared – b squared end<\/td>\n\n

\\(\\displaystyle\\frac{1}{2\\pi}\\int_{0}^{2\\pi}\\frac{d\\theta}{a+b\\sin{\\theta}}=\\frac{1}{\\sqrt{a^2-b^2}}\\)<\/p>\n<\/td>\n<\/tr>\n

left paren a plus b right paren to the n equals sum from k = 0 to n of left paren n a top k right paren a to the k space b to the begin n – k end<\/td>\n\n

\\(\\displaystyle\\left(a+b\\right)^n=\\sum_{k=0}^{n}{\\binom{n}{k}a^kb^{n-k}}\\)<\/p>\n<\/td>\n<\/tr>\n

X equals begin minus B plus or minus square root of begin b ^2 – 4 A C end end over 2A<\/td>\n\n

\\(\\displaystyle x=\\frac{-b\\pm\\sqrt{b^2-4ac}}{2a}\\)<\/p>\n<\/td>\n<\/tr>\n

Absolute value of x equals cases if x greater than or equal to 0, ampersand x next if x less than 0, ampersand -x close<\/td>\n\n

\\(\\displaystyle|x|=\\cases{\\mathrm{if}\\,x\u22650,\\,x\\cr \\mathrm{if}\\,x<0,\\,-x}\\)<\/p>\n<\/td>\n<\/tr>\n

Del cross bold cap e equals minus partial derivative of bold cap b with respect to t<\/td>\n\n

\\(\\displaystyle\\nabla\\times\\mathbf{E}=\\ -\\frac{\\partial\\mathbf{B}}{\\partial t}\\)<\/p>\n<\/td>\n<\/tr>\n

i H bar space partial over partial T space cap sigh left paren X, t right paren equals [minus h bar squared over 2M space space partial squared over partial X ^2 plus cap V left paren X, t )] cap psi left paren X, t right paren<\/td>\n\n

\\(\\displaystyle i\\hbar\\frac{\\partial}{\\partial t}\\Psi\\left(x,t\\right)=\\left[-\\frac{\\hbar^2}{2m}\\frac{\\partial^2}{\\partial x^2}+V\\left(x,t\\right)\\right]\\Psi\\left(x,t\\right)\\)<\/p>\n<\/td>\n<\/tr>\n

real part of e to the -i omega t equals cosine omega t<\/td>\n\n

\\(\\displaystyle Re{\\left(e^{-i\\omega t}\\right)}=\\cos{\\omega t}\\)<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

If you know a TeX control word, you can dictate it by saying \u201cbackslash <control word>\u201d.<\/p>\n

Math Speech Cues<\/h2>\n

Just as you may need to include words like \u201ccomma\u201d and \u201cquestion mark\u201d in ordinary dictation, you may need to include words like \u201cspace\u201d in math dictation to overrule the operator precedence of UnicodeMath. For example, the speech for the sixth equation above starts with \u201cOne over two pi space\u2026\u201d. The \u201cspace\u201d is converted to a \u201c \u201c, which instructs the UnicodeMath build-up engine to build up the 1\/2\ud835\udf0b. Else the integral that follows would end up in the denominator following the 2\ud835\udf0b. Similarly, the examples contain \u201cbegin\u201d and \u201cend\u201d or parentheses to overrule operator precedence. These special words are defined and illustrated in UnicodeMath<\/a>.<\/p>\n

In addition, there are words to choose math styles like script, bold, bold-italic, fraktur, and open-face (or double struck). Examples of such characters are \u210b\ud835\udc07\ud835\udc6f\u210c\u210d. You get these by saying \u201cscript cap h\u201d, \u201cbold cap h\u201d, \u201cbold italic cap h\u201d, \u201cfraktur cap h\u201d, and \u201cdouble struck cap h\u201d, respectively.<\/p>\n

The speech can include common English idioms as in \u201ca doesn\u2019t equal b\u201d or \u201ca isn\u2019t equal to b\u201d, both of which result in \u201c\ud835\udc4e \u2260 \ud835\udc4f\u201d.<\/p>\n

Enabling math dictation input<\/h2>\n

To dictate math in OneNote, enter a math zone with Alt+= hot key and dictate. It\u2019s important to enunciate clearly. Also set the feature gate \u201cMicrosoft.Office.SharedText.OneNoteMathDictation” to true. To dictate math in PowerPoint, do the same after setting the feature gate \u201cMicrosoft.Office.Graphics.EnableMathDictation” to true.<\/p>\n

Implementation<\/h2>\n

The translator converts incoming math speech into UnicodeMath<\/a> and builds up the result into OfficeMath<\/a> in the app backing store. Some additions have been made to the build-up engine, such as the new keyword \\abs for converting \u201cabsolute value of \u2026\u201d which is more natural than \u201cvertical bar \u2026 vertical bar\u201d. Also, the vertical bar notation is ambiguous: it can mean absolute value, cardinality of a set or group, or determinant. The speech \u201cabsolute value of \u2026\u201d is unambiguous. The current dictated math speech produced for \\(a^2+b^2=c^2\\) has some fixups that result in A ^2 + B ^2 = C ^2 , which is close to the UnicodeMath a^2+b^2=c^2. The translator deletes the spaces and converts letters to lower case. The translator also converts \u201ca squared plus b squared equals c squared\u201d. To enable math dictation, include the tomMathSpeech flag (0x10000000) in the call to MathBuildUp<\/a>().<\/p>\n

Dictation Problems<\/h2>\n

Three places to make improvements and fixes are:<\/p>\n

    \n
  • Speech recognizer, e.g., it could benefit from a \u201cmath mode\u201d<\/li>\n
  • Speech-to-UnicodeMath translator<\/li>\n
  • UnicodeMath build-up engine<\/li>\n<\/ul>\n

    Math speech recognition works remarkably well considering that we don\u2019t tell the recognizer that math is involved. Nevertheless, there are results that are wrong for math. For example, \u201cn\u201d is often replaced by \u201cend\u201d and \u201cb\u201d by \u201cbe\u201d. You can get the correct letter by preceding the letter with \u201cletter\u201d which the translator discards. \u201cSum\u201d often ends up as \u201csome\u201d and \u201cbegin\u201d may end up as \u201cbeginning\u201d. You can avoid the \u201csum\u201d error by saying \u201csummation\u201d. So, some editing may be needed before converting the speech to math. The math translation engine autocorrects \u201cbe\u201d to \u201cb\u201d, \u201csome\u201d to \u201csum\u201d, “sigh” to “\ud835\udf13”, and \u201cbeginning\u201d to \u201cbegin\u201d, since these words aren\u2019t generally used in math zones.<\/p>\n","protected":false},"excerpt":{"rendered":"

    You can dictate faster than you can write or type it, so math dictation can be handy for anyone working with math, notably on mobile devices. It can also make math more accessible. Math speech is similar to UnicodeMath, which you can use to enter equations into Word, PowerPoint, and other apps. Accordingly, we translate […]<\/p>\n","protected":false},"author":40611,"featured_media":55,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-633","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-math-in-office"],"acf":[],"blog_post_summary":"

    You can dictate faster than you can write or type it, so math dictation can be handy for anyone working with math, notably on mobile devices. It can also make math more accessible. Math speech is similar to UnicodeMath, which you can use to enter equations into Word, PowerPoint, and other apps. Accordingly, we translate […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/posts\/633","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/users\/40611"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/comments?post=633"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/posts\/633\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/media\/55"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/media?parent=633"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/categories?post=633"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/math-in-office\/wp-json\/wp\/v2\/tags?post=633"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}