OfficeMath

Murray Sargent

Microsoft Word 2007 and RichEdit 6.0 introduced the native Office math facility. PowerPoint, Excel, and OneNote followed suit in 2010, and Mac Word followed in 2011. But ironically the native math facility didn’t have a recognizable name. “Microsoft Equation Editor” (MEE) seemed natural, but that’s the name of the Design Science math editor that shipped first in Office on Windows and the Mac in 1992 and was discontinued due to security problems. In fact, the post Converting Microsoft Equation Editor Objects to OfficeMath needed a name for the native math facility since it describes how you can convert MEE OLE math objects into native math zones. Sometimes people refer to the native math facility as OMML (Office Math Markup Language), which is the XML file format that encapsulates the in-memory math model and is used in docx, pptx, and xlsx files. But the facility is more than a file format since it has TeX typographical quality, is based on Unicode, has user interfaces (UI) and works with an OpenType math font. “Microsoft Math” is a possible name, but other companies might want to ship something similar. None of the facility’s specifications are proprietary.

So, we call it OfficeMath. “Office” alludes to Microsoft Office but needn’t be exclusive. “Office” suggests a high-quality level (okay, maybe I’m biased ☺). OfficeMath might suggest calculations rather than math text, but documentation can resolve that ambiguity, which also exists for the linear formats AsciiMath and UnicodeMath. The heart of OfficeMath is its in-memory model, named “Professional” in the OfficeMath UI. This model is mirrored in the OMML file format. It features N-ary structures such as integrals with limits and integrands, subscripts, superscripts and accents with well-defined bases, and math functions with function names and arguments. This level of detail is ordinarily reserved for content math formats such as Content MathML and OpenMath. OfficeMath incorporated these structures to support high-quality math typography, with the nice side effect of facilitating symbolic manipulations and graphing (OneNote Math Assistant). This post summarizes OfficeMath’s history, model, file format support, interoperability, math font, math formatting, user interfaces, and includes links to further information in OfficeMath-oriented posts in Math in Office.

Contents

OfficeMath

History

Math Model

Supported File formats

Interoperability

Math Font

Formatting

OfficeMath User Iinterfaces

Editing Math with Keyboard

Editing Math using Ribbon, Dialogs, Context Menus

Ink

Accessibility

History

A good place to learn about the origins of OfficeMath is the post LineServices, which tells how the LineServices line-layout component came to be and how it evolved to yield TeX-quality math typography. OfficeMath depends on other technologies as well, including the creation of the math-font OpenType standard described in High-Quality Editing and Display of Mathematical Text in Office 2007 and OpenType Math Tables. For older history, the post How I got into technical WP describes the first math display program (Scroll, 1970) and predecessors of UnicodeMath.

OfficeMath was based on Unicode from the start. Unicode 3.2 (March 2002) already had most of the current Unicode math character set. The Unicode Technical Committee is committed to including all attested math symbols in the Unicode Standard, so Unicode makes an ideal foundation on which to build math functionality. It also streamlines incorporation into Microsoft Office applications, since they are based on Unicode.

Math Model

As with [La]TeX, MathML, MathType, and most other math presentation formats, OfficeMath puts math expressions and equations into math zones. Math-zone typography differs from the typography of ordinary text (see the section on Formatting below). The user creates a math zone with the Alt+= hot key or inserts one from the ribbon Insert tab.

In the OfficeMath in-memory format, mathematical objects like fraction and subscript are represented by a start delimiter, the first argument, an argument separator if the object has more than one argument, the second argument, etc., with the final argument terminated by an end delimiter. For example, the fraction 𝑎/2 is represented in built-up format by {_frac 𝑎|2} where {_frac is the start delimiter, | is the argument separator, and } is the end delimiter. Similarly, the subscript object 𝑎₂ is represented by {_sub 𝑎|2}. The start delimiter is the same character for all math objects as are the separator and end delimiters. In RichEdit, these delimiters are given by the Unicode characters U+FDD0, U+FDEE, and U+FDEF, respectively. In OMML, the start delimiter is represented by a container element, such as <f> for fraction and arguments appear within argument element containers, such as <num>…</num> for a numerator.

The type of object is specified by a character-format property associated with the start delimiter. In plain text, the built-up forms of the fraction and subscript are identical if the fraction arguments are the same as their subscript counterparts. In the example here, a plain-text search for {_frac 𝑎|2} matches {_sub 𝑎|2} as well as {_frac 𝑎|2}. Searching for OfficeMath equations involves plain-text searches like this together with comparison of the object types. The OfficeMath math objects are listed in the table in the next section along with their OMML and Presentation MathML representations. The objects are represented by prefix notation: the character formatting of the object start delimiter contains the object properties (see ITextRange2::GetInlineObject()). This differs from infix notation like a/b, which needs to be parsed. The OfficeMath in-memory format is a “built-up” format as distinguished from linear formats like UnicodeMath and LaTeX.

Supported File formats

The OMML format is the XML format that encapsulates the OfficeMath in-memory “Professional” format. When OfficeMath was designed, Presentation MathML 3.0 was nearing publication. But Presentation MathML is missing two important elements which therefore require <mrow> emulations to represent OfficeMath. Specifically, Presentation MathML doesn’t have an explicit N-ary element, nor does it have an explicit math-function element. Furthermore, OfficeMath needs to embed client (Word, PowerPoint, Excel, …) XML easily into the math XML. The MathML <semantics> element can embed such information, but it’s awkward. Accordingly, OMML was created to describe the OfficeMath in-memory format naturally. With best practices, MathML without the <semantics> element can be used to round-trip OfficeMath equations apart from non-math formatting like revision markings and embedded objects.

Here is a listing from MathML and Ecma Math (OMML) of the OMML elements and exact or approximate MathML counterparts

Other OMML references are Extracting OMML from Word 2003 Math Zone Images and OMML Specification, Version 2.

More MathML discussion is given in MathML 3.0, Improved MathML support in Word 2007, Rendering MathML in HTML5, and MathML on the Windows Clipboard.

Mathematical RTF is essentially OMML in RTF syntax. See also Office Math RTF and OMML Documentation and Updated RTF Specification.

Linear Format Notations for Mathematics include UnicodeMath and LaTeX Math in Office. See also Recognizing LaTeX Input in UnicodeMath Input Model.

Interoperability

Major interoperability is afforded via Presentation MathML and [La]TeX math. In addition, the Design Science MEE and MathType equations can be converted to OfficeMath as described in Converting Microsoft Equation Editor Objects to OfficeMath. MathType can convert OfficeMath to MathType equations. These equation facilities are compared in Equation-Editor Office-Math Feature Comparison and Other Office Math Editing Facilities. The latter also compares them to the Microsoft Word EQ Field.

With a bit of effort, equations can be imported into Office applications from Wikipedia. You can also create HTML documents with equations in them.

Math Font

A basic part of OfficeMath is the Unicode OpenType math font. The first such font, Cambria Math, and the OpenType math tables were developed together with the Office 2007 math software, each influencing the other to obtain high quality results. Some history is given in the post High-Quality Editing and Display of Mathematical Text in Office 2007. The font contains extensive math tables, glyph variants and glyphs for most of the Unicode math character set. The tables were incorporated into the OpenType standard as noted in OpenType Math Tables. Posts elaborating on the math font are Special Capabilities of a Math Font and High Fonts and Math Fonts.

Cambria Math and Cambria are serifed fonts designed to look good on digital displays. As such, the stem widths never get skinny, in contrast to Times Roman fonts. If you prefer, the STIX math font is a Times Roman font that includes the OpenType math table support and works with OfficeMath. It might be good to add a variable-font weight axis to math fonts for this purpose.

Formatting

This section discusses how OfficeMath handles math formatting involving math spacing, math styles, and alignments, and gives links to posts with further information. A math zone is defined by the math-zone character-format effect, an effect like bold or italic. As such, this is a non-nestable property, unlike math objects like fractions, which can be nested arbitrarily deeply. Adjacent math zones automatically merge into a single math zone.

An essential part of good math typography is math spacing. Within a math zone, OfficeMath follows the math spacing rules given in Appendix G of The TeXbook plus some enhancements that weren’t added to TeX for reasons of archivability. Section 3.16 of UnicodeMath summarizes the rules for the most common situations. Also see User Spaces in Math Zones for ways that OfficeMath autocorrects typical user input spacing errors. Two Math Typography Niceties shows how phantom objects can improve math spacing beyond the standard spacing rules.

Math bold and math italic define different math variables in math zones (𝐚 ≠ 𝑎 ≠ a ≠ 𝒂), while in ordinary text, bold and italic are used for emphasis. In math zones, math bold and math italic characters are different Unicode alphanumeric characters, while in ordinary text, bold and italic are character format attributes with no change in character codes. For example, 𝐚 is U+1D41A, 𝑎 is U+1D44E, a is U+0061, and 𝒂 is U+1D482. Even though the math and ordinary-text uses of bold/italic are unrelated semantically, the user can control these math styles using the usual bold and italic UI as described in Using Math Italic and Bold in Word 2007. There are other math styles that yield still different mathematical variables, such as open-face, script, Fractur, and sans serif (see Section 2.2 of Unicode Technical Report #25). In general, character formatting is controlled in math zones as described in Restricted Math Zone Character Formatting. In informal documents, people may want to use sans-serif characters instead of serif characters for aesthetic reasons rather than for defining different variables. Currently OfficeMath doesn’t support this choice, but maybe it should.

Occasionally one needs to embed ordinary text, such as words, into math zones. OfficeMath defines a character format attribute “ordinary text” for this purpose. Text with this attribute uses standard character formatting for italic, bold, etc. Unless the “ordinary text” attribute is active, the bold and italic settings only affect math alphanumerics; ASCII digits, punctuation, operators, and non-math characters are all rendered nonbold and upright.

In addition, OfficeMath has a “no-build-up” attribute to treat operator characters literally rather than use them in build-up translations. For example, if ‘/’ is marked with this attribute, build up in UnicodeMath mode leaves it as the character ‘/’ rather than converting it with the arguments around it into a built-up “stacked” fraction. This attribute can be entered in UnicodeMath by “quoting” the operator, namely preceding the operator by a backslash.

Since math zones are one level deep, you can embed ordinary text into a math zone, but you can’t nest a math zone within that ordinary text or elsewhere within the math zone. This hasn’t proven to be a limitation, although TeX can embed ordinary text inside math zones and nested math zones inside the ordinary text. It always seems to be possible to unwrap such nested math-zone scenarios into unnested math zones.

It’s useful to be able to define math properties for an entire document, rather than specify them for each math zone. This is described in Default Document Math Properties. A new property could be defined to use sans-serif math characters instead of serif characters.

There are two kinds of math zones: inline and display. For example, an inline math zone in TeX has the form $…$ and a display math zone has the form $$…$$. Inline math zones use reduced spacing and character sizes to make expressions fit better in line with normal text. In OfficeMath a display math zone starts at the start of a document or follows a hard or soft paragraph end (U+000D or U+000B, respectively) and ends with a hard or soft paragraph end. In some cases, it would be useful to apply display math-zone formatting to inline math zones, but this isn’t currently available.

Inter-equation alignment and line breaking involve multiple lines. To handle these cases and equation numbering, OfficeMath has the Math Paragraph, while MathML uses tables and MathType uses PILEs. A math paragraph is a sequence of one or more display math zones separated by soft paragraph ends (U+000B). Line breaking can be automatic or manual as described in Breaking Equations into Multiple Lines. Background on paragraph formatting is given in Paragraphs and Paragraph Formatting.

In a document with more than a few equations, it’s useful to number equations referred to from elsewhere in the document. The math paragraph has elegant equation-number support, but it hasn’t been exposed beyond prototyping. The earliest way to handle equation numbering is described in Cool Equation Number Macros for Word 2007. Later ideas are in More on Equation Numbering and equation numbering using equation arrays is described in Equation Numbering in Office 2016. This last approach isn’t quite as convenient as the ideal math-paragraph equation numbering, but it can handle virtually all cases.

OfficeMath UI

OfficeMath UI can be grouped into keyboard, menu/ribbon, ink, and accessibility categories. Let’s consider each of these in turn. The keyboard, menu/ribbon, and ink categories are discussed in Chapter 6 of the book  Creating Research and Scientific Documents with Microsoft Word.

Editing Math with Keyboard

A succinct summary of entering and editing math with a keyboard is given in the original blog’s first post, Formula Autobuildup in Word 2007. Basically, type the hot key Alt+= to insert a math zone and then type math using TeX control words for symbols. For example, in UnicodeMath mode, typing a/b=c inserts

The UnicodeMath syntax resembles that used in programming languages except that it uses many Unicode operators. Naturally there’s much more to math than symbols and fractions, and the keyboard input methods are described in UnicodeMath for the Unicode input method and in LaTeX/TeX input method for the LaTeX/TeX input method.

In UnicodeMath mode, build up to the “Professional” format is automatic as described in When Formula Autobuildup Occurs. In Word’s LaTeX mode, you must request build up. Enter Ctrl+= to build up a math zone into “Professional” format and Shift+Ctrl+= to build the math zone down into the current linear format (UnicodeMath or LaTeX). Or you can click on the corresponding options of the math-zone acetate rectangle.

In addition to the LaTeX/TeX control words, there are operator shortcuts described in Math Keyboard Shortcuts, Negated Operators, Keyboard Operator Shortcuts, Entering Unicode Characters, and Klinke’s Streamlined Math Input Notation. For example, /= autocorrects to ≠ and <= to ≤. Subscripts and superscripts are entered using _ and ^, respectively as discussed in Section 2.2 of UnicodeMath and in Keyboard Entry of Subscripts and Superscripts. Nice things to add include making the leading backslash optional and having an autocomplete drop-down menu of possible control words once you’ve entered the first few characters. For example, many control words start with \left and it would be nice to be able to select the desired one rather than type in the whole word like \leftrightarrow for ↔.

In LaTeX mode, the subscript, superscript, numerator, and other math arguments are single entities. An entity can be a character or control word for a character like \alpha for α, or it can be an expression in curly braces like {a+b}. In UnicodeMath mode, the argument can be a sequence of alphanumeric characters. You can see such a difference by comparing what a^12 becomes: in LaTeX you get 𝑎¹2 and in UnicodeMath you get 𝑎¹². To get the latter in LaTeX input mode, enter a^{12}.

Unicode has many math characters (see Section 2 of Unicode Technical Report #25, Unicode Support for Mathematics). The post Math Symbol Hierarchy divides the math operator symbols into basic, intermediate, and full Unicode math categories. Most technical papers use the symbols in the basic and intermediate categories. The remaining characters are very specialized, e.g., ⪑, so you’ll probably never need them.

Built-up math zones convert alphabetic characters to math alphabetic characters, e.g., ‘a’ becomes ‘𝑎’, which is given by the Unicode character U+1D44E. Conversion to math alphabetic is overruled for special situations like trigonometric function names and can be overruled for arbitrary text. Also, it doesn’t occur for Greek upper-case letters as noted in Math Greek Letters. Math spacing is important and User Spaces in Math Zones explains how UnicodeMath build up may remove a space that’s automatically inserted by math spacing rules. In LaTeX mode, spaces are ignored except to terminate control words.

You can navigate through a math zone Using Left/Right Arrow Keys in Mathematical Text or you can use a mouse. Math Selection is like selection of ordinary text, but if you select a math object start/end/separator delimiter, the whole object is selected. Up and down-arrow keys try to go to the logical target, e.g., up arrow in the denominator of a fraction goes to the numerator. In navigating and selecting text, it’s useful to understand the concept of the Text Insertion Point. The insertion point is in between characters, not on top of a character.

You can enter accented characters as discussed in Math Accents and in Representation of Math Accents. You can enter matrices as discussed in Entering Matrices. If you want to line up two or more equations just right, see Equation Arrays.

In OfficeMath, empty numerators, denominators, subscripts, superscripts, and other essential arguments, etc., display the place-holder character ⬚. If you want to hide the ⬚, insert a “zero-width space” given by the Unicode character U+200B as discussed in The Invisibles. In OneNote you can edit optional arguments. These arguments are normally not shown, but you can move inside them by using the left/right arrow keys. When the IP is inside an optional argument, the ⬚ is displayed and you can enter characters. For example, you can convert a square root into an n^th root by navigating into the root’s index argument and typing n. To make such changes in Word or PowerPoint, you need to use a context-menu option.

Editing Math using Ribbon, Dialogs, Context Menus

If you become familiar with keyboard entry, you’ll probably find that the fastest way to enter math (see also the Ink section next). But admittedly, it’s not obvious how to enter many things. The math ribbon displays lots of math objects in readily clickable form. As such it provides easily discoverable ways to enter common mathematical expressions. For a comparison of keyboard and ribbon, see Math Ribbon Entry of Subscripts and Superscripts.

Math Context Menus provide context-sensitive ways to modify math objects, such as changing a stacked fraction into a slashed fraction, or aligning a set of equations at their equal signs. See also More on Math Context Menus. You can use the Office Insert Symbol Dialog to insert any Unicode character including all Unicode math symbols. The more common math symbols can be inserted using the symbol galleries on the math ribbon. You can also insert many math symbols using the Windows+. hot key.

Smart phones running OfficeMath don’t sport a math ribbon, but a math on-screen keyboard could let you enter lots of math entities easily. Think of exposing math symbols instead of emoji and using surround menus. Also, smart phones can work with ink…

Ink

You can enter equations with a pen as described in OneNote Math Assistant and the links therein. Microsoft’s math ink recognition first shipped in Windows 7 with the applet called the Math Input Panel. This applet lets you enter mathematical text using a pen or a mouse. It recognizes what you enter and displays the result using a private version of RichEdit. It also lets you copy the results to Word, Mathematica, or any other application that reads Presentation MathML.

Many people may find that writing equations by hand is the easiest and fastest way to enter them into a computer. Since I’ve made similar claims for UnicodeMath entry, a colleague of mine and I decided to have a race. I chose nine equations from theoretical physics, and we started entering. The colleague entering via handwriting beat me by a nose, but had two errors, whereas I had none. But really, we both won, since we demonstrated that we could enter equations into Word remarkably fast.

Accessibility

Math accessibility falls into two categories: speech and braille. Microsoft Office Math Speech shipped in over 18 languages in January 2017. As described in Speaking of math…, math speech has two granularities: coarse-grained for fluent speech and fine-grained for editing. Together with touch typing on a keyboard, this combination enables a blind, nondeaf person to consume and edit math, both elementary and advanced.

The OfficeMath speech capability could be extended in useful ways such as offering alternate speech as discussed in Speaking Subscripts, Superscripts, and Fractions. Also, the facility “spoon feeds” the math speech to UI Automation. Some Assisted Technologies (ATs) such as NVDA and JAWS would like to get MathML for math zones and generate the math speech (and braille) themselves. Ways to do this will be the subject of a future post. Interestingly MathML can, in principle, be used both for generating math speech and for editing math as discussed in MathML and OMML User Selection Attributes and Editing Math using MathML for Speech.

Key infrastructure for math braille shipped in August 2017, namely the RichEdit build up/down machinery used by OfficeMath applications added support for entering and editing math using Nemeth Braille—the first math linear format. More work is needed for applications to expose math braille to end users. The main reason for using Nemeth math braille is given in Braille for Math Zones, which points out that the usual braille digit code ambiguities don’t exist in math zones, which is where the math is. Specifically, braille contractions aren’t used in math zones, so digits can be represented unambiguously using computer braille codes; no numerical indicator is needed for digits in Nemeth math zones (aside from an obscure case). Nemeth braille in math zones works with all languages (is globalized), whereas braille in ordinary text is localized to the language being used.

Other posts describing work on math braille include Unicode – Nemeth Character Mappings, which discusses extending the Nemeth specification to include many Unicode math symbols not in the current Nemeth specification and Nemeth Braille Alphanumerics and Unicode Math Alphanumerics, which relates how the Unicode math alphanumerics can be represented using Nemeth braille. The post Math Braille UI describes ways to reveal the math insertion point (IP) using a refreshable braille display. The braille IP location is complicated relative to that for ordinary text in that math structure characters described in OfficeMath aren’t always represented by a Nemeth code. For fractions, they are, but the start delimiter of a subscript object, for example, isn’t present in the Nemeth code.

Math dictation is another math input method for blind and sighted users alike. Imagine, you can say 𝑎² + 𝑏² = 𝑐² faster than you can write or type it! Math dictation can work with all devices, computers, tablets, and phones. See Math Dictation for how it works.

Murray Sargent Principal Software Engineer, CXE (Office Shared)

Follow