LineServices

Murray Sargent

One of the key technologies behind the high-quality display of mathematical text in OfficeMath applications like Word, PowerPoint, and OneNote is a special component called LineServices along with its sibling Page/TableServices (PTS). In addition to handling math display, various versions of LineServices are responsible for line layout in Word, PowerPoint, Publisher, OneNote, RichEdit, WordPad, and the Windows 10 Calculator. LineServices was developed by one of the most amazing teams at Microsoft. Because LineServices is used by components like RichEdit and the XAML text edit controls, it’s indirectly available to developers outside Microsoft. The low-level interfaces to run it directly can be tricky to use and aren’t documented publicly.

This post is an update of an earlier post that tells some of how LineServices came to be and developed over time. It all started in 1994 with one of Microsoft’s most talented engineers, Eliyezer Kohen, a Turkish computer scientist who obtained his PhD at the Technische Hochshule in Zürich with Nicolas Wirth (author of Pascal, among other things). Eliyezer had led the two-man teams (the other person was Dean Ballard) that developed the Microsoft TrueType rasterizer as well as the original OpenType specification. Peter Pathe was heading Word at the time and wanted to improve Word’s line layout. He figured that Eliyezer could get a team going to do this in time for Word 97.

Eliyezer was convinced he could factor out Word’s line layout, but because of Word’s mind-boggling backward compatibility requirements, he refused to agree to the Office 97 time frame. I was working on RichEdit 2.0 next door to Eliyezer back then and you should have heard the prolonged arguments between Eliyezer and Peter! In that time frame, Eliyezer had a native American working with him named Lennox Brassel. Then he hired his first St. Petersburg mathematician, Sergey Genkin. Sergey’s first job after arriving in the USA in 1990 was back East working on a TeX compatible system. The team developed LineServices 1.0, which shipped first with a little program call Greeting Cards.

Eliyezer needed more developer cycles, so he asked Sergey if knew any more smart software engineers back in St. Petersburg. Sure enough Igor Zverev could come and RichEdit was fortunate enough to have Igor’s services for a while in developing RichEdit 2.0. (RichEdit also had another St. Petersburg mathematician Andrei Burago for some of that time; more about Andrei in a bit…). Not long after, yet another St. Petersburg mathematician Victor Kozyrev joined the team.

LineServices 2.0 was developed and shipped with Word 2000, Internet Explorer 4.0, RichEdit 3.0, PowerPoint 2000, and Publisher 2000. In addition to Western text, LineServices supported several special layout objects: the reverse object for BiDi text, the ruby object for East Asian phonetic annotations, Tatenakayoko (Japanese for “horizontal in vertical”, but sounds better than HIV), and warichu (two lines in one). From my point of view ruby was just a fraction in disguise, so naturally I was very excited to see that LineServices could do such a beautiful job displaying ruby. The initial ruby handler was developed by Rick Sailor.

LineServices handles lots of tricky problems with text layout and calls the client back for all the information it needs. LineServices never makes any operating system calls directly and it can run on any operating system that has a C++ compiler. The team had the strange habit of seriously designing a product before ever writing one line of code. What’s even stranger is that when they finally wrote the code, it had very few bugs in it. My own approach is to design partially and then dive into writing code using the well-known physicist approach to evaluating things called successive approximations. I can’t figure out everything in advance, so I try something and then gradually improve on it. Those guys figured out most of the design without writing any code.

After Office 2000, Eliyezer & Co. embarked on Page/TableServices, which was natural. He had started with characters in implementing TrueType, then progressed to lines with LineServices, and so pages and table were the next items on the layout hierarchy. To pull that off he needed the help of another St. Petersburg mathematician, Anton Sukanov, who had been a whiz kid in Russia winning various computer-science puzzle competitions. So, the team developed PTS as it’s called and revised LineServices to work well with it.

About that time, I simply couldn’t stand not having some native math layout in our products, so in February 2001 I wrote a math handler for LineServices patterned after the ruby handler. While I was at it, I installed the ruby object in a recursive way, so that you could have ruby objects nested like continued fractions. This upset the authors of the HTML ruby specification, since they said ruby was not supposed to be nested. Nevertheless, the easiest way to display complex ruby is using one level of nesting. My simple math handler showed that LineServices could do mathematics, although my spacing was mediocre. More precisely, spacing was delegated to the user, who unfortunately seldom knows what correct math spacing is.

A valuable thing about my old LineServices math handler was that it convinced people that we had the infrastructure to layout math. Fortunately, I didn’t appreciate at the time how hard laying out TeX-quality math would prove to be. Else I might not have been able to persuade people to work on it. It seems that most things that are worthwhile are way harder than you think they’ll be when you start working on them.

Of course, Eliyezer didn’t need any convincing; he and his team had designed LineServices so that it could be used to layout math. They just didn’t tell anyone! So after Office 2003 shipped, Eliyezer convinced Bill Gates and Steven Sinofsky that we should handle math layout and he started studying mathematical typography in earnest. He got his hands on every math typography book he could find, but by far the most influential and useful was Donald Knuth’s The TeXbook. After a few months Eliyezer enlisted the help of Victor and Andrei to design a math handler. Andrei had been working in the Windows font group, a background that would prove to be incredibly helpful on the math project.

They’d often come into my office announcing they were planning to do this and that and I would sometimes protest that what they had in mind wasn’t workable from a client’s, e.g., RichEdit’s, point of view. So they’d revise the plan and eventually they had a math handler design that was compatible with my general approach and offered really high quality mathematical typography. While this was going on, I was developing the formula autobuildup input method (see OfficeMath UI) using my old math handler, since they didn’t have any code written.

Since TeX was so valuable in the design process, Eliyezer wanted to talk with Donald Knuth, who happened to be an old friend of Eliyezer’s PhD advisor, Nicolas Wirth. A visit was arranged in November 2003 and the four of us had the good fortune to spend an extraordinary afternoon with Knuth at his home on the Stanford University campus. Among many things, Donald showed us how he uses TeX to typeset his papers and books exactly the way he wants them to look. He applies special tweaks to achieve perfection, such as “smashing the descender” on one radicand to make a sum of square roots line up in a pleasing way and shimming characters to place them more beautifully in a formula. This interaction inspired us to think that we could automate some of Donald’s tweaks using what evolved into the OpenType math tables and associated code, such as “cut-ins” to kern superscripts and subscripts with their bases.

Eliyezer’s health gradually declined, and he decided to retire after the initial math-handler design. Sergey Genkin took over leadership of the math project. One day in the summer of 2004 they came into my office all excited and announced that they had been able to display the mathematical expression 𝑎 + 𝑏! It was a real achievement, since the spacing around the + was the desired 4/18^th em and a lot of code had checked out correctly. One of the things they soon discovered was that LineServices alone was not adequate to layout high quality mathematics: you need PTS too!

The problem is that on computer screens, unlike the printed page, the layout width varies substantially from one invocation to another. Hence you must be able to break equations to accommodate different window widths. TeX delegates most equation breaking to the user, but that’s not a good option for browsers, slide shows and other screen usages. Also, you need PTS to handle placement of equation numbers. Yet another brilliant St. Petersburg mathematician had joined the PTS team, namely Alexander Vaschillo. So, he and Anton implemented equation breaking and numbering.

At this point one can understand better how we came to use OMML (Office MathML) as a file format for mathematics rather than MathML. OMML is a close representation of the LineServices/PTS math objects. These math objects were created after extensive study of mathematical typography, rather than by study of MathML. It’s natural to have a file format that mirrors the internal format. In addition, we needed to be able to put any inline text and objects inside math zones. MathML cannot embed other XML namespaces except indirectly via parallel markup of some kind.

Murray Sargent Principal Software Engineer, CXE (Office Shared)

Follow