Clear, Functional C++ Documentation with Sphinx + Breathe + Doxygen + CMake

Sy Brand

Sy

Writing good documentation is hard. Tools can’t solve this problem in themselves, but they can ease the pain. This post will show you how to use Sphinx to generate attractive, functional documentation for C++ libraries, supplied with information from Doxygen. We’ll also integrate this process into a CMake build system so that we have a unified workflow.

For an example of a real-world project whose documentation is built like this, see fmtlib.

Why Sphinx?

Doxygen has been around for a couple of decades and is a stable, feature-rich tool for generating documentation. However, it is not without its issues. Docs generated with Doxygen tend to be visually noisy, have a style out of the early nineties, and struggle to clearly represent complex template-based APIs. There are also limitations to its markup. Although they added Markdown support in 2012, Markdown is simply not the best tool for writing technical documentation since it sacrifices extensibility, featureset size, and semantic markup for simplicity.

Sphinx instead uses reStructuredText, which has those important concepts which are missing from Markdown as core ideals. One can add their own “roles” and “directives” to the markup to make domain-specific customizations. There are some great comparisons of reStructuredText and Markdown by Victor Zverovich and Eli Bendersky if you’d like some more information.

The docs generated by Sphinx also look a lot more modern and minimal when compared to Doxygen and it’s much easier to swap in a different theme, customize the amount of information which is displayed, and modify the layout of the pages.

Doxygen's output, which has a lot of boilerplate and unused space
Doxygen output
Output from Sphinx, which is much more compact and attractive
Sphinx output

 

On a more fundamental level, Doxygen’s style of documentation is listing out all the API entities along with their associated comments in a more digestible, searchable manner. It’s essentially paraphrasing the header files, to take a phrase from Robert Ramey[1]; embedding things like rationale, examples, notes, or swapping out auto-generated output for hand-written is not very well supported. In Sphinx however, the finer-grained control gives you the ability to write documentation which is truly geared towards getting people to learn and understand your library.

If you’re convinced that this is a good avenue to explore, then we can begin by installing dependencies.

Install Dependencies

Doxygen

Sphinx doesn’t have the ability to extract API documentation from C++ headers; this needs to be supplied either by hand or from some external tool. We can use Doxygen to do this job for us. Grab it from the official download page and install it. There are binaries for Windows, Linux (compiled on Ubuntu 16.04), and MacOS, alongside source which you can build yourself.

Sphinx

Pick your preferred way of installing Sphinx from the official instructions. It may be available through your system package manager, or you can get it through pip.

Read the Docs Sphinx Theme

I prefer this theme to the built-in ones, so we can install it through pip:

Breathe

Breathe is the bridge between Doxygen and Sphinx; taking the output from the former and making it available through some special directives in the latter. You can install it with pip:

CMake

Install the latest release of CMake. If you are using Visual Studio 2017 and up, you will already have a version installed and ready to use. See CMake projects in Visual Studio for more details.

Create a CMake Project

All of the code for this post is available on Github, so if you get stuck, have a look there.

If you are using Visual Studio 2017 and up, go to File > New > Project and create a CMake project.

Create new CMake project dialogue box

 

Regardless of which IDE/editor you are using, get your project folder to look something like this:

CatCutifier/CMakeLists.txt

CatCutifier/CatCutifier/CatCutifier.cpp

CatCutifier/CatCutifier/CatCutifier.h

CatCutifier/CatCutifier/CMakeLists.txt

If you now build your project, you should get a CatCutifier library which someone could link against and use.

Now that we have our library, we can set up document generation.

Set up Doxygen

If you don’t already have Doxygen set up for your project, you’ll need to generate a configuration file so that it knows how to generate docs for your interfaces. Make sure the Doxygen executable is on your path and run:

You should get a message like:

We can get something generated quickly by finding the INPUT variable in the generated Doxyfile and pointing it at our code:

Now if you run:

You should get an html folder generated which you can point your browser at and see some documentation like this:

Doxygen's output, which has a lot of boilerplate and unused space

We’ve successfully generated some simple documentation for our class by hand. But we don’t want to manually run this command every time we want to rebuild the docs; this should be handled by CMake.

Doxygen in CMake

To use Doxygen from CMake, we need to find the executable. Fortunately CMake provides a find module for Doxygen, so we can use find_package(Doxygen REQUIRED) to locate the binary and report an error if it doesn’t exist. This will store the executable location in the DOXYGEN_EXECUTABLE variable, so we can add_custom_command to run it and track dependencies properly:

CatCutifier/CMakeLists.txt

CatCutifier/docs/CMakeLists.txt

The final custom target makes sure that we have a target name to give to make and that dependencies will be checked for a rebuild whenever we Build All or do a bare make.

We also want to be able to control the input and output directories from CMake so that we’re not flooding our source directory with output files. We can do this by adding some placeholders to our Doxyfile (we’ll rename it Doxyfile.in to follow convention) and having CMake fill them in with configure_file:

CatCutifier/docs/Doxyfile.in

CatCutifier/docs/CMakeLists.txt

Now we can generate our documentation as part of our build system and it’ll only be generated when it needs to be. If you’re happy with Doxygen’s output, you could just stop here, but if you want the additional features and attractive output which reStructuredText and Sphinx give you, then read on.

Setting up Sphinx

Sphinx provides a nice startup script to get us going fast. Go ahead and run this:

Keep the defaults and put in your name and the name of your project. Now if you run make html you should get a _build/html folder you can point your browser at to see a welcome screen.

Front page saying "Welcome to CatCutifier's documentation with links to the Index, Module Index and Search Page

I’m a fan of the Read the Docs theme we installed at the start, so we can use that instead by changing html_theme in conf.py to be ‘sphinx_rtd_theme’. That gives us this look:

The same content as above, but the visual design is more attractive

Before we link in the Doxygen output to give us the documentation we desire, lets automate the Sphinx build with CMake

Sphinx in CMake

Ideally we want to be able to write find_package(Sphinx REQUIRED) and have everything work. Unfortunately, unlike Doxygen, Sphinx doesn’t have a find module provided by default, so we’ll need to write one. Fortunately, we can get away with doing very little work:

CatCutifier/cmake/FindSphinx.cmake

With this file in place, find_package will work so long as we tell CMake to look for find modules in that directory:

CatCutifier/CMakeLists.txt

Now we can find this executable and call it:

CatCutifier/docs/CMakeLists.txt

If you run a build you should now see Sphinx running and generating the same blank docs we saw earlier.

Now we have the basics set up, we need to hook Sphinx up with the information generated by Doxygen. We do that using Breathe.

Setting up Breathe

Breathe is an extension to Sphinx, so we set it up using the conf.py which was generated for us in the last step:

CatCutifier/docs/conf.py

Breathe uses Doxygen’s XML output, which is disabled by default, so we need to turn it on:

CatCutifier/docs/Doxyfile.in

We’ll need to put placeholders in our docs to tell Sphinx where to put our API information. We achieve this with directives supplied by Breathe, such as doxygenstruct:

CatCutifier/docs/index.rst

You might wonder why it’s necessary to explicitly state what entities we wish to document and where, but this is one of the key benefits of Sphinx. This allows us to add as much additional information (examples, rationale, notes, etc.) as we want to the documentation without having to shoehorn it into the source code, plus we can make sure it’s displayed in the most accessible, understandable manner we can. Have a look through Breathe’s directives and Sphinx’s built-in directives, and Sphinx’s C++-specific directives to get a feel for what’s available.

Now we update our Sphinx target to hook it all together by telling Breathe where to find the Doxygen output:

CatCutifier/docs/CMakeLists.txt

Hooray! You should now have some nice Sphinx documentation generated for you:

Output from Sphinx, which is much more compact and attractive

Finally, we can make sure all of our dependencies are right so that we never rebuild the Doxygen files or the Sphinx docs when we don’t need to:

CatCutifier/docs/CMakeLists.txt

Try it out and see what gets rebuilt when you change a file. If you change Doxyfile.in or a header file, all the docs should get rebuilt, but if you only change the Sphinx config or reStructuredText files then the Doxygen build should get skipped.

This leaves us with an efficient, automated, powerful documentation system.

If you already have somewhere to host the docs or want developers to build the docs themselves then we’re finished. If not, you can host them on Read the Docs, which provides free hosting for open source projects.

Setting up Read the Docs

To use Read the Docs (RtD) you need to sign up (you can use GitHub, GitLab or Bitbucket to make integration easy). Log in, import your repository, and your docs will begin to build!

Unfortunately, it will also fail:

To tell RtD to install Breathe before building, we can add a requirements file:

CatCutifier/docs/requirements.txt

Another issue is that RtD doesn’t understand CMake: it’s finding the Sphinx config file and running that, so it won’t generate the Doxygen information. To generate this, we can add some lines to our conf.py script to check if we’re running in on the RtD servers and, if so, hardcode some paths and run Doxygen:

CatCutifier/docs/conf.py

Push this change and…

Full documentation page built on read the docs

Lovely documentation built automatically on every commit.

Conclusion

All this tooling takes a fair amount of effort to set up, but the result is powerful, expressive, and accessible. None of this is a substitute for clear writing and a strong grasp of what information a user of a library needs to use it effectively, but our new system can provide support to make this easier for developers.

Resources

Thank you to the authors and presenters of these resources, which were very helpful in putting together this post and process:

https://vicrucann.github.io/tutorials/quick-cmake-doxygen/

https://eb2.co/blog/2012/03/sphinx-and-cmake-beautiful-documentation-for-c—projects/

https://nazavode.github.io/blog/cmake-doxygen-improved/

http://www.zverovich.net/2016/06/16/rst-vs-markdown.html

https://eli.thegreenplace.net/2017/restructuredtext-vs-markdown-for-technical-documentation/

https://www.youtube.com/watch?v=YxmdCxX9dMk

  1. I would highly recommend watching this talk to help you think about what you put in your documentation.

 

Sy Brand
Sy Brand

Follow Sy   

3 comments

  • Avatar
    Patrick Polzin

    After reading these many instructions, I prefer to stick to my XML documentation and Atomineer Utils. Goes fast and is clean.
    It would also be nicer if Microsoft again cares more about Windows and its API – I’m just saying Raw Input, Device Management and USB HID – than about CMake and OS integration.

  • Avatar
    Tim Weis

    Lord! I’ve been waiting for this blog post for… forever. I’d been wanting to use Sphinx and reStructured Text for ages, but never quite could bridge the C++ gap. Delegating the C++-iness to Doxygen, and leaving the processing and layouting to Python is a great combination. Not new, maybe, but certainly new to me. Full CMake integration tops it all off.
    Thanks for a well written, well-paced blog post, that solves 2 of my most pressing issues on any project: Which documentation system to choose, and how to automate documentation generation.

  • Avatar
    cong feng

    You are using configure_file() in this post to generate Doxyfile. In fact, the Doxygen module provided by cmake also provides a
    doxygen_add_docs() function which will do all the job you just coded. You can also set DOXYGEN_ variables directly, which will beeffective in the generated Doxyfile. See this post andcmake official docs.

Leave a comment