Writing good documentation is hard. Tools can’t solve this problem in themselves, but they can ease the pain. This post will show you how to use Sphinx to generate attractive, functional documentation for C++ libraries, supplied with information from Doxygen. We’ll also integrate this process into a CMake build system so that we have a unified workflow.

For an example of a real-world project whose documentation is built like this, see fmtlib.

Why Sphinx?

Doxygen has been around for a couple of decades and is a stable, feature-rich tool for generating documentation. However, it is not without its issues. Docs generated with Doxygen tend to be visually noisy, have a style out of the early nineties, and struggle to clearly represent complex template-based APIs. There are also limitations to its markup. Although they added Markdown support in 2012, Markdown is simply not the best tool for writing technical documentation since it sacrifices extensibility, featureset size, and semantic markup for simplicity.

Sphinx instead uses reStructuredText, which has those important concepts which are missing from Markdown as core ideals. One can add their own “roles” and “directives” to the markup to make domain-specific customizations. There are some great comparisons of reStructuredText and Markdown by Victor Zverovich and Eli Bendersky if you’d like some more information.

The docs generated by Sphinx also look a lot more modern and minimal when compared to Doxygen and it’s much easier to swap in a different theme, customize the amount of information which is displayed, and modify the layout of the pages.

 

On a more fundamental level, Doxygen’s style of documentation is listing out all the API entities along with their associated comments in a more digestible, searchable manner. It’s essentially paraphrasing the header files, to take a phrase from Robert Ramey^[1]; embedding things like rationale, examples, notes, or swapping out auto-generated output for hand-written is not very well supported. In Sphinx however, the finer-grained control gives you the ability to write documentation which is truly geared towards getting people to learn and understand your library.

If you’re convinced that this is a good avenue to explore, then we can begin by installing dependencies.

Install Dependencies

Doxygen

Sphinx doesn’t have the ability to extract API documentation from C++ headers; this needs to be supplied either by hand or from some external tool. We can use Doxygen to do this job for us. Grab it from the official download page and install it. There are binaries for Windows, Linux (compiled on Ubuntu 16.04), and MacOS, alongside source which you can build yourself.

Sphinx

Pick your preferred way of installing Sphinx from the official instructions. It may be available through your system package manager, or you can get it through pip.

Read the Docs Sphinx Theme

I prefer this theme to the built-in ones, so we can install it through pip:

> pip install sphinx_rtd_theme

Breathe

Breathe is the bridge between Doxygen and Sphinx; taking the output from the former and making it available through some special directives in the latter. You can install it with pip:

> pip install breathe

CMake

Install the latest release of CMake. If you are using Visual Studio 2017 and up, you will already have a version installed and ready to use. See CMake projects in Visual Studio for more details.

Create a CMake Project

All of the code for this post is available on Github, so if you get stuck, have a look there.

If you are using Visual Studio 2017 and up, go to File > New > Project and create a CMake project.

 

Regardless of which IDE/editor you are using, get your project folder to look something like this:

CatCutifier/CMakeLists.txt

cmake_minimum_required (VERSION 3.8)

project ("CatCutifier")

add_subdirectory ("CatCutifier")

CatCutifier/CatCutifier/CatCutifier.cpp

#include "CatCutifier.h"

void cat::make_cute() {
  // Magic happens
}

CatCutifier/CatCutifier/CatCutifier.h

#pragma once

/**
  A fluffy feline
*/
struct cat {
  /**
    Make this cat look super cute
  */
  void make_cute();
};

CatCutifier/CatCutifier/CMakeLists.txt

add_library (CatCutifier "CatCutifier.cpp" "CatCutifier.h")

target_include_directories(CatCutifier PUBLIC .)

If you now build your project, you should get a CatCutifier library which someone could link against and use.

Now that we have our library, we can set up document generation.

Set up Doxygen

If you don’t already have Doxygen set up for your project, you’ll need to generate a configuration file so that it knows how to generate docs for your interfaces. Make sure the Doxygen executable is on your path and run:

> mkdir docs
> cd docs
> doxygen.exe -g

You should get a message like:

Configuration file `Doxyfile' created.
Now edit the configuration file and enter
  doxygen Doxyfile
to generate the documentation for your project

We can get something generated quickly by finding the INPUT variable in the generated Doxyfile and pointing it at our code:

INPUT = ../CatCutifier

Now if you run:

> doxygen.exe

You should get an html folder generated which you can point your browser at and see some documentation like this:

We’ve successfully generated some simple documentation for our class by hand. But we don’t want to manually run this command every time we want to rebuild the docs; this should be handled by CMake.

Doxygen in CMake

To use Doxygen from CMake, we need to find the executable. Fortunately CMake provides a find module for Doxygen, so we can use find_package(Doxygen REQUIRED) to locate the binary and report an error if it doesn’t exist. This will store the executable location in the DOXYGEN_EXECUTABLE variable, so we can add_custom_command to run it and track dependencies properly:

CatCutifier/CMakeLists.txt

cmake_minimum_required (VERSION 3.8)
project ("CatCutifier")
add_subdirectory ("CatCutifier")
add_subdirectory ("docs")

CatCutifier/docs/CMakeLists.txt

find_package(Doxygen REQUIRED)

# Find all the public headers
get_target_property(CAT_CUTIFIER_PUBLIC_HEADER_DIR CatCutifier INTERFACE_INCLUDE_DIRECTORIES)
file(GLOB_RECURSE CAT_CUTIFIER_PUBLIC_HEADERS ${CAT_CUTIFIER_PUBLIC_HEADER_DIR}/*.h)

#This will be the main output of our command
set(DOXYGEN_INDEX_FILE ${CMAKE_CURRENT_SOURCE_DIR}/html/index.html)

add_custom_command(OUTPUT ${DOXYGEN_INDEX_FILE}
                   DEPENDS ${CAT_CUTIFIER_PUBLIC_HEADERS}
                   COMMAND ${DOXYGEN_EXECUTABLE} Doxyfile
                   WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
                   MAIN_DEPENDENCY Doxyfile
                   COMMENT "Generating docs")

add_custom_target(Doxygen ALL DEPENDS ${DOXYGEN_INDEX_FILE})

The final custom target makes sure that we have a target name to give to make and that dependencies will be checked for a rebuild whenever we Build All or do a bare make.

We also want to be able to control the input and output directories from CMake so that we’re not flooding our source directory with output files. We can do this by adding some placeholders to our Doxyfile (we’ll rename it Doxyfile.in to follow convention) and having CMake fill them in with configure_file:

CatCutifier/docs/Doxyfile.in

#...
INPUT = "@DOXYGEN_INPUT_DIR@"
#...
OUTPUT_DIRECTORY = "@DOXYGEN_OUTPUT_DIR@"
#...

CatCutifier/docs/CMakeLists.txt

find_package(Doxygen REQUIRED)

# Find all the public headers
get_target_property(CAT_CUTIFIER_PUBLIC_HEADER_DIR CatCutifier INTERFACE_INCLUDE_DIRECTORIES)
file(GLOB_RECURSE CAT_CUTIFIER_PUBLIC_HEADERS ${CAT_CUTIFIER_PUBLIC_HEADER_DIR}/*.h)

set(DOXYGEN_INPUT_DIR ${PROJECT_SOURCE_DIR}/CatCutifier)
set(DOXYGEN_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/docs/doxygen)
set(DOXYGEN_INDEX_FILE ${DOXYGEN_OUTPUT_DIR}/html/index.html)
set(DOXYFILE_IN ${CMAKE_CURRENT_SOURCE_DIR}/Doxyfile.in)
set(DOXYFILE_OUT ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile)

#Replace variables inside @@ with the current values
configure_file(${DOXYFILE_IN} ${DOXYFILE_OUT} @ONLY)

file(MAKE_DIRECTORY ${DOXYGEN_OUTPUT_DIR}) #Doxygen won't create this for us
add_custom_command(OUTPUT ${DOXYGEN_INDEX_FILE}
                   DEPENDS ${CAT_CUTIFIER_PUBLIC_HEADERS}
                   COMMAND ${DOXYGEN_EXECUTABLE} ${DOXYFILE_OUT}
                   MAIN_DEPENDENCY ${DOXYFILE_OUT} ${DOXYFILE_IN}
                   COMMENT "Generating docs")

add_custom_target(Doxygen ALL DEPENDS ${DOXYGEN_INDEX_FILE})

Now we can generate our documentation as part of our build system and it’ll only be generated when it needs to be. If you’re happy with Doxygen’s output, you could just stop here, but if you want the additional features and attractive output which reStructuredText and Sphinx give you, then read on.

Setting up Sphinx

Sphinx provides a nice startup script to get us going fast. Go ahead and run this:

> cd docs
> sphinx-quickstart.exe

Keep the defaults and put in your name and the name of your project. Now if you run make html you should get a _build/html folder you can point your browser at to see a welcome screen.

I’m a fan of the Read the Docs theme we installed at the start, so we can use that instead by changing html_theme in conf.py to be ‘sphinx_rtd_theme’. That gives us this look:

Before we link in the Doxygen output to give us the documentation we desire, lets automate the Sphinx build with CMake

Sphinx in CMake

Ideally we want to be able to write find_package(Sphinx REQUIRED) and have everything work. Unfortunately, unlike Doxygen, Sphinx doesn’t have a find module provided by default, so we’ll need to write one. Fortunately, we can get away with doing very little work:

CatCutifier/cmake/FindSphinx.cmake

#Look for an executable called sphinx-build
find_program(SPHINX_EXECUTABLE
             NAMES sphinx-build
             DOC "Path to sphinx-build executable")

include(FindPackageHandleStandardArgs)

#Handle standard arguments to find_package like REQUIRED and QUIET
find_package_handle_standard_args(Sphinx
                                  "Failed to find sphinx-build executable"
                                  SPHINX_EXECUTABLE)

With this file in place, find_package will work so long as we tell CMake to look for find modules in that directory:

CatCutifier/CMakeLists.txt

cmake_minimum_required (VERSION 3.8)

project ("CatCutifier")

# Add the cmake folder so the FindSphinx module is found
set(CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake" ${CMAKE_MODULE_PATH})

add_subdirectory ("CatCutifier")
add_subdirectory ("docs")

Now we can find this executable and call it:

CatCutifier/docs/CMakeLists.txt

find_package(Sphinx REQUIRED)

set(SPHINX_SOURCE ${CMAKE_CURRENT_SOURCE_DIR})
set(SPHINX_BUILD ${CMAKE_CURRENT_BINARY_DIR}/docs/sphinx)

add_custom_target(Sphinx ALL
                  COMMAND
                  ${SPHINX_EXECUTABLE} -b html
                  ${SPHINX_SOURCE} ${SPHINX_BUILD}
                  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
                  COMMENT "Generating documentation with Sphinx")

If you run a build you should now see Sphinx running and generating the same blank docs we saw earlier.

Now we have the basics set up, we need to hook Sphinx up with the information generated by Doxygen. We do that using Breathe.

Setting up Breathe

Breathe is an extension to Sphinx, so we set it up using the conf.py which was generated for us in the last step:

CatCutifier/docs/conf.py

#...
extensions = [ "breathe" ]
#...

# Breathe Configuration
breathe_default_project = "CatCutifier"

Breathe uses Doxygen’s XML output, which is disabled by default, so we need to turn it on:

CatCutifier/docs/Doxyfile.in

#...
GENERATE_XML = YES
#...

We’ll need to put placeholders in our docs to tell Sphinx where to put our API information. We achieve this with directives supplied by Breathe, such as doxygenstruct:

CatCutifier/docs/index.rst

…

Docs
====

.. doxygenstruct:: cat
   :members:

You might wonder why it’s necessary to explicitly state what entities we wish to document and where, but this is one of the key benefits of Sphinx. This allows us to add as much additional information (examples, rationale, notes, etc.) as we want to the documentation without having to shoehorn it into the source code, plus we can make sure it’s displayed in the most accessible, understandable manner we can. Have a look through Breathe’s directives and Sphinx’s built-in directives, and Sphinx’s C++-specific directives to get a feel for what’s available.

Now we update our Sphinx target to hook it all together by telling Breathe where to find the Doxygen output:

CatCutifier/docs/CMakeLists.txt

#...

find_package(Sphinx REQUIRED)

set(SPHINX_SOURCE ${CMAKE_CURRENT_SOURCE_DIR})
set(SPHINX_BUILD ${CMAKE_CURRENT_BINARY_DIR}/docs/sphinx)

add_custom_target(Sphinx ALL
                  COMMAND ${SPHINX_EXECUTABLE} -b html
                  # Tell Breathe where to find the Doxygen output
                  -Dbreathe_projects.CatCutifier=${DOXYGEN_OUTPUT_DIR}
                  ${SPHINX_SOURCE} ${SPHINX_BUILD}
                  WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
                  COMMENT "Generating documentation with Sphinx")

Hooray! You should now have some nice Sphinx documentation generated for you:

Finally, we can make sure all of our dependencies are right so that we never rebuild the Doxygen files or the Sphinx docs when we don’t need to:

CatCutifier/docs/CMakeLists.txt

find_package(Doxygen REQUIRED)
find_package(Sphinx REQUIRED)

# Find all the public headers
get_target_property(CAT_CUTIFIER_PUBLIC_HEADER_DIR CatCutifier INTERFACE_INCLUDE_DIRECTORIES)
file(GLOB_RECURSE CAT_CUTIFIER_PUBLIC_HEADERS ${CAT_CUTIFIER_PUBLIC_HEADER_DIR}/*.h)

set(DOXYGEN_INPUT_DIR ${PROJECT_SOURCE_DIR}/CatCutifier)
set(DOXYGEN_OUTPUT_DIR ${CMAKE_CURRENT_BINARY_DIR}/doxygen)
set(DOXYGEN_INDEX_FILE ${DOXYGEN_OUTPUT_DIR}/xml/index.xml)
set(DOXYFILE_IN ${CMAKE_CURRENT_SOURCE_DIR}/Doxyfile.in)
set(DOXYFILE_OUT ${CMAKE_CURRENT_BINARY_DIR}/Doxyfile)

# Replace variables inside @@ with the current values
configure_file(${DOXYFILE_IN} ${DOXYFILE_OUT} @ONLY)

# Doxygen won't create this for us
file(MAKE_DIRECTORY ${DOXYGEN_OUTPUT_DIR})

# Only regenerate Doxygen when the Doxyfile or public headers change
add_custom_command(OUTPUT ${DOXYGEN_INDEX_FILE}
                   DEPENDS ${CAT_CUTIFIER_PUBLIC_HEADERS}
                   COMMAND ${DOXYGEN_EXECUTABLE} ${DOXYFILE_OUT}
                   MAIN_DEPENDENCY ${DOXYFILE_OUT} ${DOXYFILE_IN}
                   COMMENT "Generating docs"
                   VERBATIM)

# Nice named target so we can run the job easily
add_custom_target(Doxygen ALL DEPENDS ${DOXYGEN_INDEX_FILE})

set(SPHINX_SOURCE ${CMAKE_CURRENT_SOURCE_DIR})
set(SPHINX_BUILD ${CMAKE_CURRENT_BINARY_DIR}/sphinx)
set(SPHINX_INDEX_FILE ${SPHINX_BUILD}/index.html)

# Only regenerate Sphinx when:
# - Doxygen has rerun
# - Our doc files have been updated
# - The Sphinx config has been updated
add_custom_command(OUTPUT ${SPHINX_INDEX_FILE}
                   COMMAND 
                     ${SPHINX_EXECUTABLE} -b html
                     # Tell Breathe where to find the Doxygen output
                     -Dbreathe_projects.CatCutifier=${DOXYGEN_OUTPUT_DIR}/xml
                   ${SPHINX_SOURCE} ${SPHINX_BUILD}
                   WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}
                   DEPENDS
                   # Other docs files you want to track should go here (or in some variable)
                   ${CMAKE_CURRENT_SOURCE_DIR}/index.rst
                   ${DOXYGEN_INDEX_FILE}
                   MAIN_DEPENDENCY ${SPHINX_SOURCE}/conf.py
                   COMMENT "Generating documentation with Sphinx")

# Nice named target so we can run the job easily
add_custom_target(Sphinx ALL DEPENDS ${SPHINX_INDEX_FILE})

# Add an install target to install the docs
include(GNUInstallDirs)
install(DIRECTORY ${SPHINX_BUILD}
DESTINATION ${CMAKE_INSTALL_DOCDIR})

Try it out and see what gets rebuilt when you change a file. If you change Doxyfile.in or a header file, all the docs should get rebuilt, but if you only change the Sphinx config or reStructuredText files then the Doxygen build should get skipped.

This leaves us with an efficient, automated, powerful documentation system.

If you already have somewhere to host the docs or want developers to build the docs themselves then we’re finished. If not, you can host them on Read the Docs, which provides free hosting for open source projects.

Setting up Read the Docs

To use Read the Docs (RtD) you need to sign up (you can use GitHub, GitLab or Bitbucket to make integration easy). Log in, import your repository, and your docs will begin to build!

Unfortunately, it will also fail:

Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/cpp-documentation-example/envs/latest/lib/python3.7/site-packages/sphinx/registry.py", line 472, in load_extension mod = __import__(extname, None, None, ['setup']) ModuleNotFoundError: No module named 'breathe'

To tell RtD to install Breathe before building, we can add a requirements file:

CatCutifier/docs/requirements.txt

breathe

Another issue is that RtD doesn’t understand CMake: it’s finding the Sphinx config file and running that, so it won’t generate the Doxygen information. To generate this, we can add some lines to our conf.py script to check if we’re running in on the RtD servers and, if so, hardcode some paths and run Doxygen:

CatCutifier/docs/conf.py

import subprocess, os

def configureDoxyfile(input_dir, output_dir):
    with open('Doxyfile.in', 'r') as file :
        filedata = file.read()

    filedata = filedata.replace('@DOXYGEN_INPUT_DIR@', input_dir)
    filedata = filedata.replace('@DOXYGEN_OUTPUT_DIR@', output_dir)

    with open('Doxyfile', 'w') as file:
        file.write(filedata)

# Check if we're running on Read the Docs' servers
read_the_docs_build = os.environ.get('READTHEDOCS', None) == 'True'

breathe_projects = {}

if read_the_docs_build:
    input_dir = '../CatCutifier'
    output_dir = 'build'
    configureDoxyfile(input_dir, output_dir)
    subprocess.call('doxygen', shell=True)
    breathe_projects['CatCutifier'] = output_dir + '/xml'

# ...

Push this change and…

Lovely documentation built automatically on every commit.

Conclusion

All this tooling takes a fair amount of effort to set up, but the result is powerful, expressive, and accessible. None of this is a substitute for clear writing and a strong grasp of what information a user of a library needs to use it effectively, but our new system can provide support to make this easier for developers.

Resources

Thank you to the authors and presenters of these resources, which were very helpful in putting together this post and process:

https://vicrucann.github.io/tutorials/quick-cmake-doxygen/

https://eb2.co/blog/2012/03/sphinx-and-cmake-beautiful-documentation-for-c—projects/

https://nazavode.github.io/blog/cmake-doxygen-improved/

http://www.zverovich.net/2016/06/16/rst-vs-markdown.html

https://eli.thegreenplace.net/2017/restructuredtext-vs-markdown-for-technical-documentation/

https://www.youtube.com/watch?v=YxmdCxX9dMk

1. I would highly recommend watching this talk to help you think about what you put in your documentation. ↑