Documenting Multiple Programming Language APIs with Sphinx

The Goal

I wrote before about using Sphinx to generate documentation in multiple human languages, but Sphinx can also generate documentation for APIs implemented in multiple programming languages. Most programming languages have some kind of API documentation generation tool (either bundled with the language implementation or provided as a separate utility) for documenting software written in that language: Javadoc, JSDoc, RDoc, Doxygen, etc. These are really useful when you’re primarily working in a single programming language, but most of them start to show some limits in a multiple-language software project. If you’re writing a Django web application (Python) with a rich client UI (JavaScript) that leverages existing web services code (Java), then resources and flow of execution can be shared between languages. In this case, it’s nice to have a single searchable source of technical documentation, and with a little configuration Sphinx can do this.

Python

Sphinx was originally written as a tool for writing the documentation of Python itself, so it stands to reason that it has very good support for generating Python API documentation (in fact, it’s pretty much Python’s official tool for this purpose). The documentation for Python and Django are examples that many other projects follow, encouraging a style of documentation which reads more like a technical book than a raw listing of class and function descriptions. Here’s an example (from Django) of the type of wiki markup Sphinx uses for this:

Available ``Meta`` options
==========================

.. currentmodule:: django.db.models

``abstract``
------------

.. attribute:: Options.abstract

    If ``abstract = True``, this model will be an
    :ref:`abstract base class `.

Sphinx also includes utilities for auto-generating partial (autodoc) or complete (sphinx-apidoc) documentation for a Python API, fetching descriptions of each item from docstrings in the source code when available. For example, to generate reST files which describe all the Python code under a particular directory:

sphinx-apidoc -f -o docs/python src

This doesn’t take much work, and generates something that looks a little more like Javadoc output. Speaking of which…

Java

Javadoc was one of the first tools to really popularize in-source-code API documentation. There’s been some debate over whether this is really a good way to write the main API documentation for a software project, but at any rate, most decent Java projects include fairly complete documentation right in the source code. Using the Javadoc tool is by far the most common way to generate HTML documentation from these source code comments, but they can be used to generate Sphinx documentation as well. There’s a Sphinx extension called javasphinx which includes a tool to parse these comments and generate Sphinx reST files from them. Usage (after installing and configuring the extension as described in its own Sphinx-based documentation) is very similar to the equivalent Python utility described above:

javasphinx-apidoc -f -o docs/java src

And if you prefer the book-like style used for Sphinx documentation for projects like Python and Django, javasphinx provides a “domain” of reST markup extensions that can be used to describe a Java API in conjunction with wiki-formatted prose:

.. java:type:: public interface List extends Collection, Iterable

   An ordered collection (also known as a *sequence*)

   :param E: type of item stored by the list

(Example stolen shamelessly from the javasphinx documentation.)

The output may not be amazing enough to convince a Java development team to switch over from Javadoc output, but it has the big advantages of being combinable with other Sphinx source files (either hand-written or generated from other programming languages) and allowing generation of output formats other than HTML (such as PDF, EPUB, LaTeX, etc.) Additionally, the same techniques I described in my previous blog post for using gettext and Transifex to translate Sphinx documentation into multiple written languages can be used to translate the generated API documentation as well. The JavaScript search engine provided in the Sphinx HTML output is another nice bonus. Which leads us to…

JavaScript

JavaScript is a particularly important “second language” for a documentation tool because its virtual monopoly in web browsers means that developers of web services in many other programming languages also need to deal with it, but in some respects it’s one of the most difficult ones to support. Whereas most other programming languages have a standard enough structure that vaguely useful API documentation can be generated by automated tools even if the developer didn’t bother to comment his code, this has proven exceptionally difficult in JavaScript; unless a human identifies how the code is structured, software can’t reliably describe it in a way that’s particularly useful to humans. From this perspective, using the JavaScript domain that Sphinx provides for describing JavaScript APIs probably involves no more work than any of the alternatives:

.. js:function:: $.getJSON(href, callback[, errback])

   :param string href: An URI to the location of the resource.
   :param callback: Get's called with the object.
   :param errback:
       Get's called in case the request fails. And a lot of other
       text so we need multiple lines
   :throws SomeError: For whatever reason in that case.
   :returns: Something

(Again, this should look familiar to anybody who followed the link above.)

Nevertheless, many developers came to JavaScript from languages (especially Java) which support auto-generation of docs from source code, and heroic efforts have been made to enable this in JavaScript as well; a number of projects have endeavored to comment their source code in ways that work well with tools such as JSDoc, YUI Doc, JSDuck, and others. Of these, JSDoc is probably the most widely used so far, and there happens to be a fairly nice utility (by the somewhat long name of “JsDoc Toolkit RST-Template“) for getting JSDoc to generate reST files for Sphinx rather than outputting HTML directly. JSDoc uses the Rhino Java-based JavaScript interpreter, so the command for running this one is more typical of the Java world:

ant -Djs.src.dir=src -Djs.rst.dir=docs/javascript build

Some examples of its output can be found here. It unfortunately doesn’t seem to work with JSDoc 3 yet, adding support for that would be a nice project for somebody with a little free time.

Putting It All Together

The API documentation for each of the languages described above (and others as well) all get initially created as reST (reStructuredText) files. Once generated, they can be treated like any other Sphinx source file (although you probably wouldn’t want to edit them directly if you ever plan to recreate them from source). Phrases can be extracted for translation, they can be combined with each other and other documents to form a larger documentation package, they can be combined with the output of other cool Sphinx extensions (seriously, even this is only a partial list), and so on. You’d probably want to write some kind of automation script to handle the details for you (here at Safari Books Online I wrote a Django management command to do it), but once set up you have a very nice tool for generating and maintaining a pretty comprehensive set of technical documentation.

One thought on “Documenting Multiple Programming Language APIs with Sphinx

  1. Pingback: Documenting code : a crucial point | Quantum Post

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s