Changes to How We Manage DefinitelyTyped

Orta Therox

The Definitely Typed repository on GitHub is now a regular fixture in GitHub’s annual Octoverse report as one of the top 10 repos with the most contributors. If you haven’t heard of Definitely Typed, it’s a repository that holds type declaration files (*.d.ts files) for thousands of JavaScript projects, and it’s likely you’ve seen the results of this repo in the form of the npm modules @types/something.

What started off as entirely community-driven effort became so critical to the TypeScript ecosystem that the TypeScript team committed to helping maintain Definitely Typed. The TypeScript team takes the responsibility of maintaining the Definitely Typed (DT) repo very seriously, and since mid 2016 the team has had a rotation where a member focuses on merging pull requests and repo up-keep for a week. We work with other Definitely Typed maintainers to handle about ~250 pull requests a week, and until last week we handled merging over 90% of these pull requests.

For the past year, we’ve been working on tooling to protect the stability of the Definitely Typed ecosystem to the point where we can offer a more streamlined experience when contributing to DT repo – definition owners can now help merge PRs to their modules!

How did we get here?

We have a few stability safety nets for Definitely Typed which gives the team more confidence in the robustness of the ecosystem

We run performance and integration tests on every change.

The performance tests check two things: whether the change increased the amount of memory to compile the files, and whether they would slow down editor operations like go-to-definition and code completion. Performance matters because we have seen changes in one library cause dramatic compiler time increases for a libraries which depend on it. By providing useful feedback during the pull request, maintainers can make well-informed decisions about whether these trade-offs are acceptable at the time.

The integration tests verify both:

  • The shape of your exports roughly matches the shape of the exports of the JavaScript using dts-critic, which gives everyone some more confidence that their types fit the JavaScript being represented.
  • That the change doesn’t break downstream DT libraries which depend on the libraries being changed.

TypeScript Development Verification.

It’s possible that improvement to the TypeScript language can break the definitions in DT. This could be a a natural part of improving the compiler’s control flow analysis, or by fixing bugs which a definition accidentally relied on.

The team does not want to release a new version of TypeScript and then discover that it breaks a significant amount of types in DT. There are two systems in place to protect this.

The first is that we made it easy to test all of the DT tests against a work-in-progress pull request of the compiler. If there are breaks in DT, then it is the responsibility of the author of the compiler pull request to send fixes to DT before the changes can be merged.

The second is that we run the DT tests against the master branch of TypeScript every night.

This helps the team catch issues early, and moves the responsibility of fixing issues to the people making language changes.

dtslint powered testing on a per-module basis.

Definitely Typed packages are always committed with “test” files. The tests are .ts or .tsx files which use the .d.ts files that eventually get shipped to users. We recommend that a package’s tests start out with something similar to example code for the library to test core scenarios, and then get built up as new types are added to the d.ts files.

These tests can verify the d.ts accurately describes the JavaScript, and also that future changes don’t break older clients by accident.

With these systems in place, we can more easily scale to the number of PRs coming in every day while avoiding long-running delays. That’s why we recently enabled definition owners to pull in changes themselves!

What Pull Requests Can Be Merged By Definition Owners?

For any pull request to Definitely Typed there are 3 groups of interested parties:

  • The DT maintainers who want to ensure the stability of the entire DT ecosystem
  • The definition owners who use the library and are interested in keeping the library up-to-date and well documented
  • People who are interested in the changes but have not requested to be a definition owner

Historically, only the DT maintainers have had the ability to merge pull requests. Through a new GitHub bot we’ve given access to definition owners to merge their changes.

First up in deciding if a pull request can be merged by a definition owner, we divide the changes to DT into two major sub-groups:

  • PRs which add new libraries, these will always need a DT maintainer to validate and are out of scope
  • PRs which edit existing libraries

We can bucket changes to existing DT libraries depending on their ecosystem impact:

  • PRs which affect extremely popular libraries like Node, React, Lodash or Jest
  • PRs which affect all the other libraries

Changes to extremely popular libraries with over 5,000,000 downloads per month on npm cannot be merged by definition owners and still require a DT maintainer to validate changes. However, these changes are rare and most of the pull requests we receive are quick changes to smaller libraries

When Can a Pull Request Be Merged By Its Author?

After determining that a pull request is one which a Definition Owner could merge, our bot starts to walk the author through the process. We set up four conditions for a pull request to be merged:

  • There are no merge conflicts
  • CI has passed tests for this library, and its dependents
  • There are corresponding test changes for any *.d.ts changes
  • The most recent commit has an approved review by either a definition owner or a DT maintainer

As the pull request evolves through these different stages there is a bot updating a comment which describes the current state of movement towards auto-merging.

We’ve tried to have the bot handle a lot of different states a pull request can be in and offer direct feedback on what to do next, providing links to the revised documentation for contributors.

Image 1 Image 2 Image 3 Image 4

We’re starting to see about 20 pull requests a day merged by the community through this new system. The pull requests which are merged by definition owners are typically handled within a day, and aren’t blocked on the time of whoever is on rotation from the TypeScript team. For example, a bug was found in the type definition for the module theme-ui – the PR to fix it was published, reviewed by the definition owners and merged within an hour. This is a perfect example of why we refined the process.

When thinking about how we maintain such a busy repo, it’s a combination of careful application of automation and a lot of human effort. Definitely Typed is a massive community project with over 10,000 contributors, and we’re really proud of how far it has come, here’s to making the life easier for the next 10k contributors.

1 comment

Discussion is closed. Login to edit/delete existing comments.

  • Josh H 0

    The new bot is promising. I had some questions about the process for owners here.

    Is this issue still working as intended or was the suggestion implemented now?

Feedback usabilla icon