Introduction

Many of the popular Visual Studio (VS) Code extensions like prettier or docker lack any true unit tests, which can be run without the vscode dependency. In fact, in taking a look at many other extensions and samples, nearly all are missing true unit tests. According to the VS Code Testing Extensions documentation, the way VS Code recommends running and debugging tests for extensions is solely through integration tests that utilize a special test instance of VS Code, the Extension Development Host. Extensions can then use this instance of VS Code to write integration and some form of “unit” tests in this environment. However, true unit tests are required for an extension, which should be runnable without a VS Code instance, VS Code provides no resources or guidance in this case. This poses a problem when it is necessary to create production quality code that adheres to strict engineering fundamentals, which require unit tests covering the majority of all components (>90% if possible). This blog will explore potential solutions to this issue and how teams can write sufficient unit tests for their VS Code extensions.

Potential Solutions

Under normal circumstances, relying on a dependency should certainly not be a problem for unit testing; the dependency can be mocked using a number of different frameworks. However, in this case, the vscode dependency does not exist as a normal dependency and is not listed in the dependencies member of the package.json, but rather the engines field. Thus, if the vscode Application Programming Interface (API) is mocked as shown below using normal TypeScript Unit Tests with the Jest Framework, a Cannot find module 'vscode' from 'src/tests/test1.ts' error will appear on the import statement:

// src/tests/commands/command1.test.ts
import * as vscode from 'vscode';
...
// mock a part of the vscode API used
vscode.window.showInformationMessage = jest.fn()

To address the above issue of mocking the vscode API while it not being a normal node dependency, there are at least two viable solutions.

Option 1: Wrap the VS Code API

Option 1 is to wrap all methods of the vscode API with different classes, as follows:

// src/vscode/windowApi.ts
import * as vscode from 'vscode';

export class WindowAPI {
  ...
  showInformationMessage(message: string): void {
    vscode.window.showInformationMessage(message);
  }
  ...
}

These wrapper classes can then be injected into other classes as needed:

// src/commands/command1.ts

export class Command1 {
  constructor(windowApi: WindowAPI, ...) {

  }
  ...
  run() {
    ...
    this.windowApi.showInformationMessage('Hello World!');
  }
  ...
}

Then when unit testing, these wrapper classes can be mocked, injected into the constructor of the unit of code under test, and tests can run without issue since the unit under test depends on the abstraction layer above the vscode API, and not the vscode API itself.

Unfortunately, this means that all calls to the vscode API must be wrapped in new classes and it would not be possible to unit test them since they’d still depend on calls to the vscode API. Care must also be taken to wrap the methods as closely as possible as to not lose or change functionality.

Option 2: Use Jest Manual Mocks

Option 2 involves taking advantage of Jest Manual Mocks to mock the utilized vscode API. Following the JavaScript process outlined in this blog post, the same can be done in TypeScript. All the basic mocks of each part of the vscode API used in the source code can be created and placed in the __mocks__/vscode.ts file:

// __mocks__/vscode.ts
const mockWindow = {
  showInformationMessage: jest.fn(),
  ...
};
...
const vscode = {
  window: mockWindow,
  ...
}

module.exports = vscode;

In the test code, these mocks can then be overridden to provide specific functionality to that test case:

// src/tests/commands/command1.test.ts
...
vscode.Uri.parse = jest.fn().mockReturnValue(mockUri);
...

Assertions can also be made against particular mocked items as required:

// src/test/command1.test.ts
...
expect(vscode.Uri.parse).toBeCalled();
expect(vscode.window.showInformationMessage).toBeCalledWith('Hello World!');
...

NOTE: make sure mocks are cleared after each test by using jest.clearAllMocks() in an afterEach block:

afterEach(() => {
  jest.clearAllMocks();
});

One issue using this method arises when a class inherits from a vscode class. Because the vscode API is manually mocked, the base class constructor and members will not exist. A workaround is then needed to test these concrete classes.

For instance, the following excerpt from Microsoft’s vscode source code shows a concrete class that extends a vscode class.

/*---------------------------------------------------------------------------------------------
 *  Copyright (c) Microsoft Corporation. All rights reserved.
 *  Licensed under the MIT License. See License.txt in the project root for license information.
 *--------------------------------------------------------------------------------------------*/
import * as vscode from 'vscode';

export class UriEventHandler extends vscode.EventEmitter<vscode.Uri> implements vscode.UriHandler {
  public handleUri(uri: vscode.Uri) {
    this.fire(uri);
  }
}

In order to test this class and assert on the this.fire(uri) method in the tests, an EventEmitter “mock” implementation would need to be added to the rest of the mocks:

// __mocks__/vscode.ts

// Because we extend the EventEmitter Class we have to create this "mocked" EventEmitter Class
class EventEmitter {
  event = jest.fn();
  fire: () => void = jest.fn();
  dispose: () => void = jest.fn();
}

...

const vscode = {
  EventEmitter,
  ...
}

module.exports = vscode;

Other VS Code Testing Solutions

Even though VS Code provides guidance for integration testing using their special instance of VS Code, in some cases this is not enough to perform sufficient integration tests. For example, when it is necessary to get (rather than set) some data, like a custom context or the content of an information message, from the vscode API. However, both of these cases are not possible, as there are no vscode API methods to get this data. In these cases, it is not possible to complete sufficient integration tests as the ability to verify certain results that come from the vscode API is missing. A potential workaround found was to use an end-to-end (e2e) user-interface (UI) testing framework, such as vscode-extension-tester, to achieve the test coverage desired.

The example below uses the vscode-extension-tester to open an extension’s treeview, check the content, and click a treeview item – all things that cannot be done solely via integration tests in the special VS Code instance.

  import { ActivityBar, VSBrowser, WebDriver, ModalDialog, SideBarView, By } from 'vscode-extension-tester';

  ...

  // Opens extension's custom side bar view
  sideBarView = await (await new ActivityBar().getViewControl('MyExtension'))?.openView();
  // get access to the Treeview Section named 'General'
  const treeview = await sideBarView!.getContent().getSection('General');
  const title = await treeview.getTitle();
  assert.strictEqual(title, 'General');
  // Click treeview item that opens Google
  await driver.wait(async () => await treeview.openItem('General', 'Google'), 2000);
  const dialog = new ModalDialog();
  const message = await dialog.getMessage();
  assert.strictEqual(message, 'Do you want Code to open the external website?');
  const details = await dialog.getDetails();
  assert.strictEqual(details, 'https://www.bing.com/');
  await dialog.pushButton('Cancel');

Final Thoughts

Testing extensions for VS Code poses a lot of challenges. There are not many resources on the topic and often the testing approach VS Code recommends (utilizing the Extension Development Host) falls short of the testing requirements needed for a production-quality extension. This blog post can hopefully help fill in some of the gaps and provided a good starting point for thoroughly testing VS Code extensions.

Related Resources

• Mocking the VS Code API for Unit Tests with Jest
• VS Code Testing Extensions Documentation
• An Article on Testing VS Code Extensions

Note: Blog Post Featured Image taken from Microsoft PowerPoint stock images.