Status on Visual Studio feature suggestions

Mads Kristensen

Mads

Visual Studio receives over 500 feature suggestions from customers every month on the Developer Community website. Handling that amount is a huge effort and we’d like to share with you how we handle this volume and the steps that we take to respond to them all. What happens to suggestion tickets after they’re opened, how many make it into Visual Studio, and what happens to the rest? Let’s find out.

Let’s start with the breakdown of incoming suggestion tickets in the past 6 months and what state they are in today. We find that around 15% of the suggestions are challenging to act on, and they typically fall into the following buckets.

11% – Closed as duplicate
3% – Closed due to missing info from customer
1% – Closed because they were not suggestions for Visual Studio

We do our best to follow up with customers to get more information where we can and move them into the next stage. For example, when making a suggestion to add a command to a context menu, it is important for us to know which context menu you meant.

That leave us with 85% left which are currently moving their way through the system. Here is the status of those tickets currently in our system:

40% – Closed for a number of reasons (more info below)
20% – New, not yet processed or triaged
28% – Under review and gathering votes and comments
3% – Awaiting more info from customer
3% – On roadmap (under development)
6% – Completed and released

Now let’s dig in and see what’s behind those numbers.

From New to Under Review

We have a filtering system that automatically routes incoming suggestions to the appropriate team within the Visual Studio organization. Within my team, we have established a weekly process to triage these routed suggestions and review status. The process we follow looks like this:

  1. Does this bug belong to my team?
    • If not, move it to the right team
  2. Is the suggestion a duplicate of an existing suggestion?
    • If so, close it and transfers all votes to the original ticket (happens automatically)
  3. Does the suggestion contain all needed information?
    • If not, ask customer for more information
  4. Was this suggestion already completed or in active development?
    • If so, close it as either Completed or On Roadmap
  5. If it made it this far, mark it Under Review to gather votes and comments for 90 days

By following these steps, most suggestions end up Under Review as we gather more data, refine any repos or requirements. These make up over a quarter of all suggestions.

Every time someone adds a new comment to an existing ticket, we receive an email, so we know what’s going on with each ticket along the way, and can respond if needed.

Moving on from Under Review

Within 90 days, we attempt to address items that are still marked Under Review. Our options are:

  1. Mark it as Completed because we implemented the suggestion
  2. Mark it as On Roadmap because it’s in active development or will be very soon
  3. Close it because it didn’t get any votes and/or we’re not able to prioritize it

When we implement a suggestion, we mark it Completed or On Roadmap. Currently, approximately ~10% of the incoming suggestions go on to be implemented or added to the roadmap.

But what about the ones that don’t?

Reasons for closing suggestions

Most suggestions are good suggestions and it’s always painful to close them. Especially because a lot of them are some that we personally would like to see implemented. As developers, you know that time and resources are finite, which means we can’t implement all suggestions.

The reason we close suggestions is a mix of multiple factors, such as:

  1. It didn’t receive any votes after 90 days as Under Review
  2. It got a few votes, but an implementation will not fit within our available resources
  3. It involves areas in Visual Studio that see little usage by our customers
  4. It has negative side-effects such as degraded performance, accessibility etc.

Over a third of all suggestions end up closed due to one or more of the above reasons.

On the positive side, even for some suggestions that we close, we do move the capability into an experimental extension for Visual Studio. This allows us to lower the cost of delivering a quality product investment, and where we can draw more interest from the community.

Suggestion completed

6% of all actionable suggestion tickets end up marked as Completed. It may not sound like much, but it is about 1 suggestion per weekday. Let that sink in. Every single weekday, the Visual Studio engineering team implements a community submitted suggestion.

Before we implement a suggestion, we first write a spec for it if needed. Then we schedule the work item in a sprint for engineering to pick up. The implementation sometimes require work by multiple teams to coordinate, so they can each do their piece of the feature.

After automated test and compliance runs have finished, it’s time for code review before the code starts its journey toward the Visual Studio master branch. More automated testing runs and finally manually testing follows. After fixing all identified bugs, the completed suggestion makes its way to a Visual Studio Preview release for user testing and stabilization.

So, how do we decide to implement suggestions and how can you optimize the chances of your suggestion making it? We look at several things:

  1. Suggestions with many votes and continuous votes over time
  2. Suggestions in areas that see lots of usage by our customers
  3. Suggestions that are easier to implement
  4. Suggestions that would improve Visual Studio’s competitive advantage
  5. Well written suggestion with all relevant information in the description

A different way to think about it is to turn it around. Imagine someone wanted you to implement a feature in your product. It’s in the best interest of our product and customers to complete as many suggestions as possible, and we strive to do so.

The best times are when we get to make a lot of people happy with a feature implementation based on a suggestion.

We can must do better

We’ve gotten feedback that this process feels like a black box. Customers feel like they don’t get a response and they don’t know the status of their suggestions.

After submitting a suggestion, there is no transparency into the process, and it ends up closed without any good reason 6 months later. I end up feeling frustrated and angry. I don’t want to submit another suggestion just to be ignored. – Anonymous Visual Studio user

This is not acceptable. We must do better.

Some ideas that we are working on within the team are as follows. And, we welcome your feedback on what we might do more of to help you understand the process better.

First up, we want to be much more transparent about the process. That’s exactly what this blog post aims to achieve.

Secondly, we must be faster at responding to new suggestions. That means triaging them within the first week, so we can bring down the 20% of new untriaged suggestions to a minimum. It also means not leaving any suggestions to linger for months. This will add visibility into what is going on with the suggestions much earlier and throughout its various phases. We’ve made great progress with this in the past 6 months, but still have a bunch of open tickets to go.

Thirdly, we need to be better at giving reasons for closing tickets. Individually written by the program manager that closed them and not an automated response. As we’re getting better at handling the vast amount of incoming suggestions, this is where we’ll focus next.

Feedback

I hope this blog post helps shed light on the way we handle suggestion and how we plan to improve. Completing a suggestion every single weekday will hopefully encourage you to continue opening suggestion tickets.

In closing, we’d really like to hear your thoughts or questions. What could we do better and what do we do well? Let us know in the comments below.

Mads Kristensen
Mads Kristensen

Senior Program Manager, Visual Studio Extensibility

Follow Mads   

102 comments

Comments are closed.

  • Avatar
    David Roff

    I realise that the default tone for all comments is sarcastic, negative and often rude, but I would like to offer an exception.
    I think that Visual Studio is still far and away the best tool for my C++ developement needs. I’ve tried and discarded many others. Better still, it seems to be improving steadily, with stability and responsiveness leading the way. The newer C++ compiler and CMake support get a big thumps up from me. Sure, I have my gripes, but I can empathise with your team and I realise not all changes will be approved. Surely anyone can understand that not all suggestions can be implemented and there perhaps isn’t time, or resource, to explain why some suggestions don’t make the cut. 
    Keep up the good work.
    P.S. I don’t, and never have, worked for Microsoft

  • Avatar
    Mike Diack

    There is too little transparency and accuracy from Microsoft:1) Many regressions in the recent 15.8+ builds of VS 2017 have been flagged. The response has almost always been “it’s fixed in VS 2019, use that”. This has angered MANY people, because the answer is inexact and TOTALLY misses the point – the customers need the fix in VS 2017, which was working but has become broken (these are genuine regressions, not things that never worked to begin with). I have raised this with Nick. ATL wizards broken, #import wrong, wizards creating unbuildable code etc. You as a group need to fix this as well. 2017 Update 9 is intended as a long term support version of VS 2017, but has significant problems.2) When problems are resolved, the details of a fix becoming available are vague (they rarely refer to a build or version number exactly), instead saying “it’s now fixed”. This is hopeless for anyone outside of Microsoft. Why can’t we know exact builds etc that the fixes have gone into?

  • Avatar
    Mike Diack

    Agreed – the closure process is terrible. The lack of response beyond “Closed” – is why so many people are so angry with the VS team – because it feels like the much heralded community are having all their feedback tipped in the bin. Frankly it feels like we’re ignored. Many many many people want the start/new project work “fixed”. Many more people said – please don’t mess around with the VS IDE UI (the menu bar etc), but that feedback too, was mostly ignored. Much as with Windows 10. It feels like you are fiddling around with the periphery rather than getting the basics right.

  • Avatar
    Sam Jost

    What about translation bugs? Should I post wrong translations as “feature suggestion” or is there a bug forum for these?
    For example the new testing dialogue in VS2019 has ‘state’ in german as ‘bundesstaat’, which would be, like, Texas, New York, … which of course is plain wrong.

    • Mads Kristensen
      Mads Kristensen

      Yikes, that’s a bad translation. Bugs are reported through the Help -> Send Feedback -> Report a Problem… menu in Visual Studio. Those bugs are opened directly in our bug database.

      • Avatar
        Sam Jost

        Yep, it’s one of the worst translations in VS – there are quite a few which are not wrong, but ugly. Like when you copy/past a file in english the new file will have ” – copy” attached to the file name. In german it attaches ” – kopieren”, which is more like ” – copying”. Better would be ” – Kopie”. 

        • Avatar
          Tsahi Asher

          I wouldn’t have tried using VS in anything but English, even if it had a translation to my language.

  • Avatar
    James Foye

    Lost cause. I myself, after decades of using Visual Studio, switched to Rider. I come here now to the VS blogs because I find comments entertaining. (I have a dark sense of humor). Despite all the protestations of “We are listening”, you continue down the road of changing things for the sake of change, then springing them on the community, then dealing with the negative feedback after the fact, when it’s too late to do anything. Plus at some point you fired too many QA folks, as the bugginess of the product seems to get worse and worse.

  • Scott Ward
    Scott Ward

    I have a suggestion on there that seems to be totally ignored apart from the (I’m guessing) automated first response from Jane Wu
    I submitted a suggestion on 01 May 2019 and its now August so 3 months and no feedback what so ever, apart from non microsoft people with suggestion of how to get a response (those ideas went un-noticed as well).
    It would be nice to have some dedicated people to triage these and have some more status, eg, I can reproduce this issue, any known workarounds or state there is none, Has been added to a wish list that is actually looked at by the developers, under development, etc  
    my post: https://developercommunity.visualstudio.com/content/idea/553179/option-to-add-new-code-methods-at-cursor-not-botto.html

    I also aggree with the last commenter about the “controls for writing up features” they are terrible, it mucked up all my enter marks and could not be fixed while editing it. (On that note this page gives me a non-adjustable 4 line control for commenting (using Firefox))
    I love visual studio and use it on a daily basis for work and fun, it would be nice to be heard when there are feasible ideas that would make my and other users experience better and more productive

  • Avatar
    Ladislav Burkovsky

    I´m working daily with VS. I can tell you over the years we don’t got have significant improvements. Refactorings are at best basic (MS doesn’t use it internaly ?). Other area are mem/speed profilers. Why can you not integrate(buy) a serious tool. What happend with all the UML Tools (not integrated with class diagram). You are jumping from one theme to another. Take time and money and finish things so you can be PROUD of them.

    Laci

  • Avatar
    Ondřej Linhart

    Instead of fixing some functionality from VS2017, you just remove it in VS2019… Who wants to work in latest VS with less functionality than previous?
    How exactly you measure “little usage by our customers”?

    • Mads Kristensen
      Mads Kristensen

      What features are you missing in 2019 that was there in 2017? Perhaps I can help figure out what happened or what to use instead.

      We know what commands are invoked, tool windows used, and other such data points. So, we have very granular data on any part of the product and how much it is being used.

      • Avatar
        Ondřej Linhart

        I don’t care about your usage statistics, which is by the way irrelevant, because lot of users have it turned off.
        I care about features that were available in VS2017 and you removed them just because you did not fixed them where they did not worked.
        Save new projects when created option is out, you can easily find how devs are angry about that here or here
        – Horrendous New Project window in VS2019, who asked for that?
        Bug I reported 3 years ago was not even touched
        I can continue, but these are most important ones for me. And your solution to these problems? CLOSED

      • Avatar
        Ondřej Linhart

        Save new projects when created option was in 2017 and is not in 2019
        Clearly organised New Project dialog was in 2017 too, is not in 2019

        • Avatar
          Pratik Nadagouda

          Hi Ondrej – I really appreciate you taking the time to give us feedback. I’m sorry to hear that your original bug was not addressed. We’re working exceptionally hard to address each issue that customers report and you are correct that we need to do better. Please keep sharing your thoughts with us and I hope that you see improvement soon. 

          As Visual Studio is a 20+ year old product, we are intentional about the features that we carry forward to the next major version. In this case, we chose to drop the ‘save new projects when created’ option due to the extremely low usage and high cost to maintain. We’re open to reimplementing this feature if we see a significant number of customers requesting it but, as of now, we do not have that evidence. I hope this helps to clarify.

  • Avatar
    Matt Lavallee

    As a long, long time user of VS, what I see affecting my day-to-day has been an apparent fracturing due to trying to satisfy so many personas. I think part of VSCode’s overwhelming success is simply that it doesn’t have that burden.  When you’re an architect working with dozens of repos, dozens of branches, and hundreds of apps, feature changes like the new Start Page — which is probably perfectly fine for 50%+ of VS users who rarely context-switch — cause meaningful loss of productivity every single day.  It’d be fascinating (and helpful) to understand if the program managers are trying to weight the personas equally or if it’s just a question of pure vote volume that represents the community feedback.

  • Avatar
    Jan Heckman

    Mads, thanks for the good intentions and I understand limited resources and an overwhelming workload.
    Besides, I like improvements in VS2019, so I’m in a positive mood, so to say.
    Anyhow, given the subject, this cannot be anything else than an intro to some complaining. Sorry.
    It appears to me that your guys can ot upscale properly when a bigger issue is involved. I miss the triage proces.
    See https://developercommunity.visualstudio.com/content/problem/93889/error-c2872-byte-ambiguous-symbol.html which eventually works out to the (m)idl legacy code using type(def)s which need change because of C++17. This is in the bug department, but I feel the issues are similar. The issue is almost exactly 2 years old (under investigation), there’s plenty of comment and urgency, but all I see is some inane work-arounds and a question whether the issue can be closed, which I downvoted. The issue can imo be resolved fairly easily but requires work by MS. Here, the conductivity from problem to solution is not exactly very high and communcation is virtually absent. Better exampes do exist.
    I’m not happy about using this channel to go into details, but found this really frustrating, after feeling frustrated by (imo) a quite silly bug.These reports are often not made in the best mood and that shows.