Status on Visual Studio feature suggestions

Mads Kristensen

Mads

Visual Studio receives over 500 feature suggestions from customers every month on the Developer Community website. Handling that amount is a huge effort and we’d like to share with you how we handle this volume and the steps that we take to respond to them all. What happens to suggestion tickets after they’re opened, how many make it into Visual Studio, and what happens to the rest? Let’s find out.

Let’s start with the breakdown of incoming suggestion tickets in the past 6 months and what state they are in today. We find that around 15% of the suggestions are challenging to act on, and they typically fall into the following buckets.

11% – Closed as duplicate
3% – Closed due to missing info from customer
1% – Closed because they were not suggestions for Visual Studio

We do our best to follow up with customers to get more information where we can and move them into the next stage. For example, when making a suggestion to add a command to a context menu, it is important for us to know which context menu you meant.

That leave us with 85% left which are currently moving their way through the system. Here is the status of those tickets currently in our system:

40% – Closed for a number of reasons (more info below)
20% – New, not yet processed or triaged
28% – Under review and gathering votes and comments
3% – Awaiting more info from customer
3% – On roadmap (under development)
6% – Completed and released

Now let’s dig in and see what’s behind those numbers.

From New to Under Review

We have a filtering system that automatically routes incoming suggestions to the appropriate team within the Visual Studio organization. Within my team, we have established a weekly process to triage these routed suggestions and review status. The process we follow looks like this:

  1. Does this bug belong to my team?
    • If not, move it to the right team
  2. Is the suggestion a duplicate of an existing suggestion?
    • If so, close it and transfers all votes to the original ticket (happens automatically)
  3. Does the suggestion contain all needed information?
    • If not, ask customer for more information
  4. Was this suggestion already completed or in active development?
    • If so, close it as either Completed or On Roadmap
  5. If it made it this far, mark it Under Review to gather votes and comments for 90 days

By following these steps, most suggestions end up Under Review as we gather more data, refine any repos or requirements. These make up over a quarter of all suggestions.

Every time someone adds a new comment to an existing ticket, we receive an email, so we know what’s going on with each ticket along the way, and can respond if needed.

Moving on from Under Review

Within 90 days, we attempt to address items that are still marked Under Review. Our options are:

  1. Mark it as Completed because we implemented the suggestion
  2. Mark it as On Roadmap because it’s in active development or will be very soon
  3. Close it because it didn’t get any votes and/or we’re not able to prioritize it

When we implement a suggestion, we mark it Completed or On Roadmap. Currently, approximately ~10% of the incoming suggestions go on to be implemented or added to the roadmap.

But what about the ones that don’t?

Reasons for closing suggestions

Most suggestions are good suggestions and it’s always painful to close them. Especially because a lot of them are some that we personally would like to see implemented. As developers, you know that time and resources are finite, which means we can’t implement all suggestions.

The reason we close suggestions is a mix of multiple factors, such as:

  1. It didn’t receive any votes after 90 days as Under Review
  2. It got a few votes, but an implementation will not fit within our available resources
  3. It involves areas in Visual Studio that see little usage by our customers
  4. It has negative side-effects such as degraded performance, accessibility etc.

Over a third of all suggestions end up closed due to one or more of the above reasons.

On the positive side, even for some suggestions that we close, we do move the capability into an experimental extension for Visual Studio. This allows us to lower the cost of delivering a quality product investment, and where we can draw more interest from the community.

Suggestion completed

6% of all actionable suggestion tickets end up marked as Completed. It may not sound like much, but it is about 1 suggestion per weekday. Let that sink in. Every single weekday, the Visual Studio engineering team implements a community submitted suggestion.

Before we implement a suggestion, we first write a spec for it if needed. Then we schedule the work item in a sprint for engineering to pick up. The implementation sometimes require work by multiple teams to coordinate, so they can each do their piece of the feature.

After automated test and compliance runs have finished, it’s time for code review before the code starts its journey toward the Visual Studio master branch. More automated testing runs and finally manually testing follows. After fixing all identified bugs, the completed suggestion makes its way to a Visual Studio Preview release for user testing and stabilization.

So, how do we decide to implement suggestions and how can you optimize the chances of your suggestion making it? We look at several things:

  1. Suggestions with many votes and continuous votes over time
  2. Suggestions in areas that see lots of usage by our customers
  3. Suggestions that are easier to implement
  4. Suggestions that would improve Visual Studio’s competitive advantage
  5. Well written suggestion with all relevant information in the description

A different way to think about it is to turn it around. Imagine someone wanted you to implement a feature in your product. It’s in the best interest of our product and customers to complete as many suggestions as possible, and we strive to do so.

The best times are when we get to make a lot of people happy with a feature implementation based on a suggestion.

We can must do better

We’ve gotten feedback that this process feels like a black box. Customers feel like they don’t get a response and they don’t know the status of their suggestions.

After submitting a suggestion, there is no transparency into the process, and it ends up closed without any good reason 6 months later. I end up feeling frustrated and angry. I don’t want to submit another suggestion just to be ignored. – Anonymous Visual Studio user

This is not acceptable. We must do better.

Some ideas that we are working on within the team are as follows. And, we welcome your feedback on what we might do more of to help you understand the process better.

First up, we want to be much more transparent about the process. That’s exactly what this blog post aims to achieve.

Secondly, we must be faster at responding to new suggestions. That means triaging them within the first week, so we can bring down the 20% of new untriaged suggestions to a minimum. It also means not leaving any suggestions to linger for months. This will add visibility into what is going on with the suggestions much earlier and throughout its various phases. We’ve made great progress with this in the past 6 months, but still have a bunch of open tickets to go.

Thirdly, we need to be better at giving reasons for closing tickets. Individually written by the program manager that closed them and not an automated response. As we’re getting better at handling the vast amount of incoming suggestions, this is where we’ll focus next.

Feedback

I hope this blog post helps shed light on the way we handle suggestion and how we plan to improve. Completing a suggestion every single weekday will hopefully encourage you to continue opening suggestion tickets.

In closing, we’d really like to hear your thoughts or questions. What could we do better and what do we do well? Let us know in the comments below.

Mads Kristensen
Mads Kristensen

Senior Program Manager, Visual Studio Extensibility

Follow Mads   

102 comments

Comments are closed.

  • Avatar
    Steven Rasmussen

    Hi @Mads Kristensen – I’m curious to hear your thoughts on why one of the most voted suggestions (94 votes) with a lengthy discussion was closed.  It wasn’t even asking for new functionality, it was asking for the “old” functionality to be returned: https://developercommunity.visualstudio.com/idea/399833/bring-back-the-developer-news-on-startup.html
    Looks like another reason why issues might be closed is due to CYA – In this instance it seems like the MS UX devs don’t want to admit that perhaps their “new” UI design is less than optimal.  All we’re asking for in many instances is the ability to toggle new features on/off instead of things being shoved down our throats.  There’s absolutely NO reason this couldn’t be a toggle on the options page.

    • Mads Kristensen
      Mads Kristensen

      I don’t know why that particular suggestion got closed, but I know the team is exploring various possibilities to bring the developer news back into Visual Studio.

      • Avatar
        Steven Rasmussen

        That’s great to hear… but if that’s true why is the suggestion related exactly to that “closed”?  It would seem that the suggestion would remain “Open – Under Investigation” or some other appropriate status.  Closing the issue tells me that the decision has been made to stop any further investigation and to just “move on”.  Which one is it?  Is it still being investigated as you say?  Or has MS decided to close the book on this one?

      • Avatar
        Davis, Jon

        “I don’t know why that particular suggestion got closed, but I know the team is exploring various possibilities to bring the developer news back into Visual Studio.”
        Then what the heck is this??
        Pratik Nadagouda [MSFT]Jul 11 at 10:54 PM
        ​Thank you for the feedback and comments. At this time, we are closing the suggestion as it isn’t something we are investing in for the near future roadmap. You are free to refer to the news feed in the installer, our website at https://devblogs.microsoft.com/visualstudio/, or third party marketplace extensions for your news feed.
        You guys need to synchronize internally. This is really quite awful. I mean, awful. And infuriating. That particular thread had a lot of invested minds in interest (along with the suggestion threads showing dismay about the new modal dialog as the Start Page replacement) and that comment by Pratik basically threw the whole thing into the bin. Incredibly disrespectful to your customers.

        • Avatar
          Pratik Nadagouda

          I’m sorry that you felt disrespected, that wasn’t my intent. We honestly aren’t building anything at the moment to bring a developer newsfeed into Visual Studio. We initially thought of making a news tool window for those who wanted it back but Yann beat us to it with this excellent extension – https://marketplace.visualstudio.com/items?itemName=YannDuran.DeveloperNews. And I’m happy to recommend that developers use this extension if they want to see the news in their IDE rather than the installer or the web. I closed the suggestion, because we felt that was a sufficient solution. But if there are enough downloads of it to tell us that this is a feature that needs to go into the IDE, then we’ll definitely look into it again.

          • Avatar
            Eugene Ivanoff

            Ridiculous! “Let’s remove it in the hope someone will write an extension” policy.

          • Avatar
            Steve Parker

            So, my question then becomes… why?  Why did Yann beat you to it?  Why, after stripping such a fundamental part of the startup experience, and every single respondent asking for it to be brought back, why did it take Yann to beat you to it?

          • Avatar
            Pratik Nadagouda

            Honestly, Steve, we are hearing from many developers that they read their news primarily in the browser or other readers. We have not seen evidence that justified re-implementing a newsfeed and we’re not sure whether it has a place inside of the IDE. While we debated this, Yann went ahead and built the extension 🙂 We appreciate the contributions of our extensibility authors and a customer installing an extension is a much stronger signal for us to consider.

          • Avatar
            Steve Parker

            Oh, that’s great!  Visual Commander has had fourty-six THOUSAND installs, so you’re finally bringing back macros, right?
            What does “many developers” even mean?  What about the “many developers” who said they wanted the newsfeed restored? I’m sorry, but what you’re saying is that you’re basing decisions on faulty metrics and our feedback is worthless. Somone else has to write the extensions in order for us to get the value-added features we actually want.

  • Avatar
    Steve Parker

    You failed to mention another way that suggestions are handled.
    In the case of anything which criticized the new modal Start Window,  the discussions were completely hidden from view once they started racking up the highest vote counts of any topic.  At first, an attempt was made to dilute the discussion by closing the main thread and scattering people off to different topics.  It didn’t work, so when those topics started gaining votes, they were removed from visibility (some by being moved to an irrelevant thread or re-tagged).
    Even now, the topic “Make the Visual Studio 2019 start window non-modal” is not visible despite it having 53 votes. Of course, removing these topics from visibility ensures that they get no more votes.
    I find the level of censorship disturbing and the level of disrespect disgusting.

    • Avatar
      Pratik Nadagouda

      There hasn’t been any intentional censorship of topics. The suggestion you’re referring to https://developercommunity.visualstudio.com/idea/435168/make-the-visual-studio-2019-start-window-non-modal.html is still open and under review. We definitely are still looking into options there. I’ll discuss with the feedback team to check if there’s a bug by which topics are not showing up from search. The reason we split up the main thread into sub-suggestions was so that we could actually pinpoint the problems that were being faced and design solutions for the specific issues. We weren’t able to get a measure of what you all were voting on when the suggestion was an amalgamation of several topics. Again, thank you for the feedback and I apologize that you felt disrespected. 

      • Avatar
        Steve Parker

        Intentions notwithstanding, that is how it appears from this side.  The question as to why these threads seemed hidden was asked a number of times in a number of different threads.  In fact, the thread we are talking about asks, quite directly, why it seemed like suggestions were being “thinned” and threads seem to be inaccessible from the search.  Being non-responsive only contributes to the problem.  If nothing else, this clearly demonstrates that the current feedback system is broken at a fundamental level. People can’t vote on something that they can’t see.
        There are numerous complaints about ..every.. ..single.. ..aspect.. of the Start Window and the most disliked things about it – modality, no news, and no customization – are the very things that are NOT being addressed or discussed. What committee decided behind our backs not to include dev news after receiving so many votes and every single respondent saying they wanted it brought back? The topic was opened December of 2018. There was zero feedback before it was closed on July 11, 2019. The modal issue has been going on since January! 
        Why bother gathering suggestions if they’re just going to be ignored?

      • Avatar
        Steve Parker

        Intentions notwithstanding, that is how it appears from this side. The question as to why these threads seemed hidden was asked a number of times in a number of different threads. In fact, the thread we are talking about asks, quite directly, why it seemed like suggestions were being thinned and threads seem to be inaccessible from the search. Being non-responsive only contributes to the problem. If nothing else, this clearly demonstrates that the current feedback system is broken at a fundamental level. People can’t vote on something that they can’t see.

      • Avatar
        Steve Parker

        Intentions notwithstanding, that is how it appears from this side. The question as to why these threads seemed hidden was asked a number of times in a number of different threads. In fact, the thread we are talking about asks, quite directly, why it seemed like suggestions were being thinned and threads seem to be inaccessible from the search. Being non-responsive only contributes to the problem. If nothing else, this clearly demonstrates that the current feedback system is broken at a fundamental level. People can’t vote on something that they can’t see.There are numerous complaints about every single aspect of the Start Window. The most disliked things about it – modality, missing news, no customization – are the very things that are NOT being adressed or discussed. Which committee decided behind our backs to not restore dev news functionality? The topic was opened December of 2018. There was zero feedback before it was closed on July 11, 2019. The modal issue has been going on since January!Why bother gathering suggestions – suggestions on which everyone seems to agree – if they’re just going to be ignored?

      • Avatar
        Steve Parker

        So… every time I post a cogent response to this it gets deleted.  Any idea why?

      • Avatar
        Steve Parker

        Finally, after nearly two months, my attempts to respond magically… and suspiciously… appear.

        Now, I would like to know, do all comments “await moderation” or am I now on someone’s black list?

    • Avatar
      Chuck Ryan

      At this time I think it is safe to say Microsoft is not dealing with us in good faith. Perhaps it is time to move these discussions to a wider audience and to sites that they cannot censor. We are quite obviously not making any progress here.

      • Avatar
        Pratik Nadagouda

        Hi Chuck, I am sorry that we have given you that impression. So that I can understand how we can do better, would you be willing to get on a call? My email is prnadago at microsoft dot com.

  • Avatar
    Fawad Raza

    So many comments should tell one and only one thing to Microsoft. Your feeback system is severely broken and yes this “Pratik” guy is bit immature in dealing with an issue, it underlines a deeper problem of lack of training in Microsoft about the way things are handled.

    Don’t just say stuff, act upon it too, please.

    Regards.

  • Magnus Mikkelsen
    Magnus Mikkelsen

    How about adding a pane to the Visual Studio Installer, with links to new suggestions?
    Pick 5 – 7 new suggestions at random, and ask the user: “Which of these suggestions do you like the most?” 
    Or have reaction buttons along with the links. For example: “I need this!”, “But why?”, “Unclear: I don’t get it!”
    I think the data produced by this poll, might help your team to process and rank new suggestions. And it might also make users more aware of the suggestion process.

    • Mads Kristensen
      Mads Kristensen

      Now that’s a cool idea. Bring the suggestions more front-and-center so people can better discover them and cast their vote. Just sent an email to the team. Thanks!!

      • Avatar
        Michael DeMond

        Did you seriously track a new idea with an email rather than using the system that you posted an entire blog post on, boasting on how you intend to make it better?

          • Avatar
            Michael DeMond

            Pointing out that you basically spent a couple thousand words claiming that you are “improving” the feature suggestion instrument of your corporation, then directly did not that when presented with a “cool” idea and feature suggestion, is all.

  • Avatar
    Eleven Admin

    Consider this highly-voted ticket which is a hugely-annoying bug.  Fixed pending release for two years?!?  Then re-opened through desperation by someone, which was closed but actually triggered no progress on the matter.
    Then this ticket – it’s Closed – Fixed.  But it isn’t fixed.  The “fix” hasn’t worked for everyone.  So that’s the end of the story? 
    These are examples of why we get fed up with the feedback system.  Through goodwill and community spirit we try to help you to make VS better.  But often we’re just ignored once you’re done listening.

  • Avatar
    Andrew Truckle

    Another problem is that you are wanting to test issues with “laboratory conditions”. Many times we can’t do this. Like the MFC Class Wizard issues. The only way to resolve those it to include more debegging log info or to remove it to our particular projects. So many of my problems are closed because I can’t help you reproduce the issue. You need to try my particular projects (which are GBs is size etc.) and provide advanced error logging to produce sensible errors so that you can hone in to the real problems.
    Whilst i appreciate that VS 2019 is provided free, it is not really my responsibility to beta test the application. It should have already been tested with real life programs that started out in VS 6 and have migrated through VS2005, VS2015, VS2017, VS2019 to there they are now.
    Andrew

  • Avatar
    Andrew Truckle

    People also often add “replies” as “answers” which is wrong. And using your site on a mobile is very difficult. I have to use a PC for it to be a easy user experience.