Shell Games (Matt Gertz)

Anthony D. Green [MSFT]

I was once temporarily taken off the VB team to get an unrelated project back on track, just a mere handful of weeks before it was due to ship.  I won’t go into the gory details; suffice it to say that we had reason to believe that the product would have to ship without delay, and that any major failure in the deliverable could create some seriously undesirable problems for both us and our customers. 

In the interest of expediting the development of the product, a external team knowledgeable in the underlying issue had been hired to do the actual work.  When our testers were analyzing the results of their labors, they found wildly unacceptable performance in what had been produced.  So, besides shepherding the work on the forty or so functional issues remaining in the product, I had to address the performance issues as well.  This was bread and butter to me at the time; I’d recently come off a stint as the VB Performance Lead, and had a good grasp of the tools of the trade.  So, I set about to investigate the cause of the problems.

Without getting too deeply into the details of the project, the upshot was that each file on the operating system having a certain extension would be opened and analyzed so see if it contained any of a particular set of keywords.  The extension was not an uncommon one; thus, a *lot* of files would have to be opened from the file system and examined.  This, of course, was where I suspected the problems would be, but what I discovered in the code left me absolutely astonished.

The natural thing to do for this sort of program was to recursively iterate the directory, find the relevant files, open them , search for the data, and then add the file to a list if the data was found.  What they did instead was this:  once the relevant file was found, they had created a thread whose sole purpose was to launch an application that they had on hand which generated a log report on the file.  The thread would listen for that application to terminate, and then open the log report and scan *it* for yes/no results.  Yes, that’s right – imagine dozens of threads being opened at one, each of them launching a copy of this app, with four disk hits (find file, find & launch app, open file, create log file) for each.  Yeah, that might just cause a little slowness on a Pentium II Win98 machine!

After I calmed down (about an hour later), I asked them why they were doing it this way.  “Well, we don’t actually know the criteria that would cause a ‘yes’ answer for a given file, since we don’t actually have the source code for the app that does know it,” they replied. 

Well, good grief! 

Five minutes’ search on MSDN gave me the format of the file being scanned, and in another thirty minutes I’d ripped out all of the threading, shell code, and log-parsing code, replacing it with ten lines of code designed to open the file and scan for the appropriate data.  The ultimate savings?  A total scan of a typical machine shrunk from two hours to ten minutes.  All in all, a most satisfactory day’s work.

Yeah, yeah, yeah… what’s your point, Matt?

My point in bringing up that story is to note the incredible damage that can be done by misusing shell calls.  Shell calls are expensive and, worse, they create results that may not easily understandable by the calling program (beyond a simple succeed/fail).  If you think that you need to “shell out” to another program, remember this simple word :  don’t.

Of course, it’s very tempting to shell out when an existing application already exists.  For example, if you want to get a listing of all subdirectories in a parent directory, and you want if formatted in a manner similar to that to which you are accustomed, then it’s very tempting to make a call to dir /a:d > out.txt and to parse the results.  The alternative, after all, is to overcome inertia and learn how to navigate the file system from code, right?   Sounds great, but the best advice here is don’t do that.  That’s what snippets are for, after all.

“There is still some goodness in him – I can feel it.”

Still here?  OK, shell commands aren’t actually evil – they’re just misunderstood.  They can be powerful tools if used right.  And the best usage of a shell command is a “fire and forget” situation, where you really want the user to be interacting with a bona fide version of the application.  In most cases, if the executable is producing output AND you care about it, it’s best to in-line the functionality if at all possible.

In Visual Basic, you can “shell out” to an application in two easy ways, both of which either leverage or wrap the System.Diagnostics.Process class.  These are detailed below.

When you don’t care when the shelled application ends: 

This is rather easy to do, and there’s actually a snippet that helps you when you forget the actual code.  To find it, put your cursor in a method, launch the snippet picker by typing “?” and pressing “Tab,” then choose “Windows System – Logging, Process, Registry, Services,” “Windows – Processes,” and “Start an Application” in that order.  (That’s for VS2008 – it’s slightly different in VS2005.)  You’ll get the following line:



And you can replace “notepad.exe” with the name of whatever executable that you want to run.  There are several overloads of this function, but you need to be careful with them, since a few of them allow you to specify a username and password to do a “Run As,” and that’s information that you definitely don’t want to hardcode or persist in human-readable form.  There’s a simple overload that allows you to pass command-line arguments in a string, which can be very useful for certain apps, and you’d use it like this (for example):

        Dim proc As Process =  _



The first argument to Process.Start can also be a document which will be launched with the application that’s registered to it, and this link shows an example of that usage. 

As I imply in the second example above, Process.Start returns an instance of the Process class which points to the process of the application you just created, and you can call all sorts of neat methods using that return value, including one that kills the process.  I mention this because you might otherwise be tempted to kill the process you just created using “Stop an Application” snippet:

        Dim processList() As Process

        processList = Process.GetProcessesByName(“notepad”)

        For Each proc As Process In processList

            If MsgBox(“Terminate “ & proc.ProcessName & “?”, _

MsgBoxStyle.YesNo, “Terminate?”) = MsgBoxResult.Yes Then


            End If



This snippet iterates over all processes having the name “notepad” and asks you for each one if you want to terminate it.  But since you already have a valid Process object, you don’t need to use this snippet  – just call Kill() on the return value of Start() directly.

When you want to wait for the new application to end before proceeding: 

This involves using the Microsoft.VisualBasic.Interaction.Shell command, which gives you a few “frequently used” options for controlling applications that you launch.  Here’s a contrived example :

        Dim retval As Integer

        retval = Shell(“C:windowssystem32notepad.exe”, AppWinStyle.NormalFocus, True, 20000)


I’ve specified the path to the application in the first second.  The second argument allows me to control whether the new application is minimized, maximized, etc – in the above example, I’ve chosen for notepad to come up “normal.”  The third argument specifies if I want to wait for the new application to complete and, given that I’ve chosen “True” for that above, the fourth and final argument specifies how many milliseconds I want to wait for that to happen, with -1 indicating an infinite wait.  (I’ve coded for 20 seconds above.) 

If the application shuts down before the time limit is reached, a zero will be returned from the Shell command.  If, on the other hand, the time limit is reached and the application is still running, the program will stop blocking and return the process ID of the application – note that this will not affect the instance of the application that I launched, though.  You can then use the process ID to take whatever action you need to do – for example:

        If retval <> 0 Then

            Dim proc As Process = Process.GetProcessById(retval)

            (Call interesting methods on proc here.)


        End If


creates a Process object based on the process ID returned from the above code, and you can act on it in the same way that you used the return value from Start() in the first example.

The bottom line

If you can, I’d try to avoid shelling out to other applications.  It’s much more performant to implement the functionality in-line or, better yet, leverage an assembly that’s common to all of the applications in question.  Still, you will occasionally come across situations where you want to launch the “one true” application programmatically.  The methods I’ve listed above are easiest and will get you going quickly.

‘Til next time…



Leave a comment

Feedback usabilla icon