Lessons Learned from the 2011 Scripting Games

ScriptingGuy1

May 8th, 20110 0

Summary: The winner of the beginner division of the 2011 Scripting Games shares his thoughts and insights.

Weekend Scripter

Microsoft Scripting Guy, Ed Wilson, is here. This week we will have a combination of guest bloggers and me with a recap of the 2011 Scripting Games. Today we have Klaus Schulte, the overall winner in the Beginner category. Here is his biography.

I was born in 1960 in Hagen, North Rhine-Westphalia (NRW), Germany. After school and basic military service, I studied computer sciences (informatics) at the Technical University of Dortmund (Oh, my!…is it really already 25 years ago?) The following 10 years, I was employed at a small company (BCT), where I worked on a control unit for copy-milling (machines that can be used as scanning devices to measure free-form surfaces and build a mathematical model of the scanned objects to calculate, store, and reproduce a similar mould). I programmed some parts of the control logic in C (and a bit C++) on the computer and on the Transputer-site that was integrated as part of the proprietary designed controller hardware. I participated in further education as a Novell Certified Network Administrator, but never needed it.

Since 1999, I’ve been employed in the IT department of the Regional Associations of Statutory Health Insurance Physicians (KVWL) to coordinate, observe, and troubleshoot the clearance workflow between the health insurance funds and our doctors. We handle the quarterly submitted (online, diskette, CD, or DVD) claims of over 11,000 regional doctors. Half of the time we have to do some database research to manage processes, produce statistics, and calculate the billing results. And sometimes we have to develop and deploy small command line scripts and programs.

Hope you enjoy this photo of Cassy, our 5-month-old Australian shepherd dog, enjoying life.

But now we are here to talk about something completely different…The 2011 Scripting Games!

Please allow a short review of what we did before the Windows PowerShell age began…

Once upon a time, I started to notice that the Windows command shell is not as bad as everybody in our company might have thought. We all used DOS 3.x command scripts to process a part of our specialized software environment (public healthcare) that had to be customized to do our job. Since the advent of Windows NT 4.0, the command shell had a lot more to offer than anybody thought. I discovered that we could add an else to the if, that different kind of for-loops were available, that subroutines could easily be inline in a batch by using an :eof at the end, that character substrings could be nicely extracted from strings, that we could do some basic arithmetic with set /a, and so on.

That is good, but for several reasons it had not been good enough. VBScript followed up and we really reached a point where “real scripting” began to change the way scripts could be used in daily work. VBScript is not the holy grail, but it is an instrument to get things done that couldn’t be accomplished with command.com. The chance to work with COM objects and use structured programming on a higher level created possibilities that have been great progress—a kind of revolution. We could automate a big part of Windows and simplify complicated processes in just a couple of lines script code.

Windows PowerShell is the next generation of scripting languages that will take this evolution a step further, I think. I did a bit of programming using C# and the .NET Framework because a new software was introduced in our company that included source code and could be extended to meet special purposes. So I could imagine how powerful that framework is and what can be done in a few lines of code. Having Windows PowerShell, COM objects, and the .NET Framework at hand is the biggest evolution you can think of. Being able to use it to full extent might be impossible for one person alone. The options are endless.

I started with simple Windows PowerShell scripts two years ago, and I had some success using it in daily work. But it has been an exotic kind of thing, and because our company computers (Windows XP without the Windows PowerShell option) didn’t support it, I couldn’t use it as a platform for scripts that are executed from other computers and different accounts. Therefore, its potential remained undiscovered.

I took part in the Scripting Games 2010 with some success, but just for fun. Things have changed a bit in our company as far as the admins are concerned because they will need to learn Windows PowerShell if they want to administer the new server products (Windows Server 2008 R2, Exchange Server 2010, and SQL Server 2008 R2) from the command line. Still only a few of our company’s computers will be equipped with Windows PowerShell. I am not an admin, so I can use it for my own purposes, but not on a broader basis until we upgrade to Windows 7.

That said, I took part in Scripting Games 2011, and I really had a hard time getting into the stuff again, having rarely written more than a script a month. Most of these scripts are simple and could have been written using batch files as well. I saw the announcement on the Scripting Guy’s blog, and I read the tweet: Starting on April, 4^th the 2011 Scripting Games will begin. Ten events in two categories, Advanced and Beginner, and you can choose to take part in only one category.

Last year, it was possible to take part in both categories, which is what I did. My first thought was that I should enter the Advanced category of this year’s games. Some rules had changed this time, and we had the opportunity to look at the first events before entering a solution for this contest. We had a week to do this, and reading through the first events revealed that the Advanced events were not easy, especially if you are missing a lot of the background. Not having much time to dig deeper into it, I decided to take part in the Beginner events, and I started to develop a solution for the first event.

Solution outlines to the 10 beginner events

Event 1: Use PowerShell to Identify Private Builds of Software

If you realize that a file has a property IsPrivateBuild and that Get-Process has a FileVersion parameter, the solution is pretty easy. Just put the information together and you are done with it.

Event 2: Use PowerShell to Identify Services that Will Pause

Get-Service reveals the property CanPauseAndContinue for running services, which is the solution to this event. Display a list of all the services that have this property set, and you have solved Event 2.

Event 3: Use PowerShell to Report Event Log and ETL Log Details

This is the event where I was stuck because I did not know that Get-WinEvent gives us access to all the event logs and the ETL logs (which I had never heard of before—a big gap, as I discovered). This cmdlet is of great value to everybody who has to hunt application bugs that may have occurred. Knowing it lets you complete Event 3 in a short period of time. Just retrieve the required properties and everything else is quickly done.

Event 4: Use PowerShell to Find Accounts Used by Services

If you know WMI a little bit, it is no surprise that you may discover the Win32_Process class that can be queried for the StartName property, and you just have to exclude the given values if you want to see the services that are not started with the three standard accounts.

Event 5: Use PowerShell to Collect Basic Computer Information

WMI strikes again! If you figure out that the classes Win32_OperatingSystem (Caption) and Win32_ComputerSystem (Username, Domain, Name) contain the most wanted properties, you can use two WMI queries to get answers to this event. A formatted output can be defined with a here-string,which helps a lot in this case.

Event 6: Use PowerShell to Find Windows Update Errors

Short and quick solutions are required here. Use Get-Content to read the file $env:systemroot\windowsupdate.log, and use the match operator to find FATAL in the content. It was very easy, but still, it was a little challenge to make it short and correct!

Event 7`:` Easily Determine Days Between Dates by Using PowerShell

How many days are left until the fiscal year ends? This is a more or less easy task to complete. We should generalize the event to: Count the days between two dates (that’s what we do daily to update the number of days before our daughter will return to Germany from New Jersey …still 66 days to go). We have to calculate a timespan as the difference between date1 and date2, which is just a subtraction of two DateTime values.

Event 8`:` Use PowerShell to Report Disk Space Utilization

Just use Get-ChildItem with the file system provider to show a listing of a folder’s content and select the desired properties like the file size. Use recurse if you want to, force to include hidden files, and exclude folders by excluding PsContainers. That easy task can be quickly done.

Event 9`:` Use PowerShell to Create a File Name Based on a Date

Displaying formatted DateTime values works like charm if you know how to use the formatting patterns. $format=’yyyyMMdd’ does the whole job. Keep an eye on available formats—you can do much with them!

Event 10: Use PowerShell to Measure Time to Complete a Command

Measuring the time to run a command is easy with Windows PowerShell. You can use the Measure-Command cmdlet, and additionally, Measure-Object to get to the average time it takes to execute it. If you use a script block, it is a more general solution to the question “how long does it take to complete a bunch of statements.”

All these events were quite manageable. If you have an hour of precious time, you could easily solve these events. Some of them are so straightforward that I could have written the solution in a couple of minutes, but if you want to provide some comments and comment-based Help, including working examples that have to be tested, it takes more time to polish the solution.

You should not underestimate the value of comments for yourself and for everybody else who might be reading it. So if you like to omit comments, you could save a lot of time; but someday soon, you will regret it.

`Expect errors to happen`

And here comes the next potential timesaver: Forget about all error handling, and you are quickly done with the solution. Error handling spoils your code.

Well, do not take this seriously! Do not!

Or do you really like this scenario: Your boss wants to see you…at once—if the first users are having fun playing around with your wonderful script in daily practice. They can invent strange parameter combinations with no effort at all. Their computers have never foreseen combinations of incompatible software running on them. They may not have the rights that your script needs to run. You can be assured of one thing: The first user of your script will send you a screenshot with error messages that you have never seen before! And you will most likely say, “These errors can’t be caused by my script!” (Do you know this famous last sentence: “The chances of anything coming from Mars…are a million to one…” You said that! Didn’t you?)

That is one of the topics I have been complaining about in nearly every expert solution I have seen so far. You may be admins, dear experts, but if you are not (and hopefully most of our users are not admins), there will be completely different aspects revealed that may make the whole script worthless for any normal user. It’s an art of its own to hand a program over to somebody who will have to run it with restricted rights in an even more restricted environment.

The least thing I would expect any script author to do is to provide basic error handling, which enables the user and the author to figure out what happened if something is going wrong. A readable and easy to understand error message should be displayed to the user; and ideally, you would like to write the error details to a globally accessible location on one of your servers for later investigation. This is always a very good idea, because you get 20 different answers if your Help Desk team asks the users what they have noticed and which error message has been displayed on the screen.

The error handling capabilities of Windows PowerShell are great, but you have to distinguish at least two different kinds of errors: Those that can be caught by a Try-Catch-Finally block, and those that can’t. Some errors will be detected only by looking at the error object and others can be caught. That’s something that bothers me most of the time because you have to know if the Try-Catch-Finally block will be sufficient to catch errors, or if you have to inspect the $error object (which is always a good idea), or if you have to take care of both. I would prefer the first method if it would be able to catch each error like you can using .NET languages.

There is a small degree between trying to catch every possible error, which may lead to unreadable and slow code and the opposite approach to handle hardly any errors, which may result in profile settings where all related preference variables are given values that turn error handling off—which is a good solution if you earn your salary as a human debugger!

Finding a way in the golden middle is not easy, but we all should at least shield operations that create new objects, access other computers or network resources, rely on correct data types or values, have to deal with null or empty values, and especially, return data from user input.

It is very acceptable (and most likely) that the user may enter anything that can be copied, typed, dropped, scanned, or comes from a network resource as input to our scripts. If you give scripts to the public, expect the worst! Even though I sometimes still cannot believe what I see.

After the games (-eq before the games)

“Everything is quite OK if, in the end, you made to the top of the list,” could have been my last and only statement here. But it isn’t.

The Beginner events were OK. The Advanced events were a bit too complicated but that is only a personal impression. They have been solved by others, so you may argue that they are OK, too. But because they require a domain or at least Active Directory for extra points (reading user or computer names from Active Directory), not everybody might have been able to get these extra points.

The whole games have a lot to do with managing computers and users, which is really one major operation area for Windows PowerShell—but not something that each competitor knows by heart and uses every day. This is a major point that should be discussed for the next games: splitting the categories into Admin and Developer. I would prefer to split the events again, but the judges might kill me if they had to judge four categories.

Splitting events into a more admin-oriented category and in a more programmer-friendly category (for example, build a phone book from username, phone number, and address components; calculate Fibonacci numbers; and verify user input) does make as much sense as having Advanced and Beginner categories. To be fair, I saw the first five scripts on the admin site and the last five on the developer site, which is already a good mix and maybe a compromise that we all can live with.

These games have been better than the last games because we had feedback after our scripts were graded and because the already graded foreign scripts were published before the due date.

That is great, but still we hardly had any feedback about how to improve our scripts for the next event. In fact, it might have been a good idea to wait a week and upload the script just in time because you would have immediate feedback the next day and you could look at higher graded scripts to learn from them. But in this case, you have to enter the script for the next event daily, except on weekends, which leaves only one day to build and upload it.

In general, it should be a must for the judges to give you feedback—at least on bad scripts—including the reason and advice about how to improve it. (I’m still not sure why my second script was graded with only 1.5 stars, so I might make the same mistake again.)

I know that it has been hard, and I really appreciate the work of the judges. Maybe the deep dive should have taken place after the games to concentrate on only one event at a time; but still, they made it in time and they did an excellent job!

I think we all are awaiting the 2012 Scripting Games, and we would appreciate the opportunity of continuing our scripting efforts during the next months. I would like to propose a weekly scripting category, which would offer us the chance to enter a script for Events 1 to 52. They would be due on the same day next week. The events might have only one category, and the solution should be commented and rated; but, we may not be awarded prizes (though I won’t mind). This is a keep-in-touch-approach that may let us continuously do some work and learn the techniques. And maybe we will all get to know each other better each day!

The games have been fun, and I hope they will go on each year!

Kind regards, Klaus

Thank you, Klaus.

I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.

Ed Wilson, Microsoft Scripting Guy