Waiting for your company to provide training courses or advancement opportunities? It might be a long wait. Unfortunately many companies overlook professional development or consistently sacrifice it when the first round of annual budget cuts occur. Even when we do get training opportunities, many of us continue to select classes at random, based on what seems interesting at the moment.

We live in a field that is expanding daily, where the environment we work in is drastically different from what it was 5 years ago and from what it will be 5 years from now. Whether we plan on being in a different role or not, advancing our skills and staying in step with the market is a must.

This post covers a number of helpful links and ideas for staying up to date in our field.

Development Plans

Creating a development plan can help provide focus and give us a measure of our progress. These articles provide additional information on creating and refining our plans:

• SQL Server Professional Development Plan, Part 1 and Part 2 by Buck Woody (blog | twitter)
• How to Write a Great Individual Development Plan (IDP) and a great checklist by Dan McCarthy (blog | twitter)
• Your Own Personal Development Plan by Jeremiah Peschka (blog | twitter)

Tools and Processes

Once we have a plan, we need to be able to execute it. Having tools to help us execute the plan, methods of communicating the plan, and tricks for finding time can all help us be successful.

• The Personal Kanban site has a lot of good information on creating a visual board and process to help manage short and long term tasks
• Jes Borland (blog | twitter) helps track her progress by reporting a Quarterly Status to her friends and the wider community
• And American Express's OPEN Forum has a number of ideas on where we can find time to execute

Free and Cheap Resources

Not all development costs money. My own plan includes a certain amount of blog writing (free), reading a certain number of articles each week (free), an ongoing effort to keep up to date with webinars and webcasts (free), and attendance (and beginning to speak) at user groups and events (the free kind).

I can't afford Tech-Ed or the PASS Summit, but I can afford:

• Watching the recorded Tech-Ed sessions for free at MSDN (I've had over 11 hours of free training so far)
• Watching the 2008 PASS Summit recordings for free (requires login)
• Catching free live and recorded webcasts from Microsoft

Plus there are events like SQLSaturday, Code Camps, and user groups galore that don't cost $1000's to attend.

If you need more ideas, Ted (blog | twitter) posted an article with more SQL Resources here and I presented a list of ideas in Part 1 of the "There is Never Time For..." series.

Have more ideas or links for resources? Add them to the comments below. I'm always on the lookout for more ideas and I'm sure others would appreciate them as well.

Model-View-Presenter is an architecture pattern that defines a structure for behavior and logic at the UI level. M-V-P separates the logic of the presentation, such as interacting with back-end services and the business layer, from the mechanics of displaying buttons and interface components.

I often build small projects to help understand and grow my skills as a developer, architect, and all-around technologist (as may be apparent from the wide range of topics I post on). Today I worked with a combination of Visio and Visual Studio to build a sample project to play with the Passive View concept and to help grow my own understanding of the concept. This post will cover the Visio side of my learning-curve.

You can read more about Model View Presenter at Wikipedia and MSDN. Perhaps the best information can be found on Martin Fowler's site, where he has separate write-ups on Passive View and Supervising Controller.

Passive View

Passive View is a subset of the Model-View-Presenter pattern. In Passive View, the interface is responsible for handling interface-specific logic, such as figuring out how to put a value in a textbox or react to events from button clicks, but all actions and logic outside of the raw UI are sent to the Presenter to execute or manage. The Presenter is responsible for calling business methods in the Business model and updating the data that is available in the View.

Basic Model-View-Presenter Diagram

From the outside in, the architecture for Passive View looks something like this:

• UI - The User Interface reflects what is going on beneath it by implementing one or more View interfaces
• Presenter - The Presenter receives interactions from the UI or Model and updates the Views it is attached to
• Model - The model is a facade or black box in our diagram, behind which is a business logic layer and data layer

In a flat architecture we would collect data from the interface, perhaps do some business and data validation, and then save it directly to a database using stored procedures or inline SQL. Defining a data access layer (or data model like entity framework) allows our application to operate on cohesive, defined objects that are meaningful to the application and stored and retrieved consistently. Defining a business logic layer allows us to centralize business rules that operate on entities in our application in a manner that is consistent with the business and internally consistent in the application, minimizing the risk that occurs when making changes to the business flow. Separating the logic of populating inputs and responding to button presses on the UI from the information being communicated to the end user and conceptual responses to their input allows the system to interact with the user consistently across any number of interfaces into the same application.

The definition of each level increases our ability to automate testing and supports greater Separation of Concerns.

Implementing a Sample Project

My learning exercise has been the the creation of an ASP.Net search page that allows an end user (customer) to search for finished products from the AdventureWorks sample database. The architecture and design decisions were done as an exercise in Visio using simple shapes and layouts.

My example application has several functional and non-functional requirements:

1. Functional - Display product number, name, list price, and available quantity in tabular format
2. Functional - Provide a basic search input and button to search product names
3. Non-Functional - Implement an M-V-P pattern - Obviously the purpose of this whole exercise
4. Non-Functional - Use a simple model stack that can be easily replaced with a Service-Oriented one at a later time
5. Non-Functional - Build with the idea that we will later create a Silverlight or WPF front-end
6. Non-Functional - Make pretty pictures for article

My unwritten, final requirement was to finish the whole thing in half a day, though luckily I didn't define whether I intended that to mean 4 hours or 12.

Initial Architecture

To start I created a diagram of the application architecture:

More Extensive Model-View-Presenter Diagram

The purple layer is my presentation layer, which reflects the View. The blue layer is my Presenter layer which contains the logic for interacting between the end user and interface as well as a definition, or contract, of the information available in the View. The Green is the Model (or is behind the model, depending on your viewpoint) and exposes business functions and data entities for the Presenter to interact with.

Class Layout

Once the high level diagram was completed, I could approach the task of creating some base classes and interfaces to use in implementing the project.

• Model.IModel - Generic Model Interface to expose business calls to Presenters
• Presenter.IView - Generic View Interface that all Presenters can interact with and all screens implement
• Presenter.BasePresenter - Generic Presenter class that all Presenters will implement

To keep the project to a single morning but also allow the ability to come back and build a more architecturally sound solution, I implemented the Model in a very basic fashion that was referenced locally by the Presenter project and makes direct calls to SQL Server using ADO and parametrized, inline SQL. This buys me the benefits of having a well-defined Model (via the interface) but allows me concentrate my time and effort on the learning part of the project (ie, the M-V-P interaction and structure). Defining the model interface also leaves me open to come back and replace it with better separated code and the ability to create a model that acts as a facade to a service stack, instead of local DLLs.

• Model.BasicModel.Model - Basic implementation of a model that will interact with AdventureWorks on SQL Server
• Model.Entities.Product - A Product Entity that can be communicated between an IModel instance and Presenter
  
• Presenter.ProductSearchPresenter - A Presenter to manage product search interface
• Presenter.IProductSearchView - A view of the data involved in a product search
• ProductSearch.aspx - A web page that implements the IProductSearchView and interacts with the ProductSearchPresenter

My final Visio diagram ended up looking like this:

Diagram of Example Application

In this case the left side represents basic components (bases classes and interfaces) that are used to define common structure or contracts on the right side.

The Code

For the purposes of the example project, my view has properties for Search Text, a Search Count (number of results), Results (a generic list of the Product entity), and a boolean indicating whether there are results to display. My Web Form implements these properties, tying them to elements on the screen.

As the presenter populates properties in the view, the information is automatically reflected on the page. The actual logic of how the business functions are called and populate those properties are neatly packaged up in the Presenter and View interface and very little logic occurs in the actual web form.

To create a unit test, we define a simple view that implements the view interface, execute the presenter logic, and verify the properties are populated the way we would expect when the same presenter calls are made from the interface.

Extending the Architecture Further

Extending the application to display product search in a different manner would only require the addition of a new interface that also implements the Product Search View. A Silverlight front-end would only require creating the basic project, implementing the product search View, and wiring the new interface controls to the view properties. To replace the direct mode reference with a service reference, we could create a service facade that implemented the IModel interface, connected to a local or remote WCF service behid the scenes to handle the real model logic. And finally, instead of counting on our QA department to test all of the application interactions, we can create unit tests directly against the Presenter and Views to ensure that all of the interactions below the top surface of the application are happening consistently and to our expectation.

Your Turn

Getting this much of the architecture working is a good first step. I took a number of shortcuts on the BasicModel class in my example, but I now have a functional Model-View-Presenter application to play with. Hopefully there was enough information in the article to interest you in trying this out on your own. I urge you to read the articles linked in the top of the post (or several more in my Model-View-Presentation bookmarks) and come up with your own diagrams and sample project. Even doing a small project will force you to run into questions and considerations you wouldn't have had by simply reading about it, not to mention unrelated tidbits you will pick up along the way (for instance, I also learned about ObservableCollections today).

After our installation of SQL Server 2008 R2 a few weeks ago, I mentioned that we still had some additional setup tasks before we could consider the server to be done. This article covers setting up DatabaseMail, the internal mail client/management system that will allow us to use built-in alerts (among other things).

Technical Area: Accidental Database Administrator

Level of Difficulty:
Additional Articles:Virtual Lab entry on the LTD Wiki

Database Mail is a subsystem that acts like a SMTP client, allowing us to send emails from SQL Server processes and scripts. It replaces SQL Mail and doesn't require a local installation of additional software (like Outlook, the quick fix) to function.

Account, Profile, Service Broker, what?

Database Mail expands on the concept of sending mail to include all of the features we wanted in SQLMail. This grown up version of mail uses SMTP to communicate with mail servers, no longer requiring a MAPI component to communicate messages (the reason we used to install products like Outlook to make SQLMail work). Behind the scenes, Database Mail uses Service Broker to manage email in queues, rather than trying an immediate send. Profiles allow us to define a chain of fail-over accounts to send from, so that messages are never left undelivered due to a primary email server being unavailable. And accounts are exactly what they sound like, individual, unique accounts that we can setup to use a variety of authentication and SMTP options.

From the GUI

Database Mail is something we want to setup immediately on creating a new server. When we are working with a relatively small number of servers, the GUI is not a bad option for setting up some accounts.

We'll start by setting up an administrator profile with then intent to send critical notifications and alerts to this profile.

In SSMS we can right click the Database Mail entry under Management to begin the wizard.

DatabaseMail Menu Item in SSMS

The Intro screen is fairly boring, so lets press "Next" and head to the main configuration page. This being the first time we are configuring DatabaseMail on our new server, we want to leave the first option selected and continue.

DatabaseMail Wizard in SSMS

A quick check by the wizard and it determines that components for Database Mail aren't available on the server yet, so it asks us if we want to go ahead and install them (yes).

Profiles are used to represent a set of email addresses which allows us to represent a single person (or system) with multiple fail-over accounts. If an error occurs when the system is attempting to send mail from the first account in a profile, it fails to the next and retries, continuing until it either runs out of accounts or successfully sends it's message.

Lets enter a Profile Name of "The Accidental Admin".

DatabaseMail Wizard in SSMS

Now we'll add the first two accounts to this profile, each on a different network. Many admins will choose a non-local account with an external provider as their primary to reduce the chances that emails will not get through when other issues are happening. As many of you know, when a server room overheats the exchange server is always the first to go, so we will start with an external provider and fail-over to an internal one. Keep in mind that these are the accounts the database will send emails from when using the specified profile, not to.

Creating an account is fairly straightforward, simply enter the details for where you would like the email to be sent and any authentication options that are necessary. In this case we want to create a minimum of two accounts so the failover can work properly.

DatabaseMail Wizard in SSMS - Account Entry

Now that we have two accounts, we can move on.

DatabaseMail Wizard in SSMS - Account View

The next step is to decide whether our profile is going to be public or private. A public profile will be accessible to other people working on the system, whereas a private one will not. In this case we will make the profile public but be aware that this means anyone will be able to us the database server to send mail from this particular profile.

DatabaseMail Wizard in SSMS - Profile Security

The last interactive step is to confirm or modify some additional system parameters. This step is not specific to configuring an email profile and accounts, it is displayed as the final step of setting up DatabaseMail (when setting up for the first time). I suggest reviewing the settings to ensure your comfortable with each of them.

DatabaseMail Wizard in SSMS - System Parameters

After a final review screen, press the 'Finish' button to implement the changes.

DatabaseMail Wizard in SSMS - Finished

The last step is to test our new setup. Right-click the Database Mail menu option in SSMS and select "Send Test Email...".

SSMS Menu - Sending a Test Email

After entering an email address for our test message, we can hit the "Send" button and wait for confirmation of our ability to hit "Next, Next, Finish".

SSMS DatabaseMail - Test Email Dialog

or From a Script

Doing this on one server is pretty quick. Banging through the dialog on 20 (or with multiple profiles) is not only going to be a bit slower but probably guarantees at least one bad entry along the way. Luckily there is nothing magical about what SSMS is doing to configure these accounts and profiles, just a wizard driving system procedures behind the scenes. We can follow the same process programmatically and create a setup script. Once we have a setup script, of course, we can then use it against multiple systems and if we save it then we not only can set up future servers the same way, but we have detailed documentation on exactly how our servers were setup when we need it 3 years later.

Here are the commands we will be using:

sp_configure

This procedure is used to change SQL Server configurations

sysmail_add_profile_sp

Used to create a Database Mail Profile

sysmail_add_account_sp

Used to create a Database Mail Account

sysmail_add_profileaccount_sp

Used to tie a Database Mail Account to a Profile (Tab A in Slot

sysmail_add_principalprofile_sp

Assign access rights to a Database Mail profile

sp_send_dbmail

And sending an email, of course

And if we put it all together then we can duplicate the wizardy approach above like so:

Running this script creates a setup similar to what we created in the first section, though some of the names have been changed (and I have removed my email address from the list so I don't get server updates when someone comes along and forgets to swap out the addresses).

And Now, Uses

So now that we have Database Mail setup, what are we going to use it for? Well, besides creating our own monitoring and email scripts, the built in Alerts component in SQL Server uses Database mail to tell us when critical parts of our server have decided to misbehave. SQL Agent jobs have the option to notify us when they fail and they do so using, yep, Database Mail. Setting up Database Mail and using it in each of these situations won't prevent problems from occurring, but it is the difference between finding out when the problem is occurring and finding out after the server has melted down.

Next week will be something interesting, but I haven't determined what that will be yet, so we'll figure it out as we go.

In continuing Chrissie's recent example of helping to point out minor (yet critical) gotchas, I now present the guarantor of a few Null Reference exceptions, If() and IIf().

Both are designed as a way to provide VB.Net with an inline If, similar to the ternary operator found in C# and many other languages <condition>?<true-action>:<false-action>. If() and IIF() have the same signature: If(<conditional>,<true-object>,<false-object>) and IIF(<conditional>,<true-object>,<false-object>)

The critical difference between IIF (available from VS 2002 forward) and IF (available in VS 2005 forward) is that IIF is a function and evaluates all of its arguments prior to returning a value, while IF is an operator that executes like a short-circuiting conditional, only evaluating the true or false argument depending on the value of the conditional.

Why is this important?

So make sure you select the right function for your needs. In this case, selecting the wrong function can actually have the exact consequence you were trying to avoid (Null Ref).

Note: Not only a whole post without pictures, but also one that is less than 10 pages. And the sky didn't start falling.

In the previous article we covered the creation of a basic Domain Controller for our Virtual Lab. Using this basic domain controller and a basic SQL Server machine, we will look at the creation of a basic user account, creating a service account, and addition of a system to the domain.

Technical Area: Accidental Systems Administrator

Level of Difficulty:
Additional Articles:Virtual Lab entry on the LTD Wiki

This post picks up where the Virtual Lab: Creating a 2008 R2 Domain Controller post left off. Now that we have a new domain up and running we need to add some systems and users.

Creating Accounts

After installing the Active Directory Directory Services role, there are several new tools available from the Administrative Tools section of the Start menu. Throughout the remainder of the article we will be working through the "Active Directory Users and Computers" tool.

Domain Administrator Account

The first account we are going to create on the new domain is a Domain Administrator account. Even though this is a virtual lab environment, we don't want to get in the habit of using the built-in administrator account.

Open the "Active Directory User and Computers" screen and explore the default groups that have been created. As this is our first visit, there are no custom folders in our tree.

AD - Users and Computers

To add a new user to the default "Users" folder we will right-click on the folder, select "New", and select "User". This provides us with a dialog to enter the basic user information.

AD - Adding a User

After entering the general user information and pressing next, we are presented with a request for the user's password and password options. As we have not yet configured the password policy, the default policy is still in place.

Completing the wizard gives us a brand new member with basic Domain User permissions.

AD - Adding a User

To elevate the permissions of the user across the domain, we will assign them membership to the Domain Admins group. Open the user's properties panel (right-clicking the user and select "Properties") and select the "Member Of" tab.

AD - Configuring a User

Pressing the "Add" button brings up the search dialog. Enter "Domain Admins" and press "Check Names" to quickly find and select the Domain Admins group.

AD - Configuring a User

Press Ok twice and the user is now part of the Domain Admins group.

Service Accounts

Service accounts are an important part of a network. Many companies will rely on common accounts, like the built-in Administrator account, when they are setting up services on the Domain that require domain permissions. There are numerous reasons this is a bad idea:

• Greater Impact - a change to a single user account suddenly has a much wider impact then is intended, potentially causing service outages in disparate places acros the network
• Greater Impact, Frequently - how about password changes? Using one accuont for a wide range of services requires a wider set of work when a single password has to change
• Documentation - using a single account for multiple services requires either good documentation, good guessing, or a lot of overtime
• Security - providing an account with too much security is a potential security hole, a virus that uses a single application as an entry point suddenly has rights to everything on the network

In order to make management of service accounts (and services) clearer, I suggest each service account only have the level of permission on the domain it absolutely needs and that it only be used for a single service or application. This requires more work up front, as you have to not only create individual accounts for each application and service, but more importantly determine what "minimum requirements" means for each one. On the other hand, minimizing planned and unplanned system outages in the future is worth a little extra time.

Creating the Service User Account

Our guinea pig in this section will be the basic SQL install from a previous article. We are going to create a service account for the database server that has limited domain permissions but local administrative permissions. We will be using a middle-of-the-road configuration to create a single user account for all SQL Server services on our domain to run unser, what I would consider to be the minimum level of implementation.

On the AD DS server open the "Active Directory User and Computers" screen again. While there is a "Users" container already available in the interface, we're going to create a new Organizational Unit (OU) named "Service Accounts" to store these accounts.

To create the new OU, we right-click the Domain (in my case avl.local) and select New -> Organizational Unit. In the dialog we enter the name, in my case "Service Accounts", and press Ok.

AD - Creating a new OU

Right clicking the new "Service Accounts" container and selecting New > User opens the new user dialog.

AD - Creating a new Service User

As this is the domain account for our SQL Server services, I have chosen the name "SQL Server Service Account" and a username of "sqladmin".

On the next step we will also ensure that our password won't expire in the middle of the night by verifying the checked defaults and modifying a few.

AD - Creating a new Service User

After viewing the summary in the final step, we press Ok and create the new user. As the user is not going to have any special, domain-wide permissions we do not need to modify their group memberships or other settings. In fact we are done with the account creation until we bring our database server onto the domain and set up the local permissions for the user on that server.

Joining the Domain

Joining a server to the domain is a fairly straightforward process. In this case we will join our database server from prior articles and then we will configure our domain account from the previous section to serve as the administrative account for the database services.

The first step will be to change the network properties of the virtual server. When we first created it we used DHCP and that means that it received a dynamic address and DNS entries other than our new controller.

On the SQL Server machine we open the network settings for our adapter, select the "Internet Protocol Version 4 (TCP/IPv4)" value, and press "Properties".

SQL Server VM - Network Settings

Assign the server a static IP address on the network and point the DNS settings to the DC. At this point it would also be a good idea to check the "Validate" button to let Windows test the settings when you press Ok.

SQL Server VM - Network Settings

In my server the validation resulted in a message indicating that Windows couldn't identify the problem, this translates to "Yes It Works".

Next we will open the System Properties.

Now we want to press the "Change" button on the "Computer Name" tab. Select the "Domain" radio button and type in the name of the domain from your domain controller.

SQL Server VM - Join Domain

On pressing "Ok" we are asked to enter domain credentials with permission to join the network.

SQL Server VM - Join Domain

Once we enter our credentials and press "Ok" the system is accepted to the Domain and provides us with the customary reboot request.

back on the Domain Controller in the "Users and Computers" console, the server has been automatically added under the "Computers" container. This being a simple setup, we have not created our own containers to store our servers in, so we will leave it in the default one (for now).

SQL Server VM - Join Domain

When the SQL Server has finished rebooting we will log in for the first time with our Domain credentials. In the "Server Manager" we now see that the full computer name and Domain entries reflect the name of the domain.

Applying the SQL Service Account

The last step after adding a database server to the domain is to reconfigure it's services to use the domain service account we created above. This will allow us to later create domain resourcs that all of our SQL Server systems need to access without having to manage each one individually.

Currently our service account has no permissions on the server, other than as a basic Domain User (and for some of you, it may not even have that level). So the first step is to provide that domain account with local administrator permissions on our database server.

From the open "Server Manager" console, expand the "Configuration" and then "Local Users and Groups" containers, selecting the "Groups" folder. Right click the "Administartors" group on the right side and select "Properties".

SQL Server VM - Service Account

We enter the name of the domain account (in this case AVL\sqladmin) and press the "Check Names" button to ensure the account could be found.

SQL Server VM - Service Account

Press "Ok" and "Ok" and we are back in the Server Manager again.

The last step is to configure our services to use the Domain account, however this is done a little differently for SQL Server than you might expect.

Open the "SQL Server Configuration Manager" from the Start Menu (this would also be a good time to pin it to the taskbar).

SQL Server VM - Service Account

In the Basic SQL Server Virtual Lab post we only installed a few services and configured SQL Server and SQL Agent to use a local account.

SQL Server VM - Server Configuration Manager

Right-click on each service and select Properties to open the properties panel. Enter the domain account we created above in the Service Accounts section and it's password, then press Apply. The system will then warn us that a restart to the services is required to continue.

SQL Server VM - Changing the Account

Follow the same instructions for each of the services and we son have all of our services running under the DOmain account.

SQL Server VM - Finished

Wrapping Up

Now that we have the Domain Controller running, we've created some accounts, and we have joined our first database server to the domain, it's time to switch focus back to our database server for a while. Upcoming weeks will focus on some administrative tasks, such as configuring database mail and alerts and building out some scripts to monitor current state and state-over-time values for a few critical performance indicators.

Additional Articles:Virtual Lab entry on the LTD Wiki

A virtual domain controller (DC) can be both a powerful and lightweight addition to your virtual lab. A central AD implementation not only gives you another set of services to work with, it also provides a more realistic environment with the power of domain accounts and policies that will be useful when you start working with multi-server setups. This article will walk through the creation of a basic 2008 R2 Domain Controller.

Technical Area: Accidental Systems Administrator

Level of Difficulty:
Additional Articles:Virtual Lab entry on the LTD Wiki

Terminology

There are some terms that are commonly used in relation to AD or a Domain Controller that will appear in this and other documents:

Domain Controller

The server or servers responsible for handling authentication and informational requests for users/systems on the domain

Forest

Domains are represented as trees of directory information, a forest is the group of all trees in the organization

Kerberos

Kerberos is an authentication protocol used to allow systems on an un-trusted network a method to prove their identity to one another

FQDN

Fully Qualified Domain Name - the fully spelled out domain name for a domain, for example mydomain.com

Setting up VM

First we need to create a virtual machine to house the future domain controller (DC). In the first article we walked through the process of creating a basic VM. We will follow those same steps to create the VM for our DC except once we have Windows installed we will customize it further than we did with the original VM.

Creating the Virtual Machine

For the time being this system is going to be connected to the Host via the existing Bridged connection. At a later point in time we will look at creating virtual networks in VMWare and build out a more defined network.

Configuring the OS

Once the virtual machine is created, configured in VMWare, and Windows has been installed, it's time to move on to the Windows configuration. Some of these will be the same as our prior article and several will be new additions to prep the server for Active Directory.

First we will want to configure the server name and then assign it a new static IP Address on the local network.

Changing the System Name

After changing the name we'll open the network settings by either selecting the "Configure Networking" link on the Initial Configuration dashboard (above) or clicking the network icon in the tool tray at the bottom right, followed by "Open Network and Sharing", and then "Change adapter settings" near the top right of the screen. With the "Network Connections" screen open we can right click the network adapter and enter the "Properties" pane. Select the "Internet Protocol Version 4" item from the list and press properties to access the IP configuration panel.

At this point we need to select a static IP address that isn't already in use on our network and isn't in the DHCP range for our network. After selecting an IP Address we enter the static IP address, enter the gateway server or switches address, and then enter the servers address as the primary DNS entry and our switch or external DNS as the secondary one.

Configuring the IP Address

The last step to preparing your server for Active Directory is planning. In a non-lab installation we would want to put some forethought into how many clients we intend to support with Directory Services, whether there will be secondary DCs on the network, etc. I have an older link for capacity planning for AD on Windows 2003, but unfortunately I have not been able to find a comparable article for 2008 yet.

Installing Active Directory

In Windows 2008, Active Directory has expanded to take on a wider range of roles and capabilities. The portion we formerly considered to be AD, directory management and communications, is now called Active Directory Domain Services (AD DS). Other services include Active Directory Certificate Services (AD CS), Active Directory Federation Services (AD FS), Active Directory Lightweight Directory Services (AD LDS), and Active Directory Rights Management Services (AD RMS). Today we will be concentrating on AD DS, though I will continue to refer to as AD throughout the rest of the post due to bad habits and years of practice.

Assigning Server Roles

To start the installation and setup process, we will refer back to the Initial Configuration dashboard (click the pinned shortcut on the taskbar if you closed it already). Under section 3 there is a "Roles" category that will allow us to easily assign the necessary roles to our server.

Assigning Server Roles

Installing the Active Directory Role

The first page of the "Add Roles Wizard" presents us with some friendly reminders to set a static IP Address (check), ensure the Administrator account has a strong password (check), and make sure the server is up to date on Windows Updates (check).

Assigning Server Roles

Next we move on to the Roles selection page. As promised, we see a number of active directory services listed, the option for today being AD DS.

Installing AD DS

On selecting the AD DS option the wizard runs a quick validation and notices that our server does not appear to have a necessary component, in this case .Net Framework 3.5.

Installing AD DS - Prerequisites

After choosing to install the prerequisite (a difficult and time consuming decision, or perhaps not) and pressing next on the main screen, we are provided with a list of "Things to Note" as well as links to additional information, saving me the time of writing and linking them up myself.

Installing AD DS - Things to Note

A short confirmation displays the items we are installing, including the noted prerequisite above and a reminder that directory services won't truly be functional until we run "dcpromo".

Installing AD DS - Confirmation

After the installation and verification is complete, we are presented a handy link to launch dcpromo.

Installing AD DS - Wizard Complete

AD DS Installation Wizard (DCPromo)

The dcpromo wizard will complete the installation of AD DS on the server.

On the first step we are presented with the option to "Use Advanced Mode Setup". This option will cause several additional wizard windows to appear throughout the process that would have defaulted behind the scenes if it were not checked. Press the "advanced mode installation" hyperlink to get a help document about the additional options presented in advanced mode.

DCPromo - Beginning

After an informational screen about the suggested cryptography option and systems that may not support it, we are presented with the option to add a domain to an existing forest or to create a new domain under a new forest. This being our first DC in the virtual lab, we will be creating a new forest for our domain.

DCPromo - New Forest and Domain

Next type the fully qualified domain name (FQDN) for your domain. After typing this name the wizard will then scan to ensure the name is not already in use.

DCPromo - FQDN Entry

The wizard then prompts us to enter a Domain NetBIOS name, with a suggestion based on the name from the prior step.

DCPromo - NetBIOS Name

Next we'll set the functional level of our forest. The functional level defines the maximum level (and feature set) that our forest will support. When installing additional DCs in an existing forest or creating a business domain that you intend to support over the next ten years, this needs to be a well thought out decision. When installing in a virtual lab we will of course pick the highest possible level in less time than it took to write this paragraph.

DCPromo - Functional Level

After another quick scan of the network will tell us if DNS is available and supports the necessary capabilities for the domain controller. In this case, since we are installing a primary DC, I'm going to suggest we install DNS on the same server.

DCPromo - DNS

After selecting to install the DNS server we receive a warning that a delegation for the DNS server cannot be created because the authoritative parent zone for the domain name I entered earlier could not be found. Translated, this means that the server went looking for an authoritative DNS entry for the avl.local domain name I entered and determined that no one had ever heard of it. Since we are just now installing the DNS server that will recognize that name, this is to be expected and can be safely ignored.

The next step is to provide a location for the database and files that will be used by AD DS. In a non-lab setup this needs to be carefully planned to ensure enough capacity is available ahead of time. This being a virtual lab, we will be accepting the defaults and allowing it to carve out space from the primary drive.

DCPromo - Database and File Locations

The Directory Services Restore password is used only when major restoration activities need to be undertaken. After entering the password we will add it to our password safe so it won't be forgotten. This password will hopefully never be needed (especially in a lab environment), but not writing it down will pretty much ensure our first, future restoration step will be getting past the lack of a password.

DCPromo - Restore Password

After a brief summary of the selected configurations and the option to export these settings for an unattended installation, we are ready to let the server start installing.

DCPromo - Installing

The installer wraps up with the common Finish screen and a request to reboot.

After rebooting, our "Initial Configuration Tasks" window confirms our new roles have been installed.

Initial Configuration Tasks - Roles

Managing the Domain

Our next post will cover basic administration topics for the domain, as we add our first SQL Server system to the domain, create a new administrative user, and create a service account for our SQL Server services.

Additional Articles:Virtual Lab entry on the LTD Wiki

A virtual database server provides you the ability to not only work on database-relate tools and services, but also to try out hundreds of vendor products throughout the market that require a database backend. This article will walk through a basic database setup for SQL Server 2008 R2 on a virtual machine using mostly standard installation options. This first installation will be limited to basic settings and a virtual harddrive, leaving us ready to customize the setup for later uses.

Technical Area: Accidental Systems Administrator, Accidental Database Administrator

Level of Difficulty:
Additional Articles:Virtual Lab entry on the LTD Wiki

Setting up the Virtual Machine

In the first post we created a basic Windows 2008 R2 virtual machine to serve as a starting point for our later systems. Our first step will be to to make a copy of tis system for our new SQL Server installation. To begin with, we'll make a copy of the folder, renaming that copy to match the name we will be assigning to our new server. If we look inside the folder we will see a variety of configuration files, including the file that serves as our virtual harddrive.

VM Folder and Files

TO keep things consistent, we will rename the vmx and vmxf files to match the new server name as well. Once we have edited the names we need to open the vmx file in using our favorite editor (mine is EditPlus) and edit the values for "displayName" and "extendedConfigFile" to reflect the new file names. It's possible to rename the other files but requires editing a larger number of configurations and I prefer to leave them with their original name to reflect the system I used as a template.

To add the new, copied system to VMWare we open the menu and select the option to "Add Virtual Machine to Inventory".

Adding a Virtual Machine, Step 0

A dialog is presented with the list of stores that are set up on the local system. We select the store our copied VM resides in and then select the VM. The center column should now show us the vmx file we renamed, so we will select that and press Ok.

Adding a Virtual Machine, Step 1

The next step, once the VM is added, is to modify some of the settings. This basic VM was created with 4GB of memory, bridged networking, and an optical disk pointing to the Windows 2008 R2 ISO. These settings were to optimize the installation process, but now we should modify them to fit the needs of our new system. The optical disk will be pointed to the ISO for the SQL Server installation disk and the memory will be reduced to a more reasonable value. The network will remain in bridged mode until virtual network management is addressed in a later post.

The last step is to add a virtual drive for database storage. In later articles we will walk through comparisons of different virtual and physical drive options, but for today we will be creating a simple secondary virtual drive to serve as storage. This secondary drive will start off small, as we don't intend to work with a large amount of data on the system.

Adding a Virtual Machine, Step 2 & 3

Configuring the OS

Once the Virtual Machine is prepped, it's time to boot into our system for the first time and prepare it for the SQL Server installation.

When we first attempt to start the new virtual machine, the below error message is displayed prominently on the VMWare interface and we have to select either the "Copied" or "Moved" option to continue.

VM Startup Error Message

Before going too far with the new server, lets start by providing it with it's new name and executing the obligatory reboot.

Changing the System Name

Next up is mounting the new virtual drive. We can access the "Server Management" snap-in by right-clicking on the Computer in Explorer and selecting "Manage" or by typing "Manage" into the Start menu search and selecting "Server Manager". In the left column is a treeview with a number of options. Expand the Storage option and select "Disk Management" to see the available disks.

Server Management - Adding a Disk

Initially the disk is listed as Offline (bottom of center column). Right-clicking the offline disk will provide the option to bring it Online. Once Online the disk will require initialization, so we right-click and select Initialize.

Once the drive reports a status of "Initialized" we can allocate it by right clicking on the volume and selecting "New Simple Volume".

Server Management - Adding a Disk

Windows provides a wizard to walk through the drive allocation process. For the sake of this article we will select all of the defaults it provides, which should net a brand new 10237MB "E:" drive formatted for NTFS. Once the wizard is complete, the Server Management console will update the drive status and a dialog will appear with Windows asking if it can now format the new drive.

Server Management - Adding a Disk

When the Format screen appears, Windows provides several values and offers some defaults. Since this drive is going to be used primarily for SQL Server, we will change the "Allocation Unit Size" from 4K to 64K.

Server Management - Adding a Disk

And finally the last step in our OS setup is to create a new administrative account for the future SQL Server services. While the Server Management console is still open, expand the Configuration option in the right column, expand "Local Users and Groups", and select the "Users" option.

Server Management - Adding a New User

To enter the new user information we'll right click in the center column. To reduce future surprises we'll make sure the checkboxes are set to not require password change.

Server Management - Adding a New User

Elevating the user's permissions is fairly straightforward. After creating the user, we right click them and select "Properties". We select the "Member Of" tab and add the user to the local Administrators group.

Server Management - Adding a New User

Setting up SQL Server

If we had decided to use the host's optical drive for the VM's optical drive it would now be time to put the disc in the drive, whereas the ISO method should be good to go.

As we begin the installation process, the installer is going to check our prerequisites right away (unlike some prior versions that seemed to require everything from dog trainers to fiery hoops to get the job done). Press "Ok" and let the setup installer do it's thing.

Once this check passes (or we potentially go through the process of installing the .Net Framework and a newer version of Windows Installer), the setup process will present us with a "SQL Server Installation Center" window. If you haven't installed SQL Server recently, you are probably going to be surprised by the number of links that are available. Compared to the 5 menu options available on the SQL Server 2000 dialog, this one is almost a book.

Installation

First things first, lets make sure we have everything that SQL Server wants. The 5th option on the first screen is for the "System Configuration Checker". Selecting this option will start a dialog/wizard process that is going to go through and verify we have any additional prerequisites (such as being logged in with administrative rights) that the installer will need.

System Configuration Checker - All Green

Once everything in the configuration checker is green, we can move on to the Installation. Selecting "Installation" from the right sidebar presents us with a new set of options to select from. As we are installing a new server from scratch, lets select the first option "New Installation or Add features". After another Configuration Check, the setup presents a wizard for installing the Support Files.

SQL Server 2008 R2 Setup - Support Files (Next, Next, Install)

Once the Support Files have installed, we are presented with a dialog asking us to select whether we are trying to install SQL Server or PowerPivot for Sharepoint. Leaving SQL Server selected and pressing next will present us with a series of checkboxes to allow us to pick exactly which components we would like to install. Select the boxes the conform with what you intend to use the system for and press "Next".

SQL Server 2008 R2 Setup - Component Selection

Installation Configuration

After making selections for what components to install, the setup program provides a number of screens to allow initial configuration of the system.

First we should decide whether this will be the default instance on the server or a named instance. If we were working with an existing server than we would need to look at the existing installation to determine which way to answer this question. As we are installing our first instance no a virtual server, however, we can go ahead and select the Default option.

SQL Server 2008 R2 Setup - Instance Configuration

The disk space requirements step is always either purely informational or the beginning of a bad day. In this case we created our initial system with enough space to continue.

Next we are setting up the accounts used for the system services. Press the "Use the Same Account for all SQL Services" button, select "browse" and find the NT User we created earlier. Enter the password for the user and press "Ok" to assign it.

SQL Server 2008 R2 Setup - User Account Configuration

Next we have the Account Provisioning tab, Data Directory configuration, and FILESTREAM configuration. For Account Provisioning we have to decide whether every connection will be made using Windows Authentication or whether there is the potential that one or more of our vendors use SQL Authentication.

SQL Server 2008 R2 Setup - Account Provisioning

On the Data Directories tab we have the option of defining the default folders that files will be created in. Setting these defaults will not ensure other IT personnel will create files in the correct directories, but it will start them in the right place and catch those one-off backups people tend to leave in the SQL installation folders.

SQL Server 2008 R2 Setup - Data Directories

Continuing on through the next few screens, we should finally arrive at the "Ready to Install" screen. Pressing the "Install" button grants us one free coffee break, after which we will continue on with the initial configuration.

(insert intermission music here)

SQL Server Configuration

After installation of the software, there are still a few steps before we are done. Before continuing, let's do a reboot and then ensure all of our services started correctly.

The SQL Server Configuration Manager is available from the Start Menu (Start -> All Programs > SQL Server 2008 R2 > Configuration Tools > SQL Server Configuration Manager).

Verify that the SQL Server service is set to start automatically and verify that the SQL Agent process is set the way we want it. If the Agent service is not correct, right click it and use the Properties dialog to set the startup to the appropriate value.

SQL Server 2008 R2 Setup - SQL Server Configuration

Next we verify that the network configuration is set correctly. Since we intend to connect to the server from a remote client, we need to ensure that the TCP/IP option is enabled by right-clicking and entering the properties dialog.

SQL Server 2008 R2 Setup - SQL Server Configuration

If we have made any configuration changes in the past section we will want to do one last reboot. The SQL 2008 R2 installer did not ask for a reboot, but before we start working with the server we want to make sure that the next reboot (when a real server would be in service) will not come with any surprises.

1. Pin the "SQL Server Configuration Manager" to the Taskbar
2. Open SQL Server Management Studio (SSMS) and let it go through it's first time initialization
3. Verify that you can connect to the local server with the sa account in SSMS
4. Pin SSMS to the taskbar
5. Open SSMS on your Host system and verify you can connect to the server (depending on your VM network settings of course)
6. Record the new NT user and sa passwords in your password safe (I like Keepass)

Additional Tasks

We still have a few more tasks to complete before we can call this system ready, though I won't be walking you through the rest of the way until a later post.

Once we have the server set up this far, it's a good idea to go ahead and start setting up some of the internals, such as Database Mail and some of the Alerts. While these may be less useful on a test instance, it is good to build a habit of setting them up as part of a new instance. We also want to take this opportunity to reduce the amount of RAM assigned to SQL Server and set up our first backup schedule for our system databases.

While this seemed like a fairly straight-forward Next, Next, Next, Done installation, there was some thought put into where I was going to put files, memory sizing, etc. I highly encourage you to check out this article series for a more extensive set of pre-installation questions and thoughts.

In later weeks we will be working on the creation of a Domain Controller and Domain accounts, but afterward we will be coming back to this system and exploring other setup options, maintenance, and administrative activities.

Additional Articles:Virtual Lab entry on the LTD Wiki

Asking me what I do for living could net you different answers, depending on what day you ask. Over the years, and through the course of several jobs, I have had a wide range of responsibilities that often stretched well beyond my current organizational role. I like to call these opportunities.

Recently I was considering the idea of building a virtual lab, as I've been worried that some of the varied experiences or skills I've picked up would rust away. While in Wisconsin visiting Ted Krueger (twitter | blog) for SQL Saturday Chicago, I realized that the lab could serve two purposes. I could use it to not only regain or sharpen aging skills, but I could also use it to start blogging on more technical matters. Don't get me wrong, I love process and personal improvement topics, but I need my tech fix too.

So here we are. If you don't already have a virtual lab at work or home then I invite you to follow along with me as I build one. Each blog entry will have a general level of difficulty, from basic installation to advanced administration . As the lab grows we will be installing a wide number of systems then later configuring and tuning them to meet different circumstances and needs.

Lets get started.

Building the Lab Environment

Building your own lab can be both fun and a learning experience. A virtual lab provides you a safe place to try new technologies and configurations. When you create a really horrendous set of configurations in your lab, your halls are going to be remarkably free of end users with torches and pitchforks. The risks you should never take in production environments, even production development systems, are fair game here. But first we have to build it.

Environment

First we should determine what type of environment we want to run and what types of technologies or systems we would like to play with. Some of you will be planning no working with just development software and/or database servers, some will be want to create a virtual business environment (DC, Exchange, AD, etc), and some will even want to play with a mixed operating system setup. Identifying some general requirements will also help when your lab (or the market) grows past the current list of technologies you want to try.

Here is a sample set of questions you should consider:

1. What OS's are you intending to run? Desktop, Server, Windows, Unix, Linux ?
2. What major software packages or technical areas are you intended to explore in the lab?
3. Will any of the systems be used for active development (ie, results will be ported directly to production)?
4. Will any of the systems potentially move to production?
5. Will any of the systems need to travel with you (between sites, work and home, or to conferences)?
6. Will your systems need to run only some of the time, all the time, or a mix?
7. Will they require network access to the outside world?
8. Will the outside world be allowed to access your lab resources?
9. Will you need a private network in your lab or can they coexist on your existing network?
10. Will there be interdependencies? (for instance, Server A is running DNS and Active Directory and has to be on for Server B to work properly)

Platform Selection

Once we have an idea what we our lab will be used for, we need to determine what type of platform to use. If this is a home lab or you have limited space for equipment, you will probably want to go with a virtualization platform. Using a virtualization platform allows us to have a smaller physical (and electrical) footprint and multi-purpose our equipment, since many of our test servers will not need to be running when we aren't directly using them. Virtualization also allows us to move systems to upgraded hosts if we later find ourselves having performance problems.

The hardware you use will depend on the types of answers you provided to the questions above. Using an old desktop might be a acceptable solution or you may need to incorporate higher end systems or even spare servers. My last setup (2007-2009) used an off-the-shelf Dell Precision work station as the host without any extra equipment or over-the-top upgrades. The system performed well as a low budget SQL Server and Windows server platform. On the other hand my new environment will need to support a far wider range and number of systems, so it will require higher performance hardware.

Installing our First Virtual Server

From this point forward we will be working from the comfort of my home lab, so examples will be based on VMWare Server 2.1 and I will point out versions for other software as it is introduced. The purpose of this first virtual machine is to serve as a template for later machines, allowing me to build out one image that with a standard base configuration, installs all common services, then apply all the outstanding Windows updates. Though I will be using the term 'template' in the article, this shouldn't be confused with the template capability in VMware's vSphere.

Though the software is different, many of the concepts will be similar if your lab is going to be based on Microsoft Virtual Server 2005 and I have successfully followed almost the same exact steps through Virtual Server in the past.

Build the Server

First we need to build our virtual machine. Opening the VMWare Server Host and logging in with administrative credentials, we are presented with a dashboard view of our system.

VMWare, Freshly Installed

Clicking the "Virtual Machine" menu provides an option to "Create Virtual Machine" and a wizard to walk us through the creation of our virtual machine.

Create a new VM

While it isn't critical to pick a good name at this point, it couldn't hurt either. Later as we create servers for specific purposes the VM names will reflect the actual names of their servers.

New VM - Provide a Name

In my case I intend to use Windows 2008 R2 as my main server environment, so this basic image will reflect that in it's name and I will choose the Windows 2008 option from the operating systems tab. Before choosing the 64-bit image I made sure to run the 64-bit compatibility check offered by VMWare to ensure I could run a 64-bit guest on my host. If you are using Virtual Server 2005 then you will be limited to 32-bit guests so be sue you buy/download the correct versions.

New VM - OS Selection

For this first server I am going to allocate 4GB of RAM for use during the installation process. When making later copies of the system we will tune these values accordingly. As I intend to use these systems in a lab environment and may need the images to be portable to another single core system, I'm going to select the 1 CPU option. For later articles we may create 2- or 4-CPU options, but for most of our non-production needs a 1-CPU system will be sufficient. While technically it would be possible to "tune" the CPU setting at a later time, since we really only care about the hard-drive file at this point, changing CPU architectures on a system after installation ranges from tricky to absurdly-painful-with-no-hope-of-working.

New VM - RAM and CPU Settings

The final lasting decision (though not the last step) is how large we want to make the virtual drive and whether we want to allocate the space ahead of time or use a growth model. In my last environment I pre-allocated space for my base windows installation and then attached or created secondary drives for the installed applications. In this case, Microsoft is suggesting a minimum of 32GB of space and, while I don't want to give up the space, I've decided to use pre-allocation again to simplify the process.

New VM - Drive Settings

The final options, optical drive access and network method, are only going to be used for this image during the Windows installation process. Later copies will again have values assigned based on the project or technology that's being installed. For the purposes of the installation we will use an ISO file for the optical drive and bridged networking.

VM Setup Complete

Install the OS

We are now ready to turn our machine on for the first time and install Windows. This part is actually fairly boring, which is yet another reason that we want to limit how many times we have to do it.

The first decision the installer requires is selection between the various versions of Windows. Based on the options available, the trade-offs outlined in the feature comparison, and the fact that I have already written down my MSDN Enterprise key, I will be installing Enterprise edition. I'm not actually sure how long the installation took, I wandered off and found something else to do for a while (Peanut butter jelly time, peanut b...oh). I've installed windows a few more times than your average developer, so my bet is on 39 minutes.

After the initial login there are a couple key things we should do prior to starting the update cycle. First we are going to install the VMWare tools to make the interface easier to use and second, we are going to increase the screen resolution.

Installing VMWare tools - Server OS View

Installing VMWare tools - Guest OS View

After these two tasks we will begin by applying service packs (none for Windows 2008 R2 at this time) and then windows updates. Windows 2008 offers a handy dashboard to manage these tasks on the first install and, being lazy, I'm not going to turn it down.

Windows Server "Configuration" Dashboard

Next I am going to glance through the features and see if there any others that I would want installed on all of my servers. In this case the only one I can see if SNMP, so a quick installation and configuration and we are ready to move on.

Server Features - Configuring SNMP

The step we might normally move on to in a production or corporate environment is installation of base antivirus, client license, or and inventory management software client. This is an excellent time to install software packages that are part of your standard or that can be pre-installed and updated later. Another consideration would be whether you need to set up VNC, RDP, or another remoting technology.

The last item on my personal setup list is to assign a background image and set up BGInfo. Having a common background and system information on every server is extremely handy and helps show critical information at a glance (like disk usage or IP addresses). An easy solution is a utility called BGInfofrom SysInternals. BGInfo is a free download and there are many examples and articles available on the internet for setting it up.

Windows Server "Configuration" Dashboard

Last Shutdown of the Basic Image

Basic Image Done

The basic server is complete and ready to serve as the foundation for the rest of the lab. In coming weeks we will use this basic image to create more systems, such as a basic database server and a domain controller.

Additional Links:
Virtual Lab: Basic SQL 2008 R2 Virtual Machine
Virtual Lab entry on the LTD Wiki

Our ticket counts are down, the systems are failing less often, and we're reporting regular corrective actions to the business instead of just patches. Fixing issues as they occur, even root cause correction, is a process of stabilization. I'm not sure about you, but I don't like aiming for average.

This is the third article in a three part series on major gaps that we never seem to have the time to address as well as ideas on how to overcome and initiate processes for closing or reducing those gaps. Each article is based on one high-level subject and provides ideas on how to gain individual and group level traction.

We will initially cover some key concepts in Process Improvement and Lean, then jump right into a PC Request process that is modeled on a real process I have worked with in the past...and in one pass we will cut the PC ordering process in half.

Firefighting Is Not Our Job

Whether you're patching surface issues or solving root cause problems, all you're doing (in the paraphrased words of Dr William E. Deming) is catching the systems back up to either where they were or where they should have been. In the meantime, those services that support the company cost money while time and technology continues to improve and change the way competitors are doing their business.

On top of the inexorable march of technology there is a certain level of waste that accrues in every process. As time goes by small steps are added to processes that make sense at the time, but aren't really necessary long term or could be minimized with the advent of other new policies, long practice, or new ideas. These wastes can consist of extra costs, time, movement, or even poor quality work and reducing the wastes can not only reduce costs but more importantly unlock more of our (and others) time. And just like improving one's own skills and trying to standardize common operations, the benefit of starting process improvement is hard to quantify and requires time that we think we don't have.

Seven Wastes

The philosophy of Lean (or Lean Manufacturing) categorizes seven types of waste:

Waiting

Any time there are delays between the steps of a process we are creating waste. Delays between steps when no one is working on the task cause us to have to try and see farther into the future and provide people with longer quotes than they will be happy with. Plus it gives people more time to make minor changes or additions to their request, potentially requiring us to redo part of the work we had already completed.

Overproduction

Spending time and resources producing more of something than a company needs is a waste. That same time could have been spent doing absolutely nothing or, better, working on something that had direct value to the business. If we built or acquired something physical, then we also get a sneaky secondary waste because now we have to use up valuable space to store that item.

Unnecessary Motion

Excess walking, stretching, lifting, etc is not only a concern for safety reasons, but also a potential form of waste. As you walk back and forth to the same place 10-15 times a day consider how much time you could save if it was a shorter walk or no walk. Considering the focus many of us are required to bring to our tasks, consider how much smoother they would be if we weren't constantly stepping out of our zone to go walk somewhere and back again.

Unnecessary Inventory

Excess inventory in the IT world is not just excess equipment, but can also mean excess tasks waiting on their next step. One example would be a backlog of new equipment that is waiting for OS and software installs before it can be usable, another would be a backlog of software tasks that are waiting on QA to validate them. There are costs involved in having unnecessary inventory of anything, physical equipment takes up valuable space and time to inventory and manage, tasks that are sitting around require more extensive tracking and management procedures and start getting in the way of other tasks that are coming in.

Over Processing

When the tool being used causes more work than should be necessary to complete a job, then we are in the realm of overprocessing. This waste can range from tools being poorly located in relation to one another, using more tool than is necessary for a job, and potentially overuse of the wrong tool simply to justify the expense in purchasing it. When we try to make a tool do a job it isn't good at, we waste time and money with a risk of greater defect rates in our process or end unit.

Transporting

Excessive movement of a product or task is wasteful of time and risks additional wear, damage, or potential loss of the product or task. Even in development there is waste in transporting a task to QA and back multiple times and understanding why that task is going back and forth, which is why many Agile shops suggest collocation.

Defects

Making bad or incorrect products is a waste of the time and energy that went into making that product. Completing a task incorrectly or applying the wrong correction in response to a help request is also a waste of time and energy. Any time that we execute the wrong task or produce poorly, we are going to spend more time to determine it was the wrong thing, possibly why it was the wrong thing, and then how we can correct it or start again from the beginning. Doing this occasionally is waste, doing it regularly is an ongoing failure.

Seven wastes. How many of these looked familiar? Would you be surprised if I told you studies had shown that service businesses often ran at the 30-80% waste level? (Lean Six Sigma For Service, Michael L. George) What if your department was on the best side of that curve and could only manage to reduce waste by half, would a 15% improvement in cost or time be worth it? Reduced times from requests coming in to requests being fulfilled? What if your competitor did it?

Process Improvement

There are a lot of process improvement tools out there, as well as a number of methodologies. Lean and Six Sigma, the two I am most familiar with, focus on waste reduction and process quality improvement respectively. Lean has a number of tools for identifying wastes in a process and evaluating them, while Six Sigma brings us tools for measuring the quality of a process and improvements to that quality. Being an expert at neither of these methods, I am instead going to outline some basic methods for evaluating a process and trying solutions.

Evaluating a Process

When you are first starting out it will be easier to pick processes that you know are running slowly or that the business is commonly upset with. Later, once you have a bit more experience and a track record to show people, you should involve people that are directly interacting with IT processes and enlist them to help you find wastes in processes. Generally the people actually executing a process have a good idea when it isn't working, however it is important to start evaluating without an end solution in mind. Many times when we can identify problem areas we think we already have a solution, only to find something an order of magnitude better during our improvement process.

Select a process that is commonly considered slow or costly to perform. Starting at the beginning of the process, for instance when the request is generated or when a system triggers a process, draw each step the task goes through on it's way to completion and communications of that completion. At each step write down the average amount of time (you could also choose to write down the maximum, just be consistent). What you should end up with is something like this:

Diagram #1: Partial sample workflow of PC request process

In our example workflow it takes almost 20 days from the first request to actually get hardware in so we can start installing software. At each step in the diagram we should be asking ourselves what value is provided by the step and why it takes as long as it does. For example, the most obvious items on the diagram are probably the time for filling out the form and generating the PO. Cutting those two in half would be a 13% improvement in the process time from the request to getting the equipment in. But lets look at a couple more, what value does it provide for the IT Manager to sign off on both the request and the Purchase Order Request? What value does it provide to quote each PC request individually? We now have 4 items to work on improving.

Improving a Process

So we have some targets to start improving on from our example above. The simplest tool for reducing the waste of a step in the process is to understand the purpose of the step. By understanding the purpose we can start paring away some of the waste that has attached itself to the process over the course of time. Each step should be evaluated against the goal of the process and determine what part (if any) actually provides some value towards that goal.

I like using the 5-Why process to dig into processes, it is simple, direct, and gets good results. A good article can be found here.

Example 1) The Request Form

The request form takes 2 days to fill out on average, partially because the end user has to ask a lot of questions about values on the form and partially because it's so long it takes a day to get up the energy to get started. The value it provides is a clear checklist of everything the end user thinks should be on the final PC.
Normally people would look at the situation and ask if we could simplify the form, build a system, or better educate the end users. I would go one step further, can we get rid of the form? What if we created a standard configuration for each department and a standard list of applications for each management position? It still isn't perfect, but would it be closer or farther then having the end user try to create the list for us for each individual PC?

Example 2) Generating the PO

The process of generating a PO currently takes 3 days. We identify the value of this process as having the funds to purchase the requested PC. Unfortunately we have over-simplified the process of generating a PO by making it only one step in our diagram and I am sure the person responsible for generating them would have a few words to say about that. Instead of trying to guess, walk down and have a conversation with them. If possible map out the process they are going through, from receiving a request to completion of the PO. If you can get them on board and reduce the process, then you will be making a company wide improvement and I expect to see "Cross-Functional Process Improvement reduced processing of Purchase Orders by 50% company-wide" on your year end list of achievements. For the mean time, lets put this one to the side, as we won't be able to fix it until management sees some good savings from process improvement and we've built up a little credit.

Example 2b) Generating the PO

Wait, what? Again?
While we could help the PO process improve, our goal today is to try and improve our example process, so lets look again at what value we are achieving. Earlier we identified the value as being able to purchase the PC. Can we standardize a little? How much of our PC purchases fall into a pattern? We've undoubtedly budgeted for a certain degree of computer equipment throughout the year based on expected labor growth or turnover, would finance allow us to create blanket POs to cover this expense? If we could standardize these PC purchases and use blanket POs, then we have all the value with no cost to this process. A blanket order for new hires and a second for PC replenishment would could cut out 4 Days, approximately 20% of the current process.

Example 3) IT Manager Sign-off

Assuming we hadn't simplified the Purchase Order process yet, we had two different instances for the IT Manager to sign-off and in both cases it is taking him/her a day to do so. Before we jump into color-coding labels or special desk trays, lets take a step back and look at why they are signing twice. Is the first signature necessary? What value is it providing? If it is simply an acknowledgment of the request, do they need to do it or can they delegate that to the person processing the request? Potentially 1 day removed (5%). Did we remove the need for the per-order PO? Then that 1 day is definably removed.
And that's one less item to bug your manager about in your weekly meeting. Less stressed managers are happy managers, happy managers are much better than the alternative.

Example 4) Quoting the PC

What is the value of quoting the PC? Finding out how much it will cost. Going back to our second example, is there any way we could create some standards? How many PCs models do you need? Most vendors are willing to help you build a standard set of PCs with consistent quotes. Does each individual member of the company need their own special setup, or would a few standard models suffice? If the value of this step is to determine the cost and we can create some standard model configurations, then we just removed the custom quote step and reduced delivery time by another 2 days (or 10%).

Results

In our example above we saw the potential for reducing the first half of our new equipment process by almost 50%. These types of numbers are just as achievable in the real world as they are in the example, the difference is going to be that the real world will have more strange extra steps involved and it isn't always possible to completely replace or remove a step. Focus on adding value to the final deliverable and try replacing complex systems with simple ones. Sometimes all it takes to cut a couple days off a process is to have a labeled tray on someones desk or a clear plastic pouch to put documents in on their front door.
As we make improvements we need to communicate them to the wider company. By continuing to pare away at the wastes inherent in your processes, your area (and the company) is going to be running more smoothly and more effectively.

The next step is to grow the process, to be an agent of change in your business. There are others that are or will be interested in continuous improvement, but as you start to widen the scope you need to get buy-in from managers and executives. Having real-world examples of the savings the company has already achieved is going to be a strong argument for taking time away from day-to-day tasks and applying it towards a cultural shift that is an investment in an unknown ROI.

Why Improve...

While it may take more research and thought to identify wastes and start improving your environment, while it may seem easier to simply stand still and keep throwing more technology at problems as they appear, remember that there are companies adopting practices for improvement as part of Agile, Lean, and other cultural shifts. At the worst you will be increasing your effectiveness, decreasing costs, and improving your understanding of the business, at the best you could be starting a company-wide culture shift that requires managers to shift towards leadership and servant-leader roles, requires greater levels of respect for non-managers from the business, and enables businesses to provide high quality results at competitive prices, even when those competitors are offshore at much lower wages.

And the last benefit? Most of us are in our jobs because we like identifying problems and finding solutions for them. Process Improvement is just one more playground that we have available to us, with new and interesting problems that generally benefit from the simplest, most elegant solutions you can come up with. Sure, you're going to have to get up and talk to people. But they will be looking for you at some point anyway, so we might as well save some money, sharpen our skills, and head out to meet them first.

Miss an article in the series?
There Is Never Time For ... Part 1: Training and Development
There Is Never Time For ... Part 2: Standard Processes
There Is Never Time For ... Part 3: Process Improvement

So we might have some idea on how to improve our skills and knowledge, but it still takes 20 minutes to find a specific switch in the cabinet, the last time a server lost a fan we didn't find out until after it overheated, and we have no idea how many licenses we have purchased. And we'll figure it all out right after I finish this next help desk ticket.

This is the second article in a three part series on major gaps that we never seem to have the time to address as well as ideas on how to overcome and initiate processes for closing or reducing those gaps. Each article is based on one high-level subject and provides ideas on how to gain individual and group level traction.

[Coding|Provisioning|Reporting|...] Standards

It's always easy to put off little tasks until later. Labeling new equipment, updating a diagram of a wiring cabinet or server rack, creating monitors for performance and temperatures, testing your backups. Ok, some of these are less than small, and yet we continue to put them off.
Take a moment and list two or three items you would like to make part of your process for deploying new code, new servers, new databases, new (whatever you do all day long). Ok, now open up your calendar and look forward to the next month, two months, 6 months...where is the reminder that your going to start doing that task, that your going to go back and do it for all the work you have completed to date? Yeah, thought so. This isn't a "one day" task, it's a "Never would have done it" task. Remember, I've been there too.

Step 1: Make a Standard

The first step is to write it down. Creating a 'standard' is going to be difficult, we naturally shy away from writing these things down because writing is powerful and once it's written down we feel committed. On the other hand, we want it to get done, so that power that makes us nervous is also going to help strengthen our self-discipline and get it done.
Keep your standard as simple as possible. The more complex your standard is, the harder it will be to actually go out and follow it. if you later find items that need to be added, you can add them, but make do your best to keep it simple and to only add things that are actually practical (both in doing and in result). A simple standard is easier to follow and takes less time, letting you build good habits with less effort.
For instance, if this is a server deployment standard, list the standard set of software that has to be installed on every server, the standard settings that have to be changed, the services that have to be enabled, an item to ensure it is added to the backup schedule, and one or two of the items you pretended to list earlier when I asked (like monitoring).
After building your simple standard you should have a list of X things you already do plus one or two you always thought you should do or, when it broke, wished you had done. Save that document and each time you go to execute the task print off a copy and use it as a checklist.

Step 2: Revise your Standard, Review your Process

Next add a recurring appointment (once a month or once a quarter) to your calendar to spend 15 minutes reviewing your standard. During this scheduled time you will first assess how well you are following the standard. If there are lines you are skipping then try to root cause the issue and figure out why you are skipping them. If you end up back at "there is never time for.." then go back and review the reasoning for why you wanted to do it in the first place. If a 5 minute task for each server is going to prevent outages and millions in losses, it's going to be a little harder to make a case for not doing it.
After reviewing how well your doing each line, remove any lines you have determined were wasteful or are no longer necessary and add some additional items to the list. Again, keep your standard in a list format so you can easily use it as a check list to make sure your following your standard process, but also make sure there is enough description that a co-worker would be able to duplicate your process.

Step 3: Do It

Make it happen. You have a written standard and if you need further help staying on the straight and narrow then here are some more ideas:

Tell Your Boss

You have a reason you've created this process and your boss probably remembers the last time it cost time, money, or resources to solve a problem that could have been sidestepped. If you can convince them of the importance of your standard then they will also provide some extra [incentive|[pressure|etc] in making sure you follow it.

Tell Your Co-Workers

They're going to think your crazy, so again you will need to remind them how x minutes of work is going to save the group from that issue that happened in the middle of the night back when. If the group will be sharing responsibility then you not only want to share the standard with them, you probably want to include them in it's inception. You don't have to have everyone in a room together, but you can get them to agree it's a good idea and offer to build s starter standard that can be revised later. Remember, keep it simple so you can build good habits, later on the group can argue for a 50-page list of items.

Hold Yourself Responsible

Add an [annual|quarterly|whatever] goal to deliver the results of the process. Take the overall issue your trying to resolve (time involved in finding switches during outages, potential costs if a server fan goes out, cost to fix a code defect after it has been deployed, etc) and create a goal that says you will reduce that issue and will do so by implementing and following a standard. A good businessy goal is something your department can report directly up the chain and should be listed on your achievements for year-end review. Worst case, if you find a previously unknown issue that prevents you from reaching the goal, you can still claim success in making savings as well as uncovering a potential dangerous and unknown issue that the department can now work on resolving.

Discipline

To many of us it probably sounds crazy to voluntarily create a standard and make more work for ourselves, but it's the greater path of laziness. If we can convince ourselves to have the discipline to identify and execute those little tasks, it will mean fewer 3AM phone calls, fewer stress-filled outages, and fewer year-end inventories audits and other time consuming (and sometimes painful) activities. When you start looking at the number of risks and the amount of unplanned events you can avoid, suddenly the extra 10 or 20 minutes every few weeks, months, or years doesn't seem like so big a deal. But you have to do it, each and every time. Half the job is usually not going to net you half the results, so be disciplined enough to do it every time. If part of a task has to be put off due to an emergency deployment, schedule a meeting reminder to finish it later that day. If your a heavy procrastinator, like me, then remind yourself constantly than it is lazier to do that 5 minute task now then to take a 4AM call. Make it happen.

Next - Part 3

There is never time to sit and reflect on how your job gets done, experiment to find better ways, or attend non-technical training to improve your team or processes. In fact, if only you could reduce the time you spend on such-and-such task, you might finally have the time to focus on that. Hmm, chicken and egg.

Miss an article in the series?
There Is Never Time For ... Part 1: Training and Development
There Is Never Time For ... Part 2: Standard Processes
There Is Never Time For ... Part 3: Process Improvement