Probably the number one failure of system scaling is when people dive right in and start building. No baselines, limited measurements, no analysis, just a hypothesis and a whole lot of late nights tweaking the system. With extra complexity comes extra costs, from the initial development through more expensive maintenance. Scale poorly and not only do we take on those extra complexity costs, but also the more obvious additional costs of the actual implementation (new servers, more resources, etc).

The Somewhat Contrived Example

Recently I've been working on a system to simulate parallelizing workloads, specifically workloads that depend on external resources with rate or load thresholds. Let's use it to look at a somewhat contrived example.

So the backstory is that I have a batch process that is running more and more slowly as we take on larger and more frequent workloads.

I start by testing the batch process locally so I can see how slow it is before I make changes.

1 Client, 60 Requests, 100rpm

Locally it runs pretty quickly, but I'm betting that parallelizing the process will give me a significant increase in speed.

1 Client vs 30 Clients, 60 Requests, 100rpm

Look how well that performance improved, I achieved better than a 6x improvement in performance. My job here is done.

Except when I try to push this to production, I start getting a lot of errors.

30 Clients, 300 Requests, 100rpm - 67% Failure Rate

As it turns out, the external API my process saves the data to has a rate limit. When I exceed the allowable rate, I'm throttled for a short period of time. Any requests I make during that throttle period are returned with errors indicating I'm throttled.

Hmm. Luckily there are a number of common patterns available to retry these types of failures. I'll add an exponential back-off retry pattern so that when I get throttled my service will retry failed requests at slower rates until the service un-throttles me. While I've found plenty of code examples online, none of them seem to have recommendations, so I'll just use one of the sample settings they provide.

30 Clients, 300 Requests, 100rpm, Retry Policy #1 - 7% Failure Rate

Hmm, better. My failure rate has gone way down. What if I tweak the values?

30 Clients, 300 Requests, 100rpm, Retry Policy #2 - 29% Failure Rate

Oh, that was bad, I obviously was on the right track before. What if I just extend the retry amount a bit to try and knock out the last bit of errors.

30 Clients, 300 Requests, 100rpm, Retry Policy #1B - 0% Failure Rate

Ok, perfect. Now I have a system that is more than 6 times faster than the original, can be easily extended by throwing more workers at it, and is actually in a better position to handle occasional slow downs from my 3rd-party service.

Success!

Where I Went Wrong

Ok, so maybe not. Over the course of my little story I went wrong in a number of places. Even though this was a contrived example, I've watched very similar scenarios play out in a number of different organizations with real systems.

The Bottleneck

The first and most critical problem was that I didn't actually locate the bottleneck in my process, I simply tried to do more of the same. The Theory of Constraints tells us that we can improve the rate of a process by identifying and exploiting the constraints.

In this system, the constraint looked like it was the sequential execution of the tasks, but in reality the constraint was the time it took to call the 3rd-party API. Had we identified that bottleneck before starting, we could have approached the problem differently.

Process - Alternative Design

Rather than the parallel complexity, we can modify how the tasks are executed to try and take advantage of knowing where our bottleneck is. If the API allowed us to submit several requests in a batch, this redesign would net us several orders of magnitude improvement. Another option would be to run the results of the local processing into a queue and submit requests from there at a slow trickle, using only a percentage of our API limit so as not to disrupt any other real-time operations or batch processing the system supports. Another option we could take advantage of is not starting any of our expensive 3rd-party communications until we know that the entire job can actually be processed successfully through our local process.

Identifying the constraint unlocks the ability to turn the problem on it's head and achieve higher improvements, typically by orders of magnitude.

The Math Error

I concluded the scenario above by assuming I had found a good solution that also had a lot of headroom. Unfortunately what I actually did was find the ceiling. I have tuned the retry policy to 30 parallel systems, increasing that number could easily destabilize it further and cause more errors or delays. The headroom is, in fact, an illusion.

The Evolving Assumption

Somewhere along the way I found 30 clients to be a great improvement and didn't test other options. Then I made some assumptions about a retry policy. Then I tweaked that retry policy until the error rate disappeared. My assumptions made sense at the time, so I never questioned where they were leading me.

When I found a winning combination for my retry rate, I didn't realize I was missing other, better options:

30 Clients, 300 Requests, 100rpm, Retry Policy #1B vs 4A-C

What's worse, is that trail of assumptions along the way was never re-validated. I concluded with a 6.5x improvement, but is that still accurate?

1-50 Clients, 300 Requests, 100rpm, Retry Policy #1B

When I run the same settings on a range of 1 to 50 clients, we can see that I lost that original 6x improvement along the way. All I have managed to do is add complexity and some very explicit costs for those additional systems.

Note: What happened to retry policy #3? Well originally 1B was actually named 3, then I decided to update the name halfway through the post but was too lazy to update all of the completed images. Oh well.

Scaling the Wrong Thing

This post used fairly small numbers, had I applied a larger workload, higher throughput rates, different throttling windows, the whole problem would have turned out differently.

When we set out to make a system scale, we need to identify the real scenarios we are trying to scale for and the bottlenecks that stand in our way. Blindly performance tuning can look like an improvement, but is really just a poor short term investment that often entrenches the current performance problems even more deeply. There are a lot of questions that should be asked about the intended result, responsiveness fo the system, other operations it has to support while under load, potential overlap of that load, the type of load, etc. The patterns for one system may have relevance for another, but could just as easily be completely incorrect.

How hard it is to scale a system is going to depend on a lot of factors. Getting it wrong just happens to be the easiest option.

As I said, it's a neat trick.

Advantages

Implemented consistently, this stops being just a trick and becomes a very powerful tool. Developers, QA, even customer service and support see benefits from being able to quickly emulate end users.

Debugging

As developers the largest benefit for us is the ability to debug our systems from a variety of viewpoints. Rather than going through the trouble of creating and managing sample users for every impacted role, we can use existing user records with real data behind them. This not only reduces the time to start debugging and remove the time involved in ongoing maintenance of test accounts, but may actually force out a few extra bugs that we wouldn't catch with a new, vanilla user account.

Support

Duplicating a bug or answering a question becomes a lot easier when we can emulate the person on the other end of the bug report or phone. We can emulate the person in our production environment and in our development environment and verify that both environments break in the same way (or that development doesn't break after we fix it). We also get firsthand clues, which can knock hours off the bug-hunting process.

Customer Service

Just as developers and support benefit on the technical side, in some cases Customer Service representatives (or selected members of the business) can use emulation to provide business or first level user support. When an end user has a complex question about an order or report, the service representative no longer needs to imagine their way through the issue, but instead can emulate that user, walk through the process, and see exactly what their user is seeing. This can be even more critical in systems where users see only a subset of the functionality or data available to service representatives.

Implementation

So emulation is a useful tool as well as a neat trick. Like many things, it is generally easier to bake this in from the beginning than to add it after the fact. If the user information is accessed in a consistent fashion in existing software, it is possible to squeeze in emulation logic and clean up the few places people cut corners and accessed users outside the normal context. If the user information is loaded and accessed at will throughout the application, adding emulation will be much harder (though there is some additional benefit in that it forces you to clean up your architecture a bit).

Sample Session Context Class

Altogether not that complex a code construct, although I'm sure it will grow more so over the lifetime of the application.

As long as we consistently access user information through the exposed User property in the session and user the exposed UserIdForAuditing property for auditing purposes, then most of the work is done for us. The only other piece we need is a button on the UI to start emulating and some logic to handle the danger below.

Dangers

There are two main dangers to watch for. The first danger is making sure emulation doesn't grant users the ability to promote themselves. Either emulation needs to be reserved for administrative users, or logic needs to be added to make certain levels or roles unavailable for emulation (or assignment during emulation).

The other main danger is that you now have a much greater chance (probably guarantee) that you will have the same user "logged in" in two locations at once. Most applications handle this just fine, but there are also many that cannot. Examples of this behavior are enforcing single location sign-ons, coded security that assumes multiple location means an account has been compromised, and making the mistake of storing session data keyed only to a user id instead of a unique session.

Don't Sprint Blindly...
(care of dmitriev.com)

Creating a model forces me to refine a concept down to it's simplest elements, forces me to face the unknowns that my mind has so casually been skipping over. When done well, a model communicates a clear idea and replaces not only the thousands words required to explain it, but the 9000 I would have wasted getting there.

I model to think through processes, question my assumptions, and provide guidance towards a solution. While it probably looks like something I threw together in about ten minutes, there are actually a lot of processes going on behind the scenes.

Purpose - What are We Drawing?

No Goal? Here's a Diagram.

As with all things, a diagram should have a goal. A model that isn't trying to communicate an idea is filler for a report no one is going to read anyway. A goal should be concise and limited to a single subject or perspective:

• The data flow from the end customer to our master data system
• An order-to-cash business process
• The functional architecture of a software application
• A graphic representation of our current state

Mess around with too many factors and at the end of the day a mess is all you'll have:

• The physical network topology combined with the disaster recovery plan and data flows between the systems
• The application architecture with defined user work flows and user experience elements

Or to translate: gobbledygook.

Constraint - Less is More

A goal provides me with my first constraint, and constraints are good. Defining constraints will keep my model simpler and consistent, which means the end message will be clearer. At the same time, a well-defined set of constraints will encourage creativity, providing a better end product.

Often my constraints will include things like not allowing connections to cross, only using a very simple set of shapes, restricting myself to only a few shades of color, or setting time limits. I'll define the perspective I want to use with my goal, whether it will be a topological map, a flow, or just a set of connected shapes.

This keeps me focused instead of playing with the entire palette of colors, shapes, and page sizes available in my favorite software tools.

Content - Work on a Temporary Surface

Even with constraints and a goal, I still don't know exactly where I will end up or what I will learn along the way, so I start on the whiteboard. With a whiteboard I can start diagramming out the pieces I know, add in new items or resolve question marks as I run into them, and easily combine and rearrange my thoughts. Some of my constraints will be ineffective at this stage, but natural constraints (like the number of markers I have and the board size) will replace them in helping my creativity and thought processes.

This stage is also where I figure out my wording. Because it's so easy to see the big picture (heh) on my whiteboard, I also get a good feel for when words are too specific, not specific enough, or possibly just not quite the right word for what I am trying to communicate. Instead of focusing on getting all the boxes lined up, I can focus on using clear and consistent terminology that will help support the final model rather than detract from it.

Medium - Where is it Going?

Content needs Context

As I make the transition from whiteboard to diagramming software, the last piece of the equation is to consider the medium I am going to use to communicate the model. A standalone diagram may put higher priority on further simplicity of shapes and colors, where a presentation model may put lower priority on text and higher priority on subtleties for deeper conversation.

Will there need to be a legend? Is font-size 8 going to be a waste of time or readable font? If I use subtle shades of color will it all print the same color or show up in gloriously rendered imagery on a 12 foot display? Will adding a cartoon get a chuckle in a presentation or a frown in an executive review? Can I include a picture of my cat?

The context the diagram will be communicating in will determine the last set of constraints.

Terminology - The Wrong Word Invalidates the Model

Using the right terminology,
thanks lolcaption.com

General wording may have been roughed in on the whiteboard and some really good words may have been chosen, but these now need to be examined in light of the future medium as well as the audience. In many cases, using a word out of context can distract my audience or even negate the model's message entirely.

That being said, endless anxiety over perfection is nowhere near as good as a cool beer at the end of a work day, so we need to strike a balance between working through the night and achieving good enough. So I'll be careful, in general, when using customer terminology and try to be pragmatic in my search for the perfect name for the third box from the left.

Composition - Additional Layers of Meaning

The last stage of the model, having transferred it from whiteboard to software and applied corrections to terminology, is to add some depth that supports the initial concept.

Colors, re-arranging layout to alter proximity, fonts, and even line thicknesses are all tools I use to add subtle depth to a model. If I am planning on presenting the model, I can start the discussion on the general message of the model and dive into these subtleties as the discussion progresses. A thicker line between two systems can communicate greater bandwidth or a more secure transport layer. A common shading of colors between multiple objects communicates a relationship or similarity. As with the stages before, I try to use constraint. Applying the whole palette and a different shape for each object may seem fun, but it's going to communicate confusion (and possibly a desire for medication).

While working with the composition, I will also create temporary versions to play with drastically different layouts or shapes. This gives me a fresh look at a concept that has undoubtedly been on my whiteboard for days, giving me an opportunity to catch last minute holes or simply provide alternative layout options.

Sounds Like a Lot of Work...

There are different levels of work involved in modeling. In some cases even finding the time to stop doing and try to draw an idea may seem like a waste.

Not looking ahead?
Diagram for that too...

Jumping right into any project when we can't draw a high level summary means we're spending time and resources on something we can't adequately define. It doesn't matter how fast we're moving if we're spending that time running in random directions and ignoring cross-traffic.

I often build small projects to help understand and grow my skills as a developer, architect, and all-around technologist (as may be apparent from the wide range of topics I post on). Today I worked with a combination of Visio and Visual Studio to build a sample project to play with the Passive View concept and to help grow my own understanding of the concept. This post will cover the Visio side of my learning-curve.

You can read more about Model View Presenter at Wikipedia and MSDN. Perhaps the best information can be found on Martin Fowler's site, where he has separate write-ups on Passive View and Supervising Controller.

Passive View

Passive View is a subset of the Model-View-Presenter pattern. In Passive View, the interface is responsible for handling interface-specific logic, such as figuring out how to put a value in a textbox or react to events from button clicks, but all actions and logic outside of the raw UI are sent to the Presenter to execute or manage. The Presenter is responsible for calling business methods in the Business model and updating the data that is available in the View.

Basic Model-View-Presenter Diagram

From the outside in, the architecture for Passive View looks something like this:

• UI - The User Interface reflects what is going on beneath it by implementing one or more View interfaces
• Presenter - The Presenter receives interactions from the UI or Model and updates the Views it is attached to
• Model - The model is a facade or black box in our diagram, behind which is a business logic layer and data layer

In a flat architecture we would collect data from the interface, perhaps do some business and data validation, and then save it directly to a database using stored procedures or inline SQL. Defining a data access layer (or data model like entity framework) allows our application to operate on cohesive, defined objects that are meaningful to the application and stored and retrieved consistently. Defining a business logic layer allows us to centralize business rules that operate on entities in our application in a manner that is consistent with the business and internally consistent in the application, minimizing the risk that occurs when making changes to the business flow. Separating the logic of populating inputs and responding to button presses on the UI from the information being communicated to the end user and conceptual responses to their input allows the system to interact with the user consistently across any number of interfaces into the same application.

The definition of each level increases our ability to automate testing and supports greater Separation of Concerns.

Implementing a Sample Project

My learning exercise has been the the creation of an ASP.Net search page that allows an end user (customer) to search for finished products from the AdventureWorks sample database. The architecture and design decisions were done as an exercise in Visio using simple shapes and layouts.

My example application has several functional and non-functional requirements:

1. Functional - Display product number, name, list price, and available quantity in tabular format
2. Functional - Provide a basic search input and button to search product names
3. Non-Functional - Implement an M-V-P pattern - Obviously the purpose of this whole exercise
4. Non-Functional - Use a simple model stack that can be easily replaced with a Service-Oriented one at a later time
5. Non-Functional - Build with the idea that we will later create a Silverlight or WPF front-end
6. Non-Functional - Make pretty pictures for article

My unwritten, final requirement was to finish the whole thing in half a day, though luckily I didn't define whether I intended that to mean 4 hours or 12.

Initial Architecture

To start I created a diagram of the application architecture:

More Extensive Model-View-Presenter Diagram

The purple layer is my presentation layer, which reflects the View. The blue layer is my Presenter layer which contains the logic for interacting between the end user and interface as well as a definition, or contract, of the information available in the View. The Green is the Model (or is behind the model, depending on your viewpoint) and exposes business functions and data entities for the Presenter to interact with.

Class Layout

Once the high level diagram was completed, I could approach the task of creating some base classes and interfaces to use in implementing the project.

• Model.IModel - Generic Model Interface to expose business calls to Presenters
• Presenter.IView - Generic View Interface that all Presenters can interact with and all screens implement
• Presenter.BasePresenter - Generic Presenter class that all Presenters will implement

To keep the project to a single morning but also allow the ability to come back and build a more architecturally sound solution, I implemented the Model in a very basic fashion that was referenced locally by the Presenter project and makes direct calls to SQL Server using ADO and parametrized, inline SQL. This buys me the benefits of having a well-defined Model (via the interface) but allows me concentrate my time and effort on the learning part of the project (ie, the M-V-P interaction and structure). Defining the model interface also leaves me open to come back and replace it with better separated code and the ability to create a model that acts as a facade to a service stack, instead of local DLLs.

• Model.BasicModel.Model - Basic implementation of a model that will interact with AdventureWorks on SQL Server
• Model.Entities.Product - A Product Entity that can be communicated between an IModel instance and Presenter
  
• Presenter.ProductSearchPresenter - A Presenter to manage product search interface
• Presenter.IProductSearchView - A view of the data involved in a product search
• ProductSearch.aspx - A web page that implements the IProductSearchView and interacts with the ProductSearchPresenter

My final Visio diagram ended up looking like this:

Diagram of Example Application

In this case the left side represents basic components (bases classes and interfaces) that are used to define common structure or contracts on the right side.

The Code

For the purposes of the example project, my view has properties for Search Text, a Search Count (number of results), Results (a generic list of the Product entity), and a boolean indicating whether there are results to display. My Web Form implements these properties, tying them to elements on the screen.

As the presenter populates properties in the view, the information is automatically reflected on the page. The actual logic of how the business functions are called and populate those properties are neatly packaged up in the Presenter and View interface and very little logic occurs in the actual web form.

To create a unit test, we define a simple view that implements the view interface, execute the presenter logic, and verify the properties are populated the way we would expect when the same presenter calls are made from the interface.

Extending the Architecture Further

Extending the application to display product search in a different manner would only require the addition of a new interface that also implements the Product Search View. A Silverlight front-end would only require creating the basic project, implementing the product search View, and wiring the new interface controls to the view properties. To replace the direct mode reference with a service reference, we could create a service facade that implemented the IModel interface, connected to a local or remote WCF service behid the scenes to handle the real model logic. And finally, instead of counting on our QA department to test all of the application interactions, we can create unit tests directly against the Presenter and Views to ensure that all of the interactions below the top surface of the application are happening consistently and to our expectation.

Your Turn

Getting this much of the architecture working is a good first step. I took a number of shortcuts on the BasicModel class in my example, but I now have a functional Model-View-Presenter application to play with. Hopefully there was enough information in the article to interest you in trying this out on your own. I urge you to read the articles linked in the top of the post (or several more in my Model-View-Presentation bookmarks) and come up with your own diagrams and sample project. Even doing a small project will force you to run into questions and considerations you wouldn't have had by simply reading about it, not to mention unrelated tidbits you will pick up along the way (for instance, I also learned about ObservableCollections today).