There are about a dozen software packages on the market to help traffic engineers determine what intersection improvements should be made.  They all talk about calibrating the models to existing conditions.  But there's a real problem with needing to calibrate these software models.

How do you calibrate to something that doesn't exist?  How do you take an existing all way stop sign control intersection and calibrate to increased traffic forecasts with traffic signal or roundabout control at an intersection?  

The increased traffic alone my change driver behavior at the intersection.  But then layering on different lane configurations and traffic control make the existing condition irrelevant.

Bryant Ficek at TKDA and I have been talking about this for quite a while, so we decided to do some research.  We couldn't come up with any research on the topic (if you have it - please share, and yes we realize the whole Highway Capacity Manual is based on mountains of research).  

So we decided to collect turning movement counts, delay and queuing data at a two lane roundabout in Washington County.  Then build models of a "generic" multi-lane roundabout with the software's defaults to see how different software packages performed.

Yes, yes - we understand the software companies are going to scream at us - you need to calibrate!  But our response is - you can't calibrate to a future alternative that doesn't exist right now.  We depend on you to build software models with parameters that will get us close to what that future condition would be.  Or at least tell us what data we should collect locally to determine our region's defaults.

Bryant built models with Rodel, HCM 2010, Simtraffic, Synchro, and Vissim.  It's not that easy to come up with what defaults to use in each software package.  Here's Bryant's write-up of the inputs for each model:   Download Roundabout Software Evaluation.  And here's a spreadsheet that compares all of the results against the actual p.m. peak hour data measured - Download Comparison Matrix. 

The punchline is that none of them nailed it.  They were all reasonable on the delay results, but the queuing results were pretty far off.

This is just the start of this study.  Bryant and I want to collect data at a bunch of intersections with different types of traffic control and then test out all of the models.  It would be great if we got other folks to pitch in with results from around the country or even the world.

But we didn't want to get to far on this if there's a fatal flaw in our reasoning or methodology.  So this post is the start of a conversation.  What do you think?