Understanding the Challenge of a BIM Challenge

The histories of BIM and CAD are littered with shoot-outs, comparison tests, and pissing contests. By now you probably know my favorite Mine vs Yours, ArchiCAD vs Revit saga: The Unkillable Beast with a 1,000 heads. Though now it’s more like 1,400 heads. A lot of people are annoyed by this thread, but as I’ve said before the longer it goes on the more I love it and the more I learn.

Last week (early February 2013) a group of BIMnerds with noble goals and aspirations thought “Let’s do it! Let’s put these applications in a head-to-head contest!” The goal was to systematically compare ArchiCAD and Revit, and maybe even some other programs (the list now includes Vectorworks and the search is on for Tekla users, Bentley users, etc.). No one has succeeded in recent years to compare these applications, but to quote Arrested Development “It might work for us!” (spoiler alert: it didn’t).

It’s like comparing Apples and Oranges

No that’s not good enough.

It’s like comparing Koalas and Whistling

As I get further down the BIM path, or at least as I get further into unraveling what BIM means, I realize that comparing software is a red herring. Trying to declare one as superior over the other has little utility. Much like in earlier posts we dissected what BIM means (and what its various levels are), we need to look at BIM software with a similar critical eye. We need to get to the root of what it means to be a BIM application. Then we can better understand the value of our tools. So let’s daydream about what the BIM Challenge should or could explore.

What’s the Goal?
I really dislike the idea of saying Program A is better than Program B. What good does that do anyone? Are all the users of one program going to revolt and switch to the other? I doubt it. No, we should determine what a BIM application needs to accomplish. And if it can do those things, then it is worthy. Period. Instead of comparing software directly to each other, which by the nature of yearly product releases, is an endlessly moving target, we need to compare them to the Platonic Ideal of BIM Software. We need to think not about what our current tools CAN do, but what our dream tools SHOULD do. And then compare Revit, ArchiCAD, Vectorworks, etc. to that.

Who’s this for?
The BIM Challenge should be for the enlightenment of the entire AECO community. Of course for me the primary focus are the architects, but that’s just my bias. It’s about the love of BIM and sharing that with everyone else. It shouldn’t be for bragging rights. It isn’t a zero sum game. It shouldn’t be about winners and losers. Like I mentioned above, it should be a competition in parallel against an ideal.

What should the challenge look like? How do you remove the varied natural talent of the teams?
Replicating a full project is fraught with too many distractions (does project type really matter? does size? does construction type?). Perhaps it’s not One Grand Challenge but a lot of little ones. The challenge becomes not about the random teams that volunteered for whatever personal reasons, but little exercises that many people can do. Pros, amateurs, non-users, ex-users… The results could be aggregated. This isn’t a groundbreaking concept, it’s just standard run of the mill data collection. Now there’s a clue to why BIM Challenges and CAD shootouts in the past fail or are inconclusive. We lack enough data. We forget that answers can’t be deduced from single iterations.

How should the results be shared?
We live in a digital world. Therefore the results should be shared digitally. Physical presentations are nice, but counter to the larger mission. The challenge should NOT be about pageantry or spectacle. Effort should be focused on sharing the information with the widest audience. This suggests a website. Probably a new website so that the challenge doesn’t look like it is being run by one particular user base (ie, it’s not a subpage of a specific user group or software company). Fortunately as I write this, www.BIMchallenge.com seems to be open. As does blogspot.BIMchallenge,  BIMchallenge.tumblr.com, and plenty of other free domain names. Furthermore, instead of live events, everything should be recorded. There’s no reason every piece of the challenge can’t be shared as a video on a YouTube Channel. I’m sure that www.youtube.com/BIMchallenge is available. The BIMChallenge should be about collecting data first. Others can do the analysis by watching the results over and over and over again.

Frequency: One Time or Repeatable?
Looking to help the entire community, both BIM users and future BIM users; competing against an ideal, not each other; sharing the data online via a dedicated website, forum, or YouTube Channel, all leads me to the most important feature of the BIM Challenge. It needs to be about setting the parameters so that the Challenge can be run again and again and again. If we compare ArchiCAD 16 to Revit 2013 to Vectorworks 2013, our conclusions have a shelf-life of half a year. Sometime this year ArchiCAD 16 will be superseded by ArchiCAD 17. We already know some of the details. And we all know that Revit 2014 and Vectorworks 2014 are coming. And with Trimble now owning SketchUp, who knows what SketchUp 9.0 will be able to do (whenever it appears). And there’s always the (high) probability that some new player will appear. The Challenge needs to be forward looking and able to accommodate new entrants. It needs to be repeatable and open. Not only do we need some way to compare Revit and ArchiCAD and Vectorworks in a meaningful way, we also need to be able to compare ArchiCAD 16 to ArchiCAD 17 and ArchiCAD 17 to ArchiCAD 18.

This ability to compare not just between software products but also vertically within one product line is the greatest promise. Imagine if we can look at ArchiCAD 19 and say “look we might not have gotten that one long standing wish fixed, but-wow-look how much closer we are to the Ideal BIM Software!” Furthermore, if the BIM Challenge does a good job it can provide a road map for the software companies. The developers will be able to see clearly what the users value and how their current offerings stand up. And wouldn’t that be great? If the result of the Challenge helps each platform (and competitor) improve. That’s what I want.

What Else?

How do we describe the Ideal BIM Software?

Let’s help by doing some open source brainstorming. Do you have some thoughts about what you think needs to be in the BIM Challenge? What do you think of my ideas? What does your Ideal BIM Software look like? Share what you have to say in the comments.

Follow Shoegnome on Facebook and Twitter for more on being an Architect in the 21st Century.