Heavy Civil, Big Data

Calculating cant alignments in IFC

2024-06-14T00:00:00-04:00

Rule ALA003 of the buildingSMART validation service looks at individual alignment segments to confirm that the same geometry type is used in the business logic and in the geometric representation.

During development, there was a question raised regarding the proper entity type to be used in the representation segment that corresponds to a linear transition of cant in the business logic. The ensuing discussion in the implementer’s forum in IFC4.x-IF#145 lead to considerable confusion on my part.

Therefore I decided to develop this notebook to do some calculations for the model in question.

In [1]:

import os

import numpy as np

import ifcopenshell
import ifcopenshell.geom as geom

s = geom.settings()

IN_PATH = os.path.join("..", "assets", "models", "alignment_validation")
FILE_NAME = "ACCA_sleepers-linear-placement-cant-implicit.ifc"
IN_FILE = os.path.join(IN_PATH, FILE_NAME)

model = ifcopenshell.open(IN_FILE)

In [2]:

# per inspection, this model contains only 1 alignment
align = model.by_type("IfcAlignment")[0]
align

Out[2]:

#2497=IfcAlignment('2rc47I60124e8RihHvjEbb',#1,'Test Alignment',$,$,#2746,#2749,$)

In [3]:

prod_rep = align.Representation
prod_rep

Out[3]:

#2749=IfcProductDefinitionShape($,$,(#2750,#2751))

In [4]:

shape_reps = prod_rep.Representations
shape_reps

Out[4]:

(#2750=IfcShapeRepresentation(#2752,'Axis','Curve3D',(#2668)),
 #2751=IfcShapeRepresentation(#2752,'FootPrint','Curve2D',(#2572)))

In [5]:

seg_ref_curve = shape_reps[0].Items[0]
seg_ref_curve

Out[5]:

#2668=IfcSegmentedReferenceCurve((#2669,#2681,#2695,#2709,#2727),.F.,#2630,#2723)

In [6]:

grad_curve = seg_ref_curve.BaseCurve
grad_curve

Out[6]:

#2630=IfcGradientCurve((#2631,#2650),.F.,#2572,#2647)

In [7]:

comp_curve = grad_curve.BaseCurve
comp_curve

Out[7]:

#2572=IfcCompositeCurve((#2573,#2586,#2598,#2611),.F.)

In [8]:

# traverse the alignment cant layout

for _ in align.IsNestedBy[0].RelatedObjects:
    if _.is_a() == "IfcAlignmentCant":
        cant_layout = _

cant_layout

Out[8]:

#2541=IfcAlignmentCant('2uexyjTX5CfOJI4h9D7ngN',#1,'CANT',$,$,#2743,$,1.5)

In [9]:

for seg in cant_layout.IsNestedBy[0].RelatedObjects:
    dp = seg.DesignParameters
    print(dp)

#2551=IfcAlignmentCantSegment($,$,0.,400.,0.,0.,0.,0.,.CONSTANTCANT.)
#2557=IfcAlignmentCantSegment($,$,400.,49.999993741124,0.,1.,0.,0.,.LINEARTRANSITION.)
#2563=IfcAlignmentCantSegment($,$,449.999993741124,100.000006258876,1.,1.,0.,0.,.CONSTANTCANT.)
#2569=IfcAlignmentCantSegment($,$,550.,400.,1.,0.,0.,0.,.LINEARTRANSITION.)
#2737=IfcAlignmentCantSegment($,$,950.,0.,0.,0.,0.,0.,.LINEARTRANSITION.)

From inspection, we see that the initial transition in the cant layout, #2557, is a linear transition from 0.0 to 1.0 m of cant from distance 400.0 to 450.0.

Therefore, create a function to calculate the cant values at 5 m intervals along this transition. We’re helped by the fact that there is no change in the vertical alignment along this portion of the alignment, meaning that the calculated elevations are 0.0. Therefore, the z coordinate of the segmented reference curve corresponds to 1/2 the total cant.

The low rail is the axis of rotation and the total cant is the distance the high rail is elevated above its normal position. Therefore at the centerline of rail, where the alignment is located, the z coordinate is 1/2 the total cant amount.

Said another way:

segmented_ref_curve_z = gradient_curve_z + 0.5 * total_cant

if gradient_curve_z == 0:
    segmented_ref_curve_z = 0.5 * total_cant

In [10]:

def evaluate_u(curve: ifcopenshell.entity_instance, dist_along: float) -> np.ndarray:
    pwf = ifcopenshell.ifcopenshell_wrapper.map_shape(s, curve.wrapped_data)
    t = pwf.evaluate(dist_along)
    ar = np.array(t)
    x = ar[0][3]
    y = ar[1][3]
    z = ar[2][3]
    return np.array([dist_along, x, y, z], dtype=np.float64)

In [11]:

# set up the distances to be evaluated
distances = np.linspace(start=400.0, stop=450.0, num=11, endpoint=True, dtype=np.float64)

coords = np.array([evaluate_u(seg_ref_curve, d) for d in distances])

This is obviously far from a correctly pythonic use of numpy, but it gets the job done.

In [12]:

# slice a 2d array of [distance_along, total cant]

cant_coords = np.array([coords[:, 1], 2 * coords[:, 3]], dtype=np.float64).T
cant_coords

Out[12]:

array([[4.00000000e+02, 0.00000000e+00],
       [4.05000000e+02, 9.99950129e-02],
       [4.10000000e+02, 1.99990026e-01],
       [4.14999997e+02, 2.99985039e-01],
       [4.19999986e+02, 3.99980052e-01],
       [4.24999957e+02, 4.99975064e-01],
       [4.29999892e+02, 5.99970077e-01],
       [4.34999766e+02, 6.99965090e-01],
       [4.39999545e+02, 7.99960103e-01],
       [4.44999180e+02, 8.99955116e-01],
       [4.49998611e+02, 9.99950129e-01]])

Conclusion¶

It still may not make sense to me, but the IfcOpenShell implementation clearly calculates the correct values of cant for this example model. Therefore I’m happy to stand corrected, with thanks to RickBrice and peterrdf for their patience and explanation.

A Framework for Digital Delivery of Infrastructure Projects

2023-07-01T00:00:00-04:00

Table of Contents

Thesis
Current state
- PDF - based delivery
Common, Open Format - IFC 4.3
Data must meet specific requirements
- Information Delivery Specification
- Specific Requirements for Project Information
Model as Legal Document
Conclusion

Thesis

Infrastructure project delivery is shifting from drawing-based PDF to data-based models. Here’s a framework for digital delivery that parallels the current state of affairs in the hopes of minimizing discomfort and speeding adoption.

Current state

The vast majority of my experience in heavy civil design and construction has been in my home state of Indiana. Therefore, I’ll use the Indiana Department of Transportation (INDOT) as my basis of discussion. Note that I’m not speaking on behalf of the department or trying to come off as an authoritative voice - just making observations for the sake of discussion.

PDF - based delivery

The vast majority of Plans, Specs & Estimates (PSE) are currently transacted via PDF. This is particularly the case if we focus specifically on plans. INDOT has never dictated or forced consultant designers to use a specific CADD platform. They do publish the resources of documentation of the Microstation / OpenRoads civil workspace that they use for in-house design, but consultants are not obligated to use it.

Plan submittals are required to be made via PDF and must adhere to certain criteria:

maximum file size
look and feel (fonts, lineweights, abbreviations)
sheet naming and scales
sheet titles and arrangement order
sheet numbering for addendums, revisions and construction changes

The title sheet of each plan set has additional requirements:

project identification - name, project number, route, description of the work, features crossed
geospatial location - latitude and longitude
construction Contract Number
Des Number

Common, Open Format - IFC 4.3

Digital delivery is best accomplished via IFC 4.3. Like PDF, it is a published data format not proprietary to any specific software system or vendor. This means that there are multiple options for viewing model data - including those that are free (no cost) for read-only viewing.

This also means that there are multiple authoring tools available. Like the current state of affairs, designers have the ability to choose their preferred modeling platform for generating and checking design model data.

Lastly, like PDF, IFC is an ISO standard. Specifically, IFC 4.3 is currently in candidate status for final approval as an updated ISO standard, ensuring that model information in this format will continue to be accessible for multiple decades in the future.

Data must meet specific requirements

Information Delivery Specification

Information Delivery Specification (IDS) is a relative newcomer to the openBIM scene. IDS is a machine-readable specification that defines the digital governance requirements for an information exchange. You could think of this as Model View Definition (MVD) - lite.

You could also think of this as a BIM checklist that can be performed automatically by supporting software to validate IFC data against project or client standards. In my experience, IDS is best suited to checking attribution via the presence of specific property set definitions, property sets, and values.

Making our analogy back to current state of affairs, IDS is similar to current “look and feel” requirements for plans. You can use the authoring tool of your choice, but the PDF output must match the published example plans for fonts, abbreviations, and sheet ordering. I have focused on this analogy specifically because I remember a “look and feel” check as a specific part of our QC process and documentation from earlier in my career.

Specific Requirements for Project Information

Like the requirements listed above for a title sheet, digital deliverables should include similar project information:

geospatial positioning
contract number
project identification / Des Number
route name and reference posts / mile markers
features crossed

All of these can be built in to consistent QC processes via IDS.

Model as Legal Document

There is an industry trend towards replacing traditional construction plans with the model data itself. Certain agencies have already piloted this approach with varying degrees of success. I’ve observed a fair amount of concern from designers about this approach, stating that they don’t feel comfortable certifying model data itself.

However, they are already doing this when certifying PDF plans. That is to say that they are not certifying the binary content of the files themselves, but rather the presentation of that binary content in a viewer such as Bluebeam Revu.

It’s not different with IFC - you are certifying the representation of the model content as presented in an acceptable viewer. In fact, the situation is even better with IFC because it is a human-readable plain text format. You can open it in any text viewer and review the data itself. Try doing that with a PDF!

Conclusion

2D plan set is to PDF as 3D model is to IFC.

Boiling things down to that analogy will go a long way towards calming fears about this new approach and ultimately getting practitioners on board and speeding overall adoption in the industry. This sets the stage for better cost control and schedule performance while also paving the way towards better as-built records that flow into asset management systems and set the stage for infrastructure digital twins.

Automation investment and payoff

2023-02-11T00:00:00-05:00

I often find myself considering whether or not to automate a particular work process. In retrospect, I believe this comes primarily from some experiences very early in my career, as well as an overall lifelong interest in computing and software development.

Recently, I had a pleasant experience regarding the amount of time invested in automating a process relative to the amount of time needed to recognize a return on that investment.

Pitfalls¶

It’s easy to fall into the trap of applying automation to any task, particularly if it involves a lot of data. At the outset, it seems obvious that this approach will ultimately be more efficient. However, that’s rarely the case.

Part of the problem is the fact that you need to understand the problem fully before you can efficiently automate it. Many times what seems to be a straightforward process ends up having twists and turns and edge cases along the way.

Most importantly, if a task is only going to happen a few times - as opposed to an ongoing, reptitive need - then it is a terrible candidate for automation.

Obligatory xkcd reference¶

Of course, one of the premier geek references also addresses this conundrum with a handy table to be used as reference:

The Problem Statement¶

The business problem at the heart of this story relates to data published regularly to a website. The website is public and is updated monthly with the data of interest. This data is published as a series of PDF files, with approximately ten files - sometimes more, sometimes less - posted in any given month.

Automated processes for extracting the data were already in place, but the downloads from the web site were still a manual process. When I was keeping up with the monthly downloads as a manual process, things weren’t too bad. However, in the last two years the downloads have fallen by the wayside until the end of the year.

When you’re dealing with downloading an entire year’s worth of data, it’s not quite as tolerable to download the 100+ files manually all in one sitting.

The Solution¶

python, of course! Looking purely at the URL patterns for the site it seemed obvious that the front end was supported by a REST API on the backend that would return data according to the query parameters passed via URL. Reviewing the network traces in the browser confirmed this hunch, as well as provided the schema for the data returned from the API.

The trickiest part was teasing out the unique ID required in the request for the PDF. Once that was in place, I developed a few routines using the wonderful requests package, and fun was had by all.

The Calculation¶

So now, for the good part. As mentioned, this turned out to be the rare instance where the investment in automation paid off automatically. Let’s crunch the numbers. To do so, we’ll use the pint library to track our various denominations of time.

We’ll also define a unit pdf equal to 100 Joules. The scalar amount is arbitrary, but the SI unit of work was selected because in the abstract sense, I want to accomplish a work task.

In [1]:

import pint

ureg = pint.UnitRegistry()
ureg.define("pdf_file = 100 * Joule = pdf")
q = ureg.Quantity(10, "pdf")
q

Out[1]:

10 pdf_file

It turns out that there were 112 total pdf files to be downloaded to cover the entire current year. From timestamps on the files for the previous year, I saw that it took me 34 minutes to download 24 files when I was doing it manually.

In [2]:

manual_files = ureg.Quantity(24, "pdf")
manual_time = ureg.Quantity(34, "minutes")
manual_time = manual_time.to("seconds")
manual_rate = manual_time / manual_files

print(f"Manual rate is {manual_rate}.")

Manual rate is 85.0 second / pdf_file.

After I had developed the automated process, I was able to download the 112 files in 25 minutes.

In [3]:

automated_files = ureg.Quantity(112, "pdf")
automated_time = ureg.Quantity(25, "minutes").to("seconds")
automated_rate = automated_time / automated_files

print(f"Automated rate is {automated_rate}.")

Automated rate is 13.392857142857142 second / pdf_file.

So now we can calculate the savings realized by developing the automated process, which took 1 hour and 45 minutes in total.

In [4]:

development_time = ureg.Quantity(1, "hour") + ureg.Quantity(45, "minutes")

unit_rate_difference = ureg.Quantity(manual_rate.magnitude - automated_rate.magnitude, "seconds") / ureg.Quantity(1, "pdf")
print(f"Unit rate difference is {unit_rate_difference}")

gross_time_savings = (unit_rate_difference * automated_files)
print(f"Gross time savings is {gross_time_savings.to('minutes')}")

net_time_savings = gross_time_savings - development_time
print(f"Net time savings is {net_time_savings.to('minutes')}")

Unit rate difference is 71.60714285714286 second / pdf_file
Gross time savings is 133.66666666666666 minute
Net time savings is 28.666666666666668 minute

So not only did the investment in automation pay itself back immediately, but it also provided an extra half hour of “free” time. python for the win!

Conclusion¶

This exercise turned out to be the rare instance where developing automation pays for itself immediately. Even better, this investment will continue to pay itself back over the coming months and years as more data is posted and needs to be extracted, transformed, and loaded.

Why was this a success? As mentioned, one of the downfalls with automating a process too soon is that you don’t understand all the ‘gotchas’. In this case I had already spent multiple years doing these tasks manually.

Secondly, the development went quickly due to my understanding of web development and REST API’s in general. All of my previous dalliances in web scraping had focused on using selenium and a headless browser to simulate a user picking links and navigating through pages. Now that I have a good deal of experience with full-stack development I was able to shortcut the user interface entirely and go straight to the data exposed by the API.

SmartHouse.init()

2022-09-08T00:00:00-04:00

Table of Contents

Prologue: Meross Smart Switch
Smart Thermostat
Hardware Installation and Unexpected Re-wiring
A new life for an old Pi
Initial thoughts
Next Steps

I recently took some initial steps towards home automation by installing a smart thermostat and setting up Home Assistant. Here’s a rundown of my experience.

Prologue: Meross Smart Switch

Actually, I guess I started down this path a few months ago. In preparation for a family vacation to Seattle, my father-in-law purchased us a set of four Meross smart outlets. These units are Wifi-enabled but also have an actual manual button on the side. This is nice - you can turn things on and off “The Old Fashioned Way” without needing an app.

Before leaving on vacation, I went about setting up these outlets in four different rooms in our house and connected a lamp to each. Initially I named each of the outlets based on their serial numbers. Pretty quickly I realized that this wasn’t a good idea and revised the names to something much more sensible.

Once everything was hooked up and working, I had quite the good time turning each of the lamps on and off via the app. However, I forgot to set a programmed schedule before we left - which pretty much defeated the purpose of getting them set up in the first place.

However that turned out to prove the usefulness of these outlets. We were a few days into vacation when I was riding on I-5 and remembered that I hadn’t programmed our smart lights. So I pulled out my phone, did some quick reading of the docs, and configured a “on at dusk, off at sunrise” program for the four lights from 2,000 miles away.

This turned out to be much easier than what we had done in the past. We have a single timer that plugs in to an outlet, and I had configured it prior to leaving on multiple previous trips. However it was difficult to use and I had to look up the manual each time we wanted to use it. Having the Meross smart outlets makes this much easier moving forward.

And, as much as I hate to admit it, since returning from vacation I have turned off the reading light in our room via the app when I was too lazy to reach over and do it manually.

Smart Thermostat

A few months ago our natural gas provider sent information on an incentive program for smart thermostats. There were a number of offerings ranging from free to substantially discounted. We decided to go with the ecobee lite model, which was discounted roughly 70%.

I read the installation instructions a few times before unpacking the hardware, and there was some ominous language about “your thermostat is probably compatible”.

Hardware Installation and Unexpected Re-wiring

And so it happened that early on a Saturday afternoon I took the plunge and unboxed the hardware. The instructions estimated that installation would take roughly 30 minutes. Given my previous experience with DIY around the house I figured that would roughly correlate to three hours, give or take.

Right off the bat there was some drama with children facing the possibility of an afternoon with no screens, but it turned out that our furnace had a handy shutoff switch. With power turned off, I successfully removed the cover on the furnace and the cover on the thermostat.

Sure enough, the wiring didn’t match anything in the instruction manual:

Some internet research confirmed that existing HVAC system utilizes a manufacturer-specific “communicating” thermostat. I feared that we were potentially dead in the water, but ultimately found a very helpful thread from a pro that explained the basics of our current Carrier system. The pro ended up talking the original poster through the necessary steps for implementing a Wifi-compatible thermostat.

Most importantly - in my eyes at least - was the note that this reconfiguration would not violate any warranty provisions. And the internet is always right, so I was ready to plow ahead.

I took another look at the wiring in the wall behind the original thermostat and noticed that there were three or four additional wire strands available, should I need them. Tracing things back at the furnace control panel confirmed the same. I was also easily able to identify the same four wires at the furnace.

Without too much trouble I was able to remove the plug and disconnect each wire. Based on the ecobee manual I just needed to match the wires based on color, with the exception of the white C power cable. I moved the wires over to the conventional terminals on the front of the furnace board and tightened everything down.

I had a little bit of confusion over the Y and Y2 options, but figured that Y was the default and would make the most sense.

Upstairs, I did a similar bit of wiring to connect everything based on the defaults shown in the installation manual. This was pretty straightforward, although I did wonder if I should be using more than just these four wires.

I took a hard look again at where I had landed on the back of the thermostat and realized I needed to go back downstairs and make a small tweak at the furnace.

I flipped the power back on at the furnace and nothing threw sparks, so I figured that was a good sign. Back upstairs, I connected the thermostat per the instructions and let it power up. Everything went smoothly there and before long everything was up and running.

The initial program was pretty basic - hold 78 during the day and cool off a few degrees at night. Our house is not typically empty during the day, so there was no need to vary things based on occupancy.

I did notice that the thermostat seemed to only have two states - cooling and off. That seems to confirm my initial hunch that I will need to attach more of the available wires at both the thermostat and furnace to be able to manage heating as well. I probably should research this some more, but at this point in the build I was just about ready to call it a day.

I went ahead and created an account with ecobee and explored their website.

They seem to have some interesting data-sciency type features called HomeIQ. But you need to have at least a month worth of data before it starts providing any feedback. This fact makes it hard, at least on desktop, to see where you can do system monitoring online.

However if you scroll down past this information and look in the lower right corner you can find the link to system monitoring information.

Here’s where you can see minute-by minute stats plotted against indoor and outdoor temperature

as well as humidity.

If you come back later (as I did for this blog post) you can see charts of data over wider time spans such as weekly and monthly. One interesting graph was titled “Weather Effect” and plotted average daily temperatures and total system run time.

The website also allows you to download data locally as CSV. Once Home Assistant is connected to the ecobee cloud it will periodically download and save this information as well, so in my use case there doesn’t seem to be much value in a manual download.

After exploring the web site I spent a little bit of time messing around with the app on my phone. It was amusing to wait for the furnace to shut off, then slide the temperature back down and hear it kick back on.

A new life for an old Pi

While it was nice to be able to adjust the thermostat from an app or web site, my true interest was in gathering sensor data. My hypothesis is that by tracking the indoor temperature and humidity against ambient temperature in humidity I would be able to establish some relationship between the outdoor environment and the house’s ability to regulate the indoor environment. However there are also a few potential automations that I have in mind, particularly related to managing our attic fan and triggering it to run based on temperature.

Based on a recommendation from a co-worker, I decided to standardize on Home Assistant. Not only is it open source, but it is also primarily written in python. Those characteristics made it hard to resist. The recommended installation method was via burning their OS image to an SD card for use with Raspberry Pi.

I have an RPi 3 that has been lying about for a number of years. I initially purchased it for use with Lego Mindstorms controllers, but it has mostly sat around unused. The installation went smoothly and I soon had the Pi up and running.

I ended up following the suggestions by both installing the iOS app and reading the founder’s vision for the platform. Before long, the installation had finished and it was time to dig in.

There were some initial steps required such as creating a username / password and setting a location for your home. The iOS app required Location Services to be enabled, which was a bit interesting. I initially tried to disable this but the app would not open unless location was enabled. So I begrudgingly set it up for only while using the app.

Not surprisingly, the first device to be configured was my phone. HomeAssistant recognized my phone right away and created a number of properties (I know that’s not the right terminology but I can remember the proper term) and started tracking things like battery state and whether or not the phone was charging. I could potentially see some value there but it also tracked things like local IP address and whether or not it was connected to Wifi. It didn’t take long before I had all of that functionality turned off.

Setting up the thermostat was pretty straightforward and the HomeAssistant documentation was clear. It required registering as a developer with ecobee, then using a PIN to authenticate the application.

Initial thoughts

Thus far I have to say I am a fan of HomeAssistant. The biggest reason is that HA strives to be a unifier of multiple products across multiple manufacturers. Also I appreciate that it is focused on local data storage as opposed to cloud-only. Currently all of the thermostat data is being pulled from cloud storage, but accordingly to some research it appears that I might be able to convert it to local-only via Homekit.

Lastly, it doesn’t hurt that HomeAssistant is primarily written in python - my programming language of choice.

Next Steps

So, that’s the story of our new smart thermostat and HomeAssistant as the overall smart hub for our house. Now that I’ve accomplish a basic setup as a starting point, next steps will include:

configuring SSH to connect to the Pi from other hosts
setting up an external hard drive for redundancy and data backups
potentially moving to a different different SD card that is optimized for heavy I/O

Book Review: Python Testing with Pytest

2022-08-01T00:00:00-04:00

Table of Contents

Python Testing With pytest
My Take
Strengths and Weaknesses
- Pros
- Cons
Conclusion

It’s a bit embarrassing that I haven’t posted anything in more than a year. I need to get back on track with at least semi-regular blogging. What better way to do so than to review the first hardcopy book I’ve purchased in 10+ years?

Python Testing With pytest

I’ve been aware of this book for years. It was written by Brian Okken, who co-hosts the Python Bytes podcast. I had some expendable income earlier this spring and decided that the time was right to pick up a copy. Timing was on my side, as the 2nd edition had just been released.

My Take

Working through this book was a fantastic experience. It reminded me of earlier in my career when I had started to learn some CAD software on my own. My experience with pytest was similar - I was able to function and had picked up the basics ok, but there was a whole world of additional functionality that I wasn’t even aware of. I could hear Brian’s voice coming through in the prose and it very much reminded me of learning from a senior-level expert. In the case of this book, Brian strikes a great balance between making valuable suggestions without being overly prescriptive.

I mostly tried to work through a chapter at a time. Initially I did this purely by reading while away from the computer. This meant that I got through the content fairly quickly, but didn’t retain a whole lot. As expected, going through the content a second time by actually working the examples in python was quite valuable. I would say I spent roughly half of the time using the example code more or less verbatim, with the remainder of the time spent applying the knowledge directly on a project.

Early on, my python experience consisted primarily of fairly simple packages and scripts run on the command line. These were fairly straightforward to test, and I was typically disciplined in following a TDD approach. Correspondingly, these tools have held up well in production.

More recently I have been the primary developer of a serverless REST API connected to DynamoDB in AWS. The system architecture here is quite a bit more complicated, and I was really struggling with managing the state of the database and building out a test suite that did not depend on a specific manual state of the data. I also was confounded by repeatability - the suite would be inconsistent between runs as side effects of certain tests affected the function of other tests.

I was already using fixtures, but this book opened my eyes to all the nuances available. Particularly useful was learning about the --setup-show flag that would provide a log of which fixtures were being called at which point. I also learned about marks, test classes, and parametrization. All of these tools have been incorporated into the testing plan for this project.

At a higher level, the testing strategy listed in Part II was very useful. I was struggling to find a nice middle ground between tiny unit tests and full on, end-to-end system tests. Brian’s guidance in the book helped me realize that I don’t need to test everything involved in the app. For example, part of the data strategy relies upon DynamoDB streams. However, testing to confirm the emittance and format of a stream record is not something this app needs to be concerned with. Therefore I was able to greatly simplify the tests and build out some hard-coded test data. This allowed for testing the stream triggers on my local machine, which provides a much shorter feedback loop during debugging and testing.

Strengths and Weaknesses

No technical book is perfect, but this sure is a strong entry.

Pros

Overall I found the book to be balanced and well-paced. The chapters build on each other somewhat, but you can skip around if desired (as I did) and still get a lot of value.

Maybe most importantly, Brian makes a solid case for testing early and often without forcing the reader into a 100% - TDD approach. Chapter 7, Strategy, is one of my favorite parts of the book as it lists solid guidance that otherwise would only be acquired via multiple years of experience.

I haven’t had a use case for it yet, but the book also covers a lot of ground regarding “modern” python development using tools like pyproject.toml and tox.

Cons

The only downside I can see is that the example Cards project is very basic and primarily runs via the command line. That’s understandable as something more complicated would likely overwhelm a rookie pythonista. More involved testing, including tests of a REST API and web interface, are left as an exercise for the reader and probably are best suite to a tool like Playwright.

Conclusion

This book has become another invaluable resource in my library, both as a guide and also for reference purposes. I’m a firm believer in unit testing and code coverage, and as a result of reading this book I’m an even bigger fan of pytest. At the end of the day, it’s clear that if you write python code, you need to use pytest. If you use pytest, you need this book!

Throwback Thursday - TiddlyWiki

2021-10-28T00:00:00-04:00

Table of Contents

A New PM Needs to Get Things Done
- Getting Things Done
Enter TiddlyWiki
- This is Great!
- This is Not So Great
Tweaking Outlook
OneNote, The One Notebook To Rule Them All
Finding A Place For TiddlyWiki
A Personal, Linear Notebook For Paper
- A Paper List of What Needs To Get Done
Lessons Learned

Here’s a retrospective look back on a path to personal productivity.

A New PM Needs to Get Things Done

In 2007 I was 5 years into my career and reaching a typical inflection point from junior project engineer to independent task responsibility and project management. My transition to project management actually happened rather quickly and I soon realized I need to scale up my work planning and organizational habits.

Getting Things Done

Around this time there was a bit of a movement brewing related to the challenges of knowledge work as email volume was continuing to swell and the general expectation that everyone could multi-task effectively, all the time. One particular personality seemed to rise above the fray in promoting a “mind like calm water” philosophy for Getting Things Done. I bought the book in late fall, then obtained a label maker at Christmas and got busy building out my personal filing system.

I also built out the recommended folder structure in Outlook and got busy developing habits around reading an email once before deciding if it was a task, something to keep for reference, or something to immediately delete. I never became particularly fanatical about Inbox Zero, but to this day I do tend to keep it below 100 messages at any given time.

As the book and associated philosophy gained steam, support tools began to spring up such as a paid add-on for Microsoft Outlook that would provide further integration between the tyranny of email and the new neural pathways being etched in my cranium. I considered a purchase but my typically thrifty personality - and interest in programming for the web - lead me in a different direction.

Enter TiddlyWiki

Billing itself as a non linear notebook for the web, TiddlyWiki took the hotness of MediaWiki and turned it into a single page app before a SPA was really a thing. This was back when Wikipedia was disrupting the universe and a wiki was the solution to any and all manner of thorny knowledge sharing challenges in the workplace.

TiddlyWiki included inline JavaScript as the controller and stored data in divs called “tiddlers”. You didn’t need a database because it’s a self-contained single html file and overriding browser settings to allow file access meant you could even run it all on A USB stick if you were so inclined.

This is Great!

My team was growing and I moved more and more into oversight, coaching, and coordination. I could review my TiddlyWiki first thing in the morning, then practice some MBWA - Management By Walking Around.

This is Not So Great

Over time, TiddlyWiki turned out to be not so great. One primary shortcoming was that it didn’t integrate with Outlook. I found that nearly all tasks that I had to deal with, regardless of whether I was able to delegate or had to personally handle it, started as an email message or meeting invite. TiddlyWiki had some basic calendaring & reminder capabilities, but it required manual effort in loading it up every day and intentionally looking for things that were overdue or had been flagged in the past for me to review.

Obviously calendars and reminders are baked right in to Outlook, and this became even more obvious when I learned from a library book that you could drag a message over to the calendar icon to turn it into a calendar reminder.

Incidentally, that library book never made it back… so it earned an ongoing residence on my bookshelf. Most importantly, changes in IT policy following an acquisition meant that end users could only run approved executables. This meant I could no longer run a light instance of XAMPP from the USB stick. The only way I was able to make it work was to use IE6… yuck.

Tweaking Outlook

So as I started to learn towards Outlook for day to day task management I briefly considered investing in a plug-in to implement GTD. It turned out that it was much easier to create the standard folders:

@Next

@WaitingFor

@Future

@Someday/Maybe

I also implemented a generic Storage folder where all saved messages would go by default. Within Storage I set up a series of search folders to automatically group messages by the categories that I assigned - typically by project. This turned out to be a key cheat code in my ability to file and recall information when needed. That and threaded-message views set on by default.

I also started using the task list in Outlook. As I mentioned above, most tasks came from email so it seemed logical to flag messages so that they automatically became tasks. However I never really took to the default view in Outlook so I started to print it out regularly. But that just meant it was perpetually out of date. Also, the red text for overdue tasks was causing additional cognitive stress, so I kept adjusting dates - which caused more stress because there was so much extra work.

OneNote, The One Notebook To Rule Them All

Then I got introduced to OneNote and learned that it was the Right Way to do this sort of thing. I gave it a shot and my first impression was that it might be the Trapper Keeper UltraKeeper 2000 that I never had in grade school. Plus I could “print” an email and store it somewhere in OneNote. That eventually became my downfall. Once I had the categories and hierarchy built out, I started bumping into problems where I couldn’t find where something should go. Or maybe it should go in two places. Maybe it was related to a specific past project but might also be useful for an upcoming project and so I wanted to file it there also.

Paralysis by analysis caused more and more mental friction, but at least I didn’t need to ask for a bigger file cabinet.

Finding A Place For TiddlyWiki

It turns out that TiddlyWiki was a useful tool, just not at $DAY_JOB. I started to look for new opportunities and embarked on a career pivot towards Virtual Design and Construction. I started thinking through business plans for Potentially going at it alone as a freelancer. At the same time I spun up a presence on LinkedIn And started cold-emailing HR staff about job postings. I was tracking all of these interactions and planning my strategy In TiddlyWiki, and it was a fantastic tool. But technology alone usually doesn’t get the job done - There needs to be a personal connection between human beings. I was getting efficient at generating cover letters And resumes, but wasn’t getting any results.

It turns out that it’s really, really hard to get an interview if you don’t already know someone on the inside.

Conversely, it can be a breeze if you have a strong network. The good news is that you don’t necessarily need to spend a ton of effort to build out that network. Treat others with respect, always give your best effort, be quick to share praise and even quicker to take blame. Co-workers will head off for other pastures and ultimately become the connection that lets you bypass the line of unknown applicants and go straight to the interview and hiring process. At least, that’s what worked for me.

Ok, back to the story…

A Personal, Linear Notebook For Paper

If Dead Tree Format (DTF) is your thing, you’ll need some sort of notebook that you like. Moleskine was popular back in the day (maybe it still is). For me it worked out well that the new $DAY_JOB had a long-standing tradition of using company-branded half-sheet-ish sized notebooks for meeting notes, informal phone logs, mind mapping, you name it. Years earlier I had discovered the best pens in the universe, the Pentel P6 line, because we used them for DTF-based markups on plan sheets. I definitely needed the red 0.5 mm for markups because otherwise I couldn’t be precise enough with strike-throughs and my poor penmanship would cause the ink to run together. Then a bridge engineer showed up with a purple 0.7 - MIND BLOWN!!! Anyway, back to the present…

It turns out that when you’re married to a teacher you automatically get an inside track to the crazy-good deals on school supplies. These are the ones that show up after the back-to-school rush is over and stores need to clear out inventory.

I’m living like a king now with backups to the backups and all sorts of fancy new colors. Seriously - she’s the best!!

Anyway, my half-size-sheet-ish notebook is the go-to for day to day note-taking. Each entry is separated by a horizontal line and gets a topic and a date. Which reminds me - ALWAYS write the date on any piece of paper that is going to stick around for more than say three months. You will use the same type of paper for years and your handwriting won’t change, so going back through things later will be much easier if it’s all dated. At least, that’s what works for me. YMMV.

A Paper List of What Needs To Get Done

So, I created a template that works for me and has a grid of spaces for a typical 5-day work week with an extra overflow to capture things in the medium-term. As tasks are finished, I happily check them off and cross them out in purple. If something slips, I just draw a big arrow to another day or re-write it on the next week’s list when I start building that out Monday morning. Not only does this work well for me with minimal cerebral damage but it also is a great antidote to the feeling you get on Friday when you wonder where the week went and what you actually worked on over the past 40-ish hours.

Lessons Learned

There’s no single “right way” to organize knowledge work. I like to think that I have taken good bits from various people that I’ve worked with along the way, as well as implemented portions of specific systems when it makes sense. I already mentioned the folder structure that I use for organizing email. Similarly I also try to touch an email just once before deciding if it needs action, should be kept for reference, or should be deleted immediately.

Likewise I think I’ve landed in a comfortable spot in terms of digital work versus manual / paper work. My workspace is mostly paperless - certainly much more so than it was a decade ago - but not entirely.

So, here’s to TiddlyWiki… you were a great tool and an interesting part of my career to date.

Why You Need Civil BIM, not CAD

2021-05-10T00:00:00-04:00

Table of Contents

CAD is a graphics display system, BIM is a feature modeling platform
The Fallacy of Data Organization By Layer
Civil CAD is better, but still has gaps
Surface modeling is not BIM
How CAD is merging with BIM
Conclusion

Much of the transportation industry is still operating in a world of 2D CAD. It’s what we are familiar with and what we are comfortable with, but it’s obsolete.

CAD is a graphics display system, BIM is a feature modeling platform

CAD originally stood for Computer-Aided Drafting because its adoption basically entailed taking the traditional drafting table and Leroy machine and replacing them with MTEXT in AutoCAD. Somewhere around the mid-oughts it became fashionable to add another D and talk about Computer-Aided Drafting and Design. Whether you prefer a single D or two, CAD is primarily about geometry, dimensions and text.

The end goal is a set of construction plans, and that’s generally the end of the road. There are no attributes or metadata to provide semantic information about what is being represented. Color, lineweight and linestyle are fixed, not driven by data or attribution.

The Fallacy of Data Organization By Layer

As an aside, I’m talking about Layer in terms of an organizational construct within AutoCAD. If you come from the Bentley side of the fence, this term is interchangeable with level.

In CAD, a layer is a display property. It is all about whether or not something should show up in a particular view. Anyone with experience in a mis-configured system with poor layer display management can understand the frustration of never knowing if the sheets are going to be correct until you plot them.

Anyway, the term layer goes back to manual drafting where different types of information (topo survey, alignments, etc.) were drafted in pencil on individual mylar sheets, then stacked together in layers before final inking took place. Again, this is an example of CAD being used as a digital construct of the manual drafting era. “Here’s how we do it on the board, so let’s do the same thing on the computer”.

Over time, layers became less about managing display and more about managing information. The primary way to differentiate between different features such as survey vs. alignments was to carve out a sacred text of layer names (or numbers, pre-V8) that were to be utilized during all drafting activities. This led to more and more layers being utilized and resulted in working environments that might have upward of 2,000 standard layers.

One example I remember was a system that had separate level names for each size of each type of storm sewer pipe. This was for a DOT that had five types of pipe and maybe a dozen standard diameters. You would never have a situation where you wanted to display all of the 12” and 15” pipes but not the 18” ones on a plan sheet, so a single layer for proposed storm sewer pipe would have more than sufficed. However, this system ended up with 60 different levels to be used, requiring great amounts of effort when an iteration of storm sewer system design would change both the geometry and the pipe diameter.

As I recall, the primary driver for this arrangement was that all CAD elements were to utilize ByLevel linetypes and the system was configured to use custom linetypes for the various pipe diameters.

Civil CAD is better, but still has gaps

Legacy versions of Geopak provided rudimentary capabilities to provide flexibility beyond the “Level Name is the feature description” mantra. This was accomplished via use of adhoc attributes. I saw this used most often for quantity takeoffs and pay item information. In the example above, this would allow for the use of a single level for all proposed pipes. Integration with D & C Manager and use of plan view criteria files could theoretically drive display properties such as the custom linestyles of pipe diameters, but it was not particularly sophisticated and required expertise in text-based programming with C-based syntax. (Somewhere on a neglected server there are mountains of curly braces enjoying retirement after banging through thousands of iterations of draw, draw skip, and mark point).

Surface modeling is not BIM

Civil designers are quick to point to surface modeling and contouring from digital terrain models as 3D modeling. Since 3D modeling is an integral (but not the only!) component of BIM, we can’t really call this BIM. Two reasons:

A surface model is actually 2.5D, not 3D, because you can’t have a surface that backs up on itself or includes perfectly vertical edges. In other words, each X,Y point in the 2D plane can intercept the triangulation once and only once.
There’s no division between components. Triangles for a curb bordered by a sidewalk on one side and pavement on the other are all part of the same blob of data. These elements can’t be displayed or symbolized individually from a DTM and they can’t be utilized for detailed quantity takeoffs. For true Civil BIM, you need closed 3D meshes with metadata.

How CAD is merging with BIM

I first started developing this post a year ago, and originally had this heading as “The Future”. But truthfully, the future is now. There’s no need to wait for new tools and upcoming software releases - the current tools are more than capable of supporting Civil BIM.

This is evident in the current civil modeling platforms where model elements are specialized data types based on feature definitions. A 3D polyline isn’t just a polyline, it’s a linear feature defined as w-beam guardrail at 6’-3” post spacing. A mesh isn’t just a mesh - it’s a pavement component with material, density, and pay item definitions.

On the Bentley side, Item Types are providing the additional capability to develop specialized model elements that inherit from the basic DGN element types (line, ellipse, point) in the spirit of object-oriented programming. Symbology can be driven by item type content via display styles, regardless of what level the features reside on. Recent upgrades to the CONNECT platform have increased the capability of Item Types - particularly with the ability to limit properties to a predefined list, or domain of values. Coming back to the storm sewer example, the Pipe Type categorization would be defined as a domain-limited field with values of 1, 2, 3, 4 and 5 .

Conclusion

It’s an exciting time to be in the transportation design and construction industry. The transition from CAD to BIM is every bit as revolutionary as the previous transition from board drafting to CAD. Proper execution of a well-reasoned BIM Execution Plan not only serves the design and construction sectors, but lays the foundation for digital twins and the metaverse of the future.

(I claim bonus points for fitting in some major buzzwords there at the end.

Gemini Principles for Digital Twins

2020-11-10T00:00:00-05:00

Table of Contents

Purpose
Trust
Function
Wrapping up

I recently took a swing at describing my version of a digital twin. At the time, I wasn’t aware of much prior art other than the typical hype cycle. Recently, I was pleasantly surprised to come across a framework for digital twins developed by Centre for Digital Built Britain. More specifically, the Digital Framework Task Group has published a report outlining a common set of definitions and principles related to the development and use of digital twins. Termed the Gemini Principles, this report outlines nine key principles within three groups of purpose, trust, and function.

Purpose

Leading off, we see that a digital twin must have a clear purpose. The three principles that define the purpose include Public good, Value creation, and Insight.

Public Good

From the text:

“Must be used to deliver genuine public benefit in perpetuity”.

My take:

This is a great start. If a digital twin isn’t going to benefit its consumers, then why expend the effort to create and maintain it?

Value Creation

From the text:

“Must enable value creation and performance improvement”.

My take:

This principle drives at a key question when discussing digital twins - “What do I get?”. And the answer here isn’t one-size-fits-all. Digital twins need to be flexible and tailored to specific use cases and needs - often most clearly identified as “pain points” within the current state of an entity or process. The second part of this principle points to a common use case where an effective digital twin will improve performance as measured by key indicators such as percentage of a transit system’s on-time service or pavement condition over time relative to what was predicted by the mix design.

Insight

From the text:

“Must provide determinable insight into the built environment”.

My take:

This has been my go-to term when making the case for infrastructure digital twins. A primary challenge in maintaining the built environment is that much of the critical components are hidden out of sight. In a building, drywall conceals the framing and electrical wiring. On a bridge, the rebar is invisible underneath its concrete encasement. Stormwater drainage systems operate by gravity and therefore have to be underground to be of much use. Capturing all of that information in a digital twin provides “X-ray vision” of these hidden components via 3D viewers and Virtual Reality or Mixed Reality environments. In the bridge example, an inspector can see a surface crack in a column and correlate its physical location to the as-designed or (preferably) as-built reinforcement model within the digital twin. The criticality of the observed cracking as it relates to overall structural integrity can then be assessed in real-time without having to return to the office to track down record plans on microfiche.

Trust

Next, we see that a digital twin must be trustworthy. Trustworthiness is demonstrated by Security, Openness, and Quality.

Security

From the text:

“Must enable security and be secure itself”.

My take:

As digital citizens of the 21st century, we can all understand the need for security, even when it comes at the cost of convenience. It can be frustrating to have to reach for a hardware key when two-factor authentication is required, but the internet never forgets and you can’t put the toothpaste back in the tube after a data breach.

Openness

From the text:

“Must be as open as possible”.

My take:

Data wants to be free, and no one should be excluded from the benefits of a digital twin just because they haven’t bought in to proprietary systems or aren’t running a mainstream operating system. The Information Age is founded upon open standards such as TCP/IP and HTML and a digital twin should be no different.

Quality

From the text:

“Must be built on data of an appropriate quality”.

My take:

This immediately makes me think of Level of Development (LOD) when used in the context of Building Information Modeling (BIM). If you are going to fabricate replacement parts based on the geometry stored in the digital twin, then that data had well better meet LOD 350 or higher. Conversely it is possible to have too much of a good thing. Geometric detail requires more bytes per unit of physical asset and these storage needs may not scale well across a huge system such as a statewide transportation inventory.

Function

Rounding out the Gemini Principles, we see that a digital twin must function effectively by providing Federation, Curation, and Evolution.

Federation

From the text:

“Must be based on a standard connected environment”.

My take:

Again, my mind goes to the division of information within a BIM environment. Each design discipline is responsible for modeling their portion of the work, be it structural, electrical, civil, etc. These models are then connected and combined into a common data environment where clash detection can be performed and the proposed work can be evaluated as a whole. This principle also applies to the Operations and Maintenance phase. For example, a department of transportation may have responsibility divided between organizational units focused on bridges, pavements, environmental resources, ancillary structures, and other subsystems. Too often data is siloed and constrained by blockage between disparate systems. An effective digital twin eliminates blockage and allows for frictionless information sharing across all participants.

Curation

From the text:

“Must have clear ownership, governance and regulation”.

My take:

Absolutely! In my career I’ve experienced this need most often when sharing CAD data. Downstream users tend to distrust any information provided by a third party and governance is typically managed informally via handshake agreements. I’ve also experienced curation done well when consuming aerial imagery and digital elevation models that contain metadata identifying the party that generated the information and outlining its permissable use. More recently I’ve noted that open data portals tend to provide clear curation information that is standardized and accessed just as easily as the data itself.

Evolution

From the text:

“Must be able to adapt as technology and society evolve”.

My take:

The text mentions that the Gemini Principles are “descriptive of intent but agnostic on solutions”. Nowhere is this seen more clearly than in this final principle. A digital twin doesn’t have to chase fads or constantly be reinvented in the latest javascript framework flavor of the month. It does have to evolve to meet additional maturation of technology. A great example is the (relatively) recent shift from traditional desktop machines to mobile devices. Responsive layouts have emerged to meet the users where they are, and there are certain to be new challenges and innovations in the future that will shape the format and delivery of digital twins. My guess is that the next big shift will involve widespread adoption of augmented reality devices and the patterns that form for effective and efficient user experiences with this type of hardware.

Wrapping up

So there it is - a nice symmetry of 3 groups of 3 principles each. Short enough to be easily digested while open-ended enough to guide discussions and drill down to the value that a digital twin will provide - whatever the use case or user base. I’m grateful for the work done by the Centre for Digital Built Britain and all contributing individuals that had a role in developing this guidance. It has certainly helped me personally in providing a framework of understanding around the nebulous concept of digital twins and I’m equally certain that it will continue to be of good use in the future when seeking to develop digital twins as a means of value creation as opposed to bandwagon jumping.

Miles per burrito

2020-04-06T00:00:00-04:00

With the COVID-19 pandemic currently affecting so many of us in the US and other parts of the world, it felt like a good time to write about something more light-hearted. In that spirit, today we are going to be examining bicycle commuting - specifically, typical caloric expenditures per unit distance.

Background¶

In the not-too-distant-past, I was a regular (typically once a week) bicycle commuter. I usually rode during three of the four seasons, starting first on a used Trek hybrid and ultimately ending up with a drop bar touring bicycle.

At some point, I ran across this graphic of a bicycle claiming to provide transportation efficiency at “infinity miles per gallon”. I posted it on my cube wall and was eventually challenged to substantiate this claim. At that time, I worked out a mini-dissertation on paper. Sadly, that calculation has been lost - likely to a recycling bin in a New Year’s cleaning purge. Since the Internet never forgets, I figured that this time around I would work back through the calcs in this Jupyter Notebook.

Research¶

In order to determine a reasonable approximation of miles per gallon for cycling, we must first make accomodations for the difference in fueling mechanisms of the internal combustion engine versus a human cycle operator. The bicycle is propelled by the human, who is fueled by food. In cases such as this where a transportation mode is powered by alternative fuels, the efficiency of the alternative energy source is typically converted to a miles per gallon equivalence, or MPGe. As I recall, I did that same calculation previously. But what stands out from the past, and what I wish to cover first, is to express the efficiency of bicycle commuting in terms of miles per unit of food. Therefore, we need to select a reasonable, common unit of food with a known caloric content. Additionally, we need to determine a reasonable rate of caloric expenditure per unit of time for cycling.

Settling on a Unit of Measurement and Running the Calcs¶

Simply replacing a gallon of gasoline for a gallon of something like potato salad made sense initially. But that didn’t stand up as a suitable unit of food. How often do you run out (or order delivery) for a gallon of food at one time? No, it makes much more sense to use a more sensible portion size. While we’re at it, let’s pick something that reasonably could be consumed while cycling, such as… a burrito! Therefore, we’ll be calculating miles per burrito (MPB) when commuting by bicycle.

I’m most familiar with Qdoba as a common source of burritos, so let’s use their current nutrition guide. And, before we start slinging code, it would be a good idea to note that I will be using the pint library within python for all of the unit conversions. Therefore, let’s go ahead and sum up the ingredients for a grilled adobo chicken burrito.

In [1]:

import pint

ureg = pint.UnitRegistry()

def burrito_calories():
    """calculate the total caloric value of a typical burrito configuration"""
    tortilla = 210
    chicken = 150
    black_beans = 140
    brown_rice = 170
    lettuce = 0
    pico_de_gallo = 10
    salsa_verde = 15
    three_cheese_queso = 90
    
    all_ingredients =  tortilla + \
        chicken + \
        black_beans + \
        brown_rice + \
        lettuce + \
        pico_de_gallo + \
        salsa_verde + \
        three_cheese_queso
    
    return all_ingredients * ureg.kcalorie
    
bc = burrito_calories() 
print(f"{bc} per burrito")

785 kilocalorie per burrito

Good thing I asked that they go light on the queso. We’re almost to 800 calories without adding any chips, guac, or a drink!

Caloric Expenditure¶

The next step is to determine a typical caloric expenditure during bicycle commuting. Pretty much every fitness tracker out there will give you an estimate of calories burned for a given activity. I wanted something a little more scientific, so I went to the bookshelf to consult a copy of Bicycling Science, Third Edition. There, in Table 2.1 on page 76, we find a metabolic heat value for a touring cyclist covering 12 miles per hour equal to 6 kCal per minute. Combining this value with our total for the burrito gives:

In [2]:

metabolic_heat =  ( 12 / (6 * 60) )* (ureg.miles / ureg.kcalorie)

print(f"Energy flow rate: {metabolic_heat:.4f}")

efficiency = metabolic_heat * bc

print(f"Calculated efficiency: {efficiency:.4f} per burrito (MPB).")

Energy flow rate: 0.0333 mile / kilocalorie
Calculated efficiency: 26.1667 mile per burrito (MPB).

There you have it! 26 MPB, give or take.

MPG Equivalence¶

Let’s see how this stacks up against typical automobiles. A skim through the literature on Sonoco Racing Fuels gives us a value of 111,000 Btu per gallon of 118 octane leaded fuel. Let’s assume your typical 87 octance isn’t quite as potent and round down to 100,000 Btu.

This will give us MPGe for cycling:

In [3]:

gallon_of_gas_energy = (100000 * ureg.Btu).to('kcalorie')

mpge = metabolic_heat * gallon_of_gas_energy

print(f"Commuting by bike is equivalent to {mpge:.4f} per gallon.")

Commuting by bike is equivalent to 840.5481 mile per gallon.

Not too shabby!

Infrastructure As Code

2020-02-19T20:05:00-05:00

Table of Contents

DevOps trends
An interesting parallel to the physical world
Conclusion

Last week I reached another milestone of sorts on my journey as a developer. I took a bit of a deep dive into the AWS Cloud Development Kit (CDK). CDK is a development framework that allows a developer to stand up cloud infrastructure “stacks” by writing code in any one of a handful of popular programming languages. Not surprisingly, my personal favorite - python - gets a seat at the table.

DevOps trends

One general trend in software development is towards the “single-threaded” (AWS’s term, I can’t take credit for that) software development team. This refers to the notion of having developers run the majority of the unit tests and manage hardware deployments, in addition to writing the actual code. This development pattern typically includes a CI/CD pipeline that allows for new code versions to be automatically released to production, assuming that it passes the automated test suite. Tools like Ansible, Chef, Saltstack, Puppet, and perhaps the grand-daddy of them all - Travis CI - all serve generally this purpose as part of an integrated stack.

This approach to SaaS delivery is great for all the same reasons that scripted instructions on the command line will always run rings around picks and clicks on a screen with a mouse. Better yet, a scripted approach is traceable, auditable, and repeatable. DevOps automation is great for all the same reasons that git is the best tool for source control and Docker has revolutionized OS management:

you can copy and reuse the automation code on the next project
commands are scripted in plain text, which is lightweight and opaque
the code is self-documenting because every step of the process is written out, in order
workflows can be version controlled with branching and “time travel”

An interesting parallel to the physical world

After getting fully stuck in to the CDK tutorials and workshops, my brain was sufficiently stretched and ready for some restorative rest and relaxation. After logging off, I realized there was an even bigger parallel to the AEC industry. When we think of DevOps and deployment automation, we talk of infrastructure in terms of networking, servers, GPUs, and object storage. However, well before computing infrastructure became ‘a thing’, we had the concept of infrastructure in the form of physical assets like roads, bridges, and tunnels. The light bulb switched on when I realized that there is a similar movement afoot to also think of these physical features in terms of infrastructure as code.

For example, as digital twins become more and more commonplace, they will require code to turn sensor data into actionable information and dashboard visualizations. Also, we are seeing computational geometry as a tool for quickly evaluating multiple design options and rapidly iterating through “what if” scenarios and optioneering. These tools typically take form as visual programming, but just behind the scenes is a full-on programming language such as IronPython lurking just beneath the surface of Dynamo. And nothing to say of the 800 lb gorilla that is AI and Machine Learning. These heavy-duty algorithms and models run upon multiple layers of complex code and are poised to disrupt AEC just as much, if not more, than other market sectors.

Conclusion

It’s been said that there is crazy huge potential for tech disruption in AEC because we’ve been much slower at transitioning to digitalization than other sectors have. Even though we might be last to the table, we can use concepts from other industries such as the practice of DevOps for “just-in-time” software delivery to realize quick gains in safety, quality, and efficiency with relatively low effort. That’s why I’m happy spending my time on a posix shell picking through a forest of seemingly random characters and indentations. It’s true that this is a window to the past worlds of mainframes and time sharing. But it’s also true that this is a preview of the future.

In Praise of the WYCIWYG Editor

2020-02-05T00:00:00-05:00

Table of Contents

Starting out on WP 5.1 for DOS
Office 97 lands
Time for a change
Back to HTML
Enter Python
Learning by doing
A few standard deviations outside of mainstream?
Conclusion

Starting out on WP 5.1 for DOS

After the typewriter, WordPerfect 5.1 became my tool of choice for school papers and other assorted random documents. Our first printer at home was letter perfect (I think that was the term) because there was a wheel with protrusions for numbers, letters, and basic characters.

When you sent a doc to the printer, the wheel would spin and strike against a ribbon putting just the right amount of ink where needed. We were a few years from meeting Dan Matrix and his wife Dot, to say nothing of color laser output.

Office 97 lands

That changed in the fall of ‘97. I went from (happily) using WP 5.1 in the computer lab to marveling at the slick interface delivered by Microsoft Office 97 running on my own, brand new Optiplex.

There was even this awesome animated paper clip that would pop up and help you with baffling tasks like writing a letter! I typed a few lines and starting clicking around. Holy Schnikes! I could make the words bigger, smaller, change the font, add italics, color, crazy symbols…

This black magic was WYSIWYG (wizzy-wig) editing: What You See Is What You Get.

The promise was that when you were ready to render to dead tree format, the output would exactly match what you already saw on the screen. Of course, that presumed that you hadn’t done anything crazy like adjusting the margins or sticking page numbers in the footer.

And so it went as I navigated academia and into industry. As it worked out, I spent much more time in drafting and design software than I did in Word.

There were a few times along the way where I tried to develop technical documentation that was to be organized into sections and chapters, presumably across multiple linked *.doc files. All of those efforts fizzled out before I got too far into progressing beyond the basics in Word.

Time for a change

That changed a few years ago. We went through a major migration in our CADD platform and needed to memorialize the new configuration in order to ease the transition. I knew what I wanted: consistent output with multiple styles based on what the docs were describing - monospaced Courier for text configurations, blue italics for UI components to click on, red bold capitals for DO NOT EVER DO THIS!!!!!.

Boy, was that painful in Word. It seemed impossible to get image placement consistently correct.

I had terrible amounts of brain damage trying to get consistent styling and proper numbering across the header hierarchy. As the content grew, it got split out into multiple *.docx files, which meant there was no easy way to cross-reference content across sections and chapters.

The only real option for publishing was PDF, which meant a lot of user clicking to navigate. I tried to make everything better by building a web portal of sorts that linked to the individual chapters and included a primitive change log.

Back to HTML

It had been a long time since I had written any html - almost 15 years. I started the same way as before: hand coding html in a text editor. I had upgraded to Notepad++ by this point, but that was about it.

After doing some research, it seemed like jquery and bootstrap were taking over the world. I eventually strung something together and it worked relatively well, with a lot of copy/pasting and crazy nested <div> elements. Hmmm, I thought… What if there was some sort of templating engine that could streamline and automate all of that HTML generation?

Enter Python

Roughly at this same time, I was starting to learn python. After reading that vim and python were a match made in heaven, I started learning about hjkl and :wq. Learning vim was like dropping TI in favor of an HP calculator. Everything felt crazy-weird for about two weeks, then things got awesome.

As I explored all of the batteries included with python, I started noticing a beautiful consistency with the documentation. It had the structure and formatting of doxygen but didn’t look like it belonged on Netscape Navigator 2.0. What I was seeing was html generated by Sphinx from content written in rST. Could this replace my half-bootstrap, half-Word Frankenstein?

Learning by doing

I wasn’t writing much python at this point, but I had just wrapped up stringing together a macro in object-oriented VBA. I couldn’t get the ship turned towards python fast enough, and here was the perfect opportunity to see if this new toolchain might fit my needs for technical documentation.

If you’ve made it this far into the story, you can probably guess that the answer is a resounding YES.

I could use all the standard directives to basically line up with my requirements and get rock-solid consistent styling across the board. Building up a table of contents was as easy as writing a table of contents in plain text. Cross-referencing was a breeze, and default output to HTML meant that navigation was snappy and efficient.

Best of all, I could use that weird thing called git to track all of my changes and even travel back in time if necessary. Soon thereafter, I found I could get PDF output from the exact same content.

I probably didn’t realize it at the time, but this new tooling provided great separation of concerns - data here, logic there, presentation however and wherever you want it.

Therefore, I am now claiming a new term for this approach to content generation:

WYCIWYG (wacky-wig) - What You Code Is What You Get

A few standard deviations outside of mainstream?

I get it. Word and Google docs aren’t going to disappear tomorrow in favor of terminal emulators and model editing. However, before dismissing WYCIWYG, consider this:

Lawyers still use Word Perfect due to formatting strictness
Microsoft project managers are writing docs in markdown and using git for change management
You can read to / from multiple formats with tools like pandoc
I recently captured two full days of meeting notes in a single markdown file using VS Code with vim keybindings. It was splendid.
Have you ever heard “Wow, those styles were so easy to use and track changes stayed nice and tidy even though we had 45 people editing the Word doc simultaneously!”

I didn’t think so.

Conclusion

It’s never been a better time to be on the command line. While I know I won’t ever totally escape WYSIWYG, it’s liberating to have - and utilize - its far superior alternative, WYCIWYG.

What is a digital twin?

2020-01-30T00:00:00-05:00

Table of Contents

A Worked Example of a Digital Twin
Is that it?
- Internet of Things and the Edge
How do I get a digital twin?

There currently is a lot of talk running around the industry about this concept of a Digital Twin. Bentley Systems in particular have utilized this term quite a bit in the recent past in their marketing materials and visioning processes. While this is a catchy term, it’s not a particularly concrete concept. After a recent internal discussion, I felt it would be great to try to lasso our collective thoughts and opinions.

Conceptually, a Digital Twin is a virtual model of a physical asset that captures its geometry, geospatial position, and additional attributes of interest. It’s called a twin because (theoretically) any changes in the state of the physical asset are automatically (quickly?) propagated to the virtual representation.

A Worked Example of a Digital Twin

Imagine a state department of transportation responsible for tens of thousands of highway lane miles, including bridges, lighting, signs, and multiple other types of physical assets. An example of a digital twin for this type of organization might be a highway impact attenuator installed at the beginning of a concrete barrier run. The digital twin would capture its physical dimensions, geospatial position, linear reference (e.g. route and milepost), manufacturer name, model name, installation date, warranty information, and so on. Let’s say that this attenuator is a multi-cell crash unit with replaceable parts.

One day, an errant vehicle runs in to this attenuator. The driver is ok, because the attenuator has done its job and prevented impact with the much more rigid concrete barrier just a few feet downstream. We’ll say this is an attenuator with parallel rails and seven energy-absorbing cells. The first three sections are damaged and need to be replaced, but the remaining four are still in fine shape. First responders file a report, clean up, and report back to the DOT that this attenuator has been hit and needs some maintenance. This incident is recorded as a change in state of the digital twin and triggers a notice to maintenance staff that this unit requires servicing. The maintenance staff locates some spare parts, schedules a crew, and the repair is done. The repair work is captured as an additional change in state of the digital twin and the asset can then be reported again as being in a state of good repair.

Is that it?

If you think that sounds sorta underwhelming, you’re probably right. The scenario I’ve just described is along the lines of an incremental first step towards an integrated information system. In other words, this description by itself doesn’t really match all of the hype and excitement around digital twins. If we take this example a few steps forward, we can understand digital twins in terms of the current trends in the overall technology landscape.

Internet of Things and the Edge

Kids of the 80s, especially musicians, might think first of an iconic guitar player from Ireland. In the context of technology, “edge” computing is a general term for processing that takes place on the edge of the cloud. In other words, the hardware is closer to the user as opposed to being “far away” in the middle of the data center cloud.

The proliferation of IOT sensors and devices are a large driver of a general shift in focus to the edge as the next growth area in information technology. The concept of a digital twin is piggybacking on this trend. Imagine a bridge with temperature, stress, and other sensors that could report an overweight non-permitted load the second that it happens. The vision of a physical asset that is fully instrumented and able to maintain a digital representation in real time then becomes an incredibly rich information source. Rather than only having inspectors visit a bridge every two years, the bridge can report condition in real time and give advance notice of deterioration before it reaches a critical, unsafe condition.

How do I get a digital twin?

Lots of asset information is decidedly non-digital. One way to begin the transformation to digital twins is with a robust information management strategy including data collection, change management, data storage, system integration, and a host of other thorny issues. Reality Capture will no doubt be a large part of the initial generation of digital twins. The question will be how to scale up current processes to address a statewide collection effort with the attendant concerns over how long that will take and how much it will cost. Perhaps there is an opportunity to leverage big data and machine learning patterns that have been utilized to great effect in other industries such as finance. Whatever the path forward ultimately looks like, there’s no better time to get started than right now.

Is IFC the format of the future?

2020-01-02T00:00:00-05:00

Table of Contents

Background
Obligatory XKCD Reference
IFC is a Good Bet
The Argument Against
IFC is a Schema, Not a File Format
Conclusion

A while ago, I asked my team for blog post ideas and my supervisor floated this topic as having potential for sparking some interesting discussion. After giving it some thought, it’s clear to me that if we’re going to truly transform the industry, IFC has to be the format of the future.

Background

There is currently a pooled fund study underway in the US regarding BIM for Bridges. This concept, often referred to as BrIM for Bridge Information Model, is focused on data modeling standards for electronic information sharing related to design, construction, operations, and maintenance of bridges. This working group recommends Industry Foundation Classes (IFC) as the recommended format for this proposed standardized data exchange. The question at hand is in regards to whether this current standardization effort will truly revolutionize the industry.

Obligatory XKCD Reference

Industry veterans have seen similar discussions play out in the past with data formats such as DXF for graphical CAD data and a hodge-podge of standard formats for ASCII text files utilized for sharing survey topography data.

Another example I remember from around the turn of the century is the discussion of DWF vs PDF. DWF is a proprietary format developed by Autodesk for viewing, printing, and annotating 2D and 3D content. While in many ways potentially technically superior than PDF (particularly, at the time, in file size), DWF failed to topple the momentum that PDF had already gained as the de-facto digital equivalent of the printed roll of plans. In fact, by the early 2010’s, electronic-only was the default and my colleagues and I began utilizing DTF (dead tree format) to specify that a printed, physical copy was required.

Given the multi-year effort required to research, implement, train, and require a particular file format, it’s understandable that infrastructure asset owners may be hesitant to jump on the bandwagon of IFC as ‘yet another’ file format.

IFC is a Good Bet

The BIM For Bridges effort here in the states mimics standards being adopted worldwide. BIM requirements are already a present-day fact of life for AEC practitioners in markets including the UK, Europe, Middle East, and Asia. In almost every case, IFC is preferred as the common data standard. IFC is an ideal format for AEC data because:

it is an open format published as an ISO standard (ISO 16739-1:2018)
it is mature, having published version 1.0 in 1996
it is developed in the open, by consensus, and overseen by a non-profit organization
there is a thriving ecosystem of open-source tools and programs that can read and write the format
it is wide-ranging, including data models for:
- personal contact information
- company organizational structure
- quantity takeoffs and material details
- critical path scheduling of tasks

The Argument Against

One of the most common rebuttals of an open, neutral format is that each software vendor will end up implementing the standard in slightly different ways, which only serves to further fragment the industry. Also, proponents of proprietary binary formats often make claims of specific “magic” features that their software tool of choice provides and allegedly can’t be replicated in a different format. However, data lock-in to closed, proprietary formats is not in anyone’s best interests, excepting of course the software vendors.

IFC is a Schema, Not a File Format

Up to this point, we’ve been thinking primarily in terms of a single container manifesting itself as a *.ifc file stored on e.g. a hard drive. However, this is short-sighted, because IFC is a schema, not a file format. As we move to data streaming continuously from physical IoT sensors back to a Digital Twin, the BrIM data will be dynamic. For this reason, among others, BrIM will take on a service-oriented architecture with versioning and time stamping. Data will most likely live in a cloud environment in a data lake or similar distributed storage system, with multiple clients connecting simultaneously at any given time. Indeed, there currently are multiple common data formats for IFC data, including XML and JSON in addition to the typical STEP file. This points to a great staying power for IFC. And indeed, this object-oriented, schema-based focus is reflected in the name - Industry Foundation Classes.

Conclusion

It seems like there is more and more momentum around IFC with each passing month. As we kick off a new year and a new decade, I’m definitely hitching my wagon to IFC - particularly for bridges and other linear civil works that have been slow to make the leap to an Information Model focus.

Modeling the St. Louis Arch - Part 2

2019-11-23T00:00:00-05:00

Continuation¶

When we left off last time, we had gotten to a 2D representation of the centroid of the arch cross-sections. However, the centroid is used as “construction” geometry in this model - it doesn’t represent something physical. Rather, it is used for constructing, or laying out, portions of the model that will actually be built. Yes, this is a little confusing because we are talking about construction in terms of concrete and steel, but construction geometry as a term used in this context is common in 2D CADD and other modeling packages.

To get started, we’ll import the same packages, use the same constants, and utilize the same functions previously developed.

Note: Typically at this point in a data science project, this would be a great time to start factoring notebook cells out into scripts and modules. However, I’m going to make the call that I can live with some technical debt for the time being.

In [18]:

import math

import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = [11, 8.5]
from mpl_toolkits.mplot3d import Axes3D
import numpy as np
import mpmath
import pandas as pd

fc = 625.0925 # Maximium height of centroid
Qb = 1262.6651 # Maximum cross-sectional area at arch base
Qt = 125.1406 # Minimum cross-sectional area at arch top
L = 299.2239 # Half of centroid at arch base
A = fc / ((Qb / Qt) - 1)
C = math.acosh(Qb / Qt)

def El(Y):
    return fc - Y

def X(Y):
    return (L / C) * (math.acosh(1 + Y/A))

Intrados and Extrados¶

The NPS web site has served us well thus far. However, we could certainly use some more detail on a methodology for calculating the inner and outer edges of the arch to fully develop a 2D elevation view. The detailed methodology for developing the arch geometry was laid out in the blueprints.

A Word About Symmetry¶

The equations developed for cross-sectional charactistics of the arch were developed for the right-hand side of the arch. Speaking from experience, bad things will happen if you try to apply the same relationships to the left (x < 0) side of the arch. Therefore, we’ll just calculate the right leg, then mirror about the X axis to get the other half of our model.

With that important caveat in mind, let’s build pythonic versions of these equations:

In [19]:

def Y(X):
    ret =  A * (math.cosh((C * X) / L) - 1)
    if ret == 0:
        ret += 1e-16
    return ret
 
def slope(Y):
    return (L / C) * (1 / math.sqrt((2 * A * Y + Y**2)))

def alpha(slope):
    return math.atan(slope)

def Q(Y):
    """cross-sectional area at any Y"""
    return ((Qb - Qt) / fc) * Y + Qt

def H(Q):
    """height of the equilateral triangular section"""
    return math.sqrt(Q * mpmath.cot(mpmath.radians(30))) 

def W2(H):
    """side of the equilateral triangular section"""
    return 2 * H * math.tan(math.radians(30))

# test some known values
print(f"Value of Y at X=0 is {Y(0):.4f}")
print(f"Elevation at X=0 is {El(Y(0)):.4f}")
print(f"Maximum cross-sectional area is {Q(0)}, or Qb = {Qt}")
print(f"Maximum cross-sectional area is {Q(Y(L))}, or Qt = {Qb}")
print(f"Height of triangle at the top is {H(Qt):.4f}")
print(f"Length of triangle side at the top is {W2(H(Qt)):.4f}")
print(f"Height of triangle at the bottom is {H(Qb):.4f}")
print(f"Length of triangle side at the bottom is {W2(H(Qb)):.4f}")
print(f"Slope angle (alpha) near top of arch is {math.degrees(alpha(slope(0.000001))):.4f}")

Value of Y at X=0 is 0.0000
Elevation at X=0 is 625.0925
Maximum cross-sectional area is 125.1406, or Qb = 125.1406
Maximum cross-sectional area is 1262.6651, or Qt = 1262.6651
Height of triangle at the top is 14.7224
Length of triangle side at the top is 17.0000
Height of triangle at the bottom is 46.7654
Length of triangle side at the bottom is 54.0000
Slope angle (alpha) near top of arch is 89.9933

Great! Everything is checking out so far.

Side note: ideally, we would be using a test suite such as pytest in lieu of print-mode debugging. However, in a notebook such as this the focus is more on telling a story than creating a piece of production software.

Data Structure + Function = Class¶

Initially, my next step was to start defining individual functions to calculate points on the interior and exterior of the arch at any given location along the centroid. But, that would have duplicated a lot of the calculation logic. So, let’s think about what we actually want to do: given any x coordinate on the centroid, calculate the y coordinate of the centroid, along with the coordinates of the intrados and extrados. This is the perfect time to write our first class. This will allow us to encapsulate the calculations and results into a single, tidy spot.

While we’re at it, let’s go ahead and make the leap to three dimensions. One wrinkle here is that our coordinate system defined thus far is consistent with a “Y-up” convention that is common in graphics. This can be confusing for those of us that are used to “Z-up” in the CADD world. For the arch, the centroid and intrados will be on the Z-plane (Z=0). So that means we only need to worry about calculating Z coordinates for the extrados. Since the arch doesn’t have any torsion (twist) by design, we actually only need to calculate a single Z offset, then assign the positive value to one extrados point and the negated value to the other extrados point. This Z offset will be one-half of W, the length of each side of the triangular cross-section.

In [20]:

class NormalSection():
    """calculates the triangular cross-section of the arch at any provided
    value of x (offset from axis of symmetry)"""
    def __init__(self, x):
        max_x = 299.5458
        if x < 0 or x > max_x:
            raise ValueError(f"Invalid value for x (must be between 0 and {max_x})")
            
        if x == 0:
            x += 1e-6
            
        y = Y(x)
        a = math.atan(slope(y))
        h = H(Q(y))
        h1 = 2 * h / 3
        h2 = h1 / 2
        z = 0.5 * W2(h)
        
        # intrados
        x3 = x - h1 * math.cos(a)
        y3 = y + h1 * math.sin(a)
        
        # extrados
        x1 = x + h2 * math.cos(a)
        y1 = y - h2 * math.sin(a)
        
        # counterclockwise winding
        self._pt1 = [x3, El(y3), 0]
        self._pt2 = [x1, El(y1), -z]
        self._pt3 = [x1, El(y1), z]
        
    @property
    def vertices(self):
        return np.array([self._pt1, self._pt2, self._pt3], dtype='float64')

An easy way to test this class is to confirm that the extrados point at the top of the arch is equal to the total height of the arch (630 feet). The distance between legs of the arch at the base should be 630 feet as well.

In [21]:

print(f"Maximum height of arch is {NormalSection(0).vertices[1][1]:.4f}.")
print(f"Total width of arch at base is {(2 * NormalSection(299.5458).vertices[1][0]):.4f}.")

Maximum height of arch is 630.0000.
Total width of arch at base is 630.0003.

Blueprint Stations¶

Now that we have a class defined to provide triangle vertices at any value of x on the centroid construction geometry, we need to know what values of x should actually be evaluated. The arch blueprints included a list of 72 stations with these values.

We can use pandas to parse this file so that we are creating our virtual model of the arch geometry exactly as intended per the blueprints.

Note: I renamed my local copy to have a *.txt extension so that it didn’t wreak havoc with the static site generator tooling.

The read_html method returns a list of dataframes corresponding to all of the tables in an html document. From inspection of the input data, we know that we only want to keep the first one.

In [22]:

df = pd.read_html('../static/centroidxdat.txt', header=4, index_col=0)[0]
df.drop(df.columns[1:], axis="columns", inplace=True)
df.rename({df.columns[0]: "centroid_x"}, axis=1, inplace=True)

stations = df.loc[:, "centroid_x"]
y = np.array([El(Y(_)) for _ in stations])
sections = [NormalSection(_) for _ in stations]

x3 = np.array([_[0] for _ in [p.vertices[0] for p in sections]])
y3 = np.array([_[1] for _ in [p.vertices[0] for p in sections]])

x1w = np.array([_[0] for _ in [p.vertices[1] for p in sections]])
y1w = np.array([_[1] for _ in [p.vertices[1] for p in sections]])

x1e = np.array([_[0] for _ in [p.vertices[2] for p in sections]])
y1e = np.array([_[1] for _ in [p.vertices[2] for p in sections]])

triangles2d = np.float64([[[s.vertices[0, 0], s.vertices[0, 1]], [s.vertices[1, 0], s.vertices[1, 1]]] for s in sections])
x2d = triangles2d[:, :, 0]
y2d = triangles2d[:, :, 1]

axes = plt.gca()
axes.set_aspect('equal')
axes.set_ylim(0,650)
plt.plot(stations, y, label='centroid', color='green')
plt.plot(-stations, y, label='centroid - north', color='green')
plt.plot(x3, y3, label='intrados - south', color='red')
plt.plot(-x3, y3, label='intrados - north', color='red')
plt.plot(x1w, y1w, label='extrados - south', color='blue')
plt.plot(-x1w, y1w, label='extrados - north', color='blue')
for xs, ys in zip(x2d, y2d):
    plt.plot(xs, ys, color='gray')
    plt.plot(-xs, ys, color='gray')
plt.title("Fig. 1 - Elevation View of Extrados, Centroid, Intrados, and Sections")
plt.show()

Confirm Results¶

Ok, so that looks good. However, I said earlier that we were going to go ahead and make the jump to 3D. Our NormalSection class calculates 3D points for all of the triangular cross sections. So, with a few more lines of code, we should be able to make a similar plot, but with the inclusion of the Z-axis, which is the direction coming in and out of the screen in Figure 1.

In [23]:

z3 = np.array([_[2] for _ in [p.vertices[0] for p in sections]])
z1w = np.array([_[2] for _ in [p.vertices[1] for p in sections]])
z1e = np.array([_[2] for _ in [p.vertices[2] for p in sections]])

# add the first point again at the end to make a closed shape
triangles3d = np.float64([[[s.vertices[0, 0], s.vertices[0, 1], s.vertices[0, 2]],
                           [s.vertices[1, 0], s.vertices[1, 1], s.vertices[1, 2]],
                           [s.vertices[2, 0], s.vertices[2, 1], s.vertices[2, 2]],
                           [s.vertices[0, 0], s.vertices[0, 1], s.vertices[0, 2]]] for s in sections])

x3d = triangles3d[:, :, 0]
y3d = triangles3d[:, :, 1]
z3d = triangles3d[:, :, 2]

fig= plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.plot(xs=stations, ys=np.zeros_like(stations), zs=y, color='orange')
ax.plot(xs=-stations, ys=np.zeros_like(-stations), zs=y, color='orange')
ax.plot(xs=x3, ys=z3, zs=y3, color='red')
ax.plot(xs=-x3, ys=z3, zs=y3, color='red')
ax.plot(xs=x1w, ys=z1w, zs=y1w, color='blue')
ax.plot(xs=-x1w, ys=z1w, zs=y1w, color='blue')
ax.plot(xs=x1e, ys=z1e, zs=y1e, color='green')
ax.plot(xs=-x1e, ys=z1e, zs=y1e, color='green')
for x3d, y3d, z3d in zip(x3d, y3d, z3d):
    plt.plot(xs=x3d, ys=z3d, zs=y3d, color='gray')
    plt.plot(xs=-x3d, ys=z3d, zs=y3d, color='gray')
ax.set_xlim(-350, 350)
ax.set_ylim(-350, 350)
ax.set_zlim(0, 700)
plt.title("Fig. 2 - 3D View of Extrados, Centroid, Intrados, and Sections")

Out[23]:

Text(0.5, 0.92, 'Fig. 2 - 3D View of Extrados, Centroid, Intrados, and Sections')

Serialize output¶

Now that we’ve calculated all of the geometry, let’s serialize it (write coordinates to a text file) so that we can potentially re-generate this model in another platform such as Blender.

We’re going to reverse the order of the triangles (arch cross-sections) so that it starts at the bottom and progresses up to the top. That way we can build our model in the same order as the way that actual construction was carried out.

In [24]:

import json

with open("triangles.json", "w") as f:
    json.dump(triangles3d.tolist()[::-1], f, indent=2)

Conclusion¶

Given the density of the sections, it’s hard to pick out the centroid in the 3d plot. This makes sense, given that the centroid is merely ‘construction’ geometry and does not represent anything physically present in the real world.

Matplotlib does not provide any interaction out of the box. This means we cannot zoom, pan, or otherwise adjust our view of the arch model. However, now that we have our geometry, we can use any number of external libraries for rendering the geometry dynamically in the browser. Similarly, we could write the geometry as a DXF polyface mesh and load it into a CAD package such as MicroStation. These next steps will be left for a (potential) future blog post.

Why all this code? Why not just model the arch directly from a spreadsheet or other data source?¶

There are three main benefits of utilizing computational geometry for 3D modeling such as I have done with the Gateway Arch:

1. Parametric Values for Maximum Flexibility and Iteration Speed¶

The entire process, from start to finish, is based upon parameters that define the overall arch geometry. If this were a proposed project still in the design phase, we could easily do multiple variations as part of the ‘what if?’ iterative nature of design. For example, we could change one or more of the constants to scale the arch to a greater height, or adjust another to vary the arch from the 1:1 aspect ratio.

2. Change management with source control tools such as git.¶

As any architect or engineer will tell you, change is a constant during the design process. Ideas will be discarded one day only to be resurrected days, weeks, or months later. Because we are developing all of the geometry with python code, the parametric relationships are all captured in plain-text files versus proprietary binary formats typical in CADD and BIM software. Utilizing a tool such as git means that every iteration can be checkpointed with a commit hash, timestamp, and optional tag. Reverting to a previous version of the design (or even carrying multiple variations forward at the same time) is as easy as checking out a specific commit or branch in the revision history.

3. Reducing human error¶

When geometry is being created or entered by a human operator, there is always the possibility of ‘fat-fingering’ the data via typos or other transpositional bugaboos. By allowing a computer to generate the coordinate geometry algorithmically, we eliminate this particular source of error. Yes, there is still the potential for bugs or other errors in the code, but careful checking along the way either manually (as I have done in this example) or via an automated test suite (best practice when using code in production), those potential bugs can be fleshed out early in the process.

Modeling the St. Louis Arch

2019-10-28T00:00:00-04:00

Learning By Doing¶

Last year over Spring Break, we visited St. Louis. Our trip included a stop at the arch, and we managed to take a successful trip to the observation deck at the top. Shortly thereafter, I happened across some information on the National Park Service page regarding the geometry of the arch. Being a civil engineer with an acute interest in geometry, I couldn’t resist tinkering around with these equations as a means of diving into computational geometry.

Research¶

A good first step before diving into any scientific challenge is to perform research. In doing so, I found a number of resources in addition to the official NPS website:

Gateway to Mathematics Equations of the St. Louis Arch

How the Gateway Arch Got its Shape

Owner’s Manual For the Gateway Arch

This initial research led to two important initial findings:

The geometry of the arch can indeed be specified mathematically.
The primary form of the arch is derived from the shape of a flattened catenary curve.

Catenary curves¶

Wikipedia is a good starting point for exploring the computational geometry needed to accurately model the arch. The basic equation for a catenary involves a hyperbolic cosine (cosh) and a single constant that defines the aspect ratio, or flattening factor, of the resulting curve.

In [7]:

import math

import matplotlib.pyplot as plt
plt.rcParams['figure.figsize'] = [11, 8.5]
import numpy as np
import mpmath 

def catenary(x, a):
    y = a * np.cosh(x / a)
    return y

x = np.arange(-4, 4, 0.1)
y1 = catenary(x, 1.0)
y2 = catenary(x, 2.0) 
yhalf = catenary(x, 0.5)

plt.plot(x, y1, label='a = 1.0')
plt.plot(x, y2, label='a = 2.0')
plt.plot(x, yhalf, label='a = 0.5')
axes = plt.gca()
axes.set_ylim(0, 6)
plt.title('Fig. 1: Sample Catenary Curves')
plt.legend()
plt.show()

Flattened Catenary¶

As mentioned in the reference documents, the shape of the arch is a modified, or flattened, catenary curve. This means that there are two constants needed in the catenary equation to properly scale and proportion the initial curve that will form the basis of the arch geometry.

Constants from NPS web site¶

To develop a mathematical model for the arch geometry, the limits of X and Y need to be developed. Additionally, the arch is a 3-dimensional shape created by a series of equilateral triangles that are larger at the bottom and smaller at the top. A flattened catenary equation provides the location of the centroid of each triangle. Two additional values are required in order to set the “width” or “thickness” of the arch at a given location.

These four values (width at base, height at apex, maximum thickness, and minimum thickness) are listed on the NPS website as follows:

In [8]:

fc = 625.0925 # Maximium height of centroid
Qb = 1262.6651 # Maximum cross-sectional area at arch base
Qt = 125.1406 # Minimum cross-sectional area at arch top
L = 299.2239 # Half of centroid at arch base

Equations from NPS web site¶

As noted previously, the arch is a flattened catenary and therefore requires two constants. These constants were influenced by the minimum and maximum cross-sectional areas as noted above. The constants are listed on the NPS website as follows:

In [9]:

A = fc / ((Qb / Qt) - 1)
C = math.acosh(Qb / Qt)

print(f"A = {A:.4f}")
print(f"C = {C:.4f}")

A = 68.7673
C = 3.0022

Interestingly, there is a small rounding difference in the fourth decimal place of the calculated value for A when comparing it to the published value of 68.7672.

Next, we develop a function to calculate the value of X at any provided Y, where Y is the vertical axis. However, there is an important adjustment required: A typical catenary curve “opens up” as Y increases upwards as shown in Fig. 1.

This would be a very unstable arrangement for the arch. Therefore, the equations provided in the construction blueprints inverted the direction of the Y axis so that the catenary would have maximum width at ground level. In other words, the positive direction of Y is towards the ground.

In construction, we typically think in terms of elevation as a distance above ground level. Therefore, we will define a “helper function” to convert Y values to elevation before plugging that value into the function for X.

Note that because we are solving for X at a given Y value instead of the other way around as shown earlier, the arch equation uses the inverse hyperbolic cosine.

In [10]:

def El(Y):
    return fc - Y

def X(Y):
    return (L / C) * (math.acosh(1 + El(Y)/A))

Let’s test our function. We know that the maximum height of the arch is given by fc. At Y = fc, X should be zero. Correspondingly, at Y = 0, X should be at a maximum, or roughly equal to L.

In [11]:

print(f"Value of X at Y = fc is {X(fc):.4f}")
print(f"Value of X at Y = 0 is {X(0):.4f}")
print(f"L = {L}")

Value of X at Y = fc is 0.0000
Value of X at Y = 0 is 299.2239
L = 299.2239

Let’s plot our function and see how things are looking. We will need to vectorize the function to apply it over an array of elevation values. We also need to plot a second, mirrored version so that we can get the whole picture.

In [12]:

elevations = np.arange(0, fc, 0.1)

X_vectorized = np.vectorize(X, otypes=[np.float])

centroid = X_vectorized(elevations) 

plt.plot(centroid, elevations)
plt.plot(-centroid, elevations)
axes = plt.gca()
axes.set_aspect('equal')
plt.title('Fig. 2: Centroid of Gateway Arch')
plt.show()

Interestingly, this plot mirrors the construction sequence where the two legs of the arch were constructed individually and then met at the top to form a single structure.

Conclusion¶

We’ve gotten a good start on developing our model via computational geometry. However, we certainly can’t call this a 4D VDC model yet. The next post will look at adding the triangular cross sections in order to calculate the inner (intrados) and outer (extrados) edges of the arch.

Code That Doesn’t Exist Doesn’t Have Bugs

2019-10-16T00:00:00-04:00

Table of Contents

I was catching up on the Talk Python to Me podcast on the morning commute when I heard this tidbit of advice. It struck me as one of those nuggets of wisdom or rules of thumb that you collect over years of experience.

Digging in to this axiom a bit, I realized that this aligns well with the choice of python as a programming language. Python is designed to be compact and succinct. Typically, it takes fewer lines of code to accomplish something in python than other languages such as C++ and Java. I’ve been working a lot lately with the python bindings for OpenCV and this is definitely apparent when comparing C++ tutorials to their equivalent versions in python.

Fewer lines of code equals a smaller attack surface for bugs to creep in. The same holds true for security issues.

Python isn’t the best solution to every problem, but it certainly is flexible enough to cover multiple use cases. I specifically appreciate python bindings to C and C++ codebases that let you have the best of both worlds: low barrier of entry at the python level along with high execution speed at the lower levels.

Does the US need a BIM mandate?

2019-09-30T07:09:00-04:00

Table of Contents

Current State
Parallels to the Built Environment
Summary

As BIM continues to mature and move from early adoption to the mainstream, some organizations around the world have developed specific standards for the organization of BIM data and mandated its usage for public projects. For example, there are multiple British Standards currently ratified and in use in the UK, e.g. BS EN ISO 19650. However, the United States has been slow to follow suit, and currently there is no universal BIM standard or mandate. While some might argue that a BIM mandate in the US is needed to drive adoption, that’s not necessarily the case. The dramatic improvements in collaboration and overall quality that result from a model-based design approach will be the primary drivers of change in the industry. Universal standards are valuable and their development will benefit AEC as a whole, but a universal mandate is not needed to effect the inevitable transition from CADD to BIM.

Current State

The US arm of the international buildingSMART Alliance has developed a National BIM Standard-United States. Similar to the US National CAD Standard (NCS), these resources provide guidance for standardization and implementation of BIM processes. From my vantage point, it seems that the BIM standard is enjoying some level of adoption within the vertical industry. There are also signs that this standard is starting to see some uptake for linear projects. This is encouraging, as it could help avoid the fragmented CADD standards adopted to varying degrees by individual state DOTs. Although road and bridge construction is very similar from one state to another, there is some friction in context switching between interstate CADD environments. A pooled effort to develop standard workflows, business processes, and terminology would be a great benefit and avoid duplicate effort at the individual state level. Vendors such as steel suppliers could then count on the fact that fabrication data supplied via IFC in one state would be consistent with others, thereby reducing overall production costs and RFIs.

Probably the biggest obstacle to BIM implementation is that our industry in general - and especially contractors - are wary of new tech and comfortable with the status quo. There is risk in that which is unknown. Risk ends up being priced as additional costs in bids as contractors insulate themselves when undertaking more work. This can be compounded by experiences of the ghost of tech rollouts past that have not gone smoothly. However, this issue seems to be resolving itself naturally as older generations continue to age out of industry and are replaced with younger workers that are more comfortable with technology in general, having grown up with mobile devices and social networking.

Parallels to the Built Environment

We have national standards for physical items such as steel sections and precast concrete beam configurations. Just as these standards for physical components help drive uniformity and efficiency, so too would standards for the data structure of the corresponding digital twin be beneficial. As these standards are implemented and battle-tested, they will be better understood by practitioners and better implemented by software vendors. A precaster wouldn’t want to buy a separate set of forms for each DOT they were prequalified in - especially if the variations were minor and seemingly aimed at individual preference as opposed to specific function. Similarly, software developers would much rather avoid significant configuration differences to accommodate multiple jurisdictions. Uniformity reduces both the initial development effort as well as the cost of ongoing maintenance and new feature development for a digital product offering.

Summary

A universal mandate for BIM seems attractive in theory, but it’s not a requirement for the disruption that is already underway in AEC. Practitioners that have already embraced BIM and model-based design would never dream of going back to 2D-only workflows. The improvements by utilizing a shared model as the single source of truth will continue to naturally drive adoption, as has been evidenced in the vertical industry over the past 15+ years. Universally applied standards for BIM data and workflows will be a force multiplier during this transition, but the transition will happen with or without a universal mandate.

Email is Not Document Management

2019-08-01T00:00:00-04:00

Table of Contents

The problem
The symptoms
The cure
Conclusion
- A parting wish

Email is for communication, not for document collaboration. Sure, the path of least resistance for sharing docs is to send an email attachment. However, when you find yourself focusing too intently on paperclips your email client, that’s probably a good sign that a different approach is needed. When possible, a better route is to send a link to a shared version of a document. Solutions such as Google Docs or Microsoft OneDrive go one better by allowing multiple users to concurrently access and edit a shared document.

The problem

Modern knowledge work often involves collaboration across a remote, geographically disbursed team. Even with a co-located team, other external users such as clients and subconsultants will typically be working in a separate system. Within the traditional design firm, low bandwidth performance across the WAN usually precludes a traditional network share pattern for a remote team. Above a certain threshold, more heavyweight solutions such as ProjectWise provide a great environment for collaboration, change management, archiving, and all manner of other business processes and data workflows. However, that threshold is fairly high and makes the most sense when the work is centered around a CADD platform.

Email is electronic mail, which is the digital equivalent to interpersonal letter writing in the past. Therefore, it’s difficult to keep the right people in the loop, especially as a thread grows over time in length and audience. Discussions are often ‘forked’, causing side conversations and splintered resolutions. Another pain point with email-centric collaboration is the fact that forwarding an email typically doesn’t maintain any attachments from the original sender.

The symptoms

Document management via email is most likely to germinate in remote teams where the primary means of communication is via email, versus some other tool such as Slack or Teams. Symptoms may also present in conversations along the lines of It was attached to the kickoff meeting invite and Did you get that attachment?

The cure

Luckily, this is a common malady with a well-known treatment. There are abundant options for modern document collaboration using tools such as OneDrive / SharePoint Online, DropBox, box, ProjectWise, BIM360, Newforma, Aconex, Procore, and many others. Most of these utilize a client-server pattern with local caching of files that is comfortable for users that are accustomed to traditional desktop and local network share workflows. Specifically for our team, SharePoint Online ticks most of the boxes for our needs. It’s a modern re-factoring of a system that people love to hate. The current version is reflective of the Satya era at Microsoft and is a much more pleasant experience than the SharePoint of the past. Another benefit of utilizing this tool internally is that it allows us to “eat our own dogfood” by using the same environment that we often propose in our solution patterns.

Conclusion

Document collaboration in Outlook is a common malady, but it is easily treatable with a few key principles:

share links, not documents (or in software terms, pass by reference, not by value)
make use of change management tools such as check in/out
everyone works from a single data store

A parting wish

Having spent a fair amount of time utilizing git for source code management, I get sparkly eyes thinking of a future where Word and Excel are fully integrated with git. Indeed, the past few years have seen more and more online documentation using git and Github for managing updates and corrections. I also recently heard of an internal trend at Microsoft where Project Managers are using git for day to day task management. That sounds very appealing, but may also be a case of seeing every problem as a nail best suited to the hammer I’m currently swinging.

Elevator Pitch

2019-04-23T12:05:00-04:00

Table of Contents

Civil Integrated Solutions
Why? What’s in it for me?
How about another analogy?
… for the low, low price of ZERO dollars!!!!
No silver bullet
Summary

This has actually been a common question since transferring to the Technology Solutions Center. I’m working in the same office, but not doing the same sorts of tasks as before.

Civil Integrated Solutions

This is the name of our new section, but it’s not great at explaining our mission. Our primary goal as a group is to help designers transition from 2D CADD to 5D Geospatial BIM. This is best achieved by thinking in terms of model-based design that focuses effort on a federated virtual model of the proposed project.

Why? What’s in it for me?

One of the perks of my job is that I get the opportunity to explore and utilize lots of new, shiny tech. Truthfully, there is a certain amount of 80’s hair band glitz in things like gaming engines that can photo-realistically render a model at 60 frames per second. But technology has to serve a purpose, and the purpose is this: We are going to build the project virtually first, so that we know how to do it for real in the field.

To answer this question specifically: when done correctly, model-based design can drastically reduce rework and eliminate monotonous tasks. Most road designers have battle scars from fighting through cross-section sheets where some revision needs to be made by hand over and over again. The equivalent on the bridge side is the junior engineer that spends an afternoon counting rebar, then punches those numbers into Excel and hopes that that the link to Microstation is still working so that the Bill of Materials will be updated properly. The key linchpin in our proposal is utilizing a shared model as the single source of truth for all design data. The plan sheet deliverables are then just a byproduct of the modeling effort a snapshot of the state of the design at a particular point in time during the design process. The quality of the plan sheets is greatly increased because they are all derived from the same source of data. Where in the past you might have found yourself cross-checking each and every strip grade elevation on a profile sheet against the annotation on the cross sections, now you can spot check a few select locations and rest assured that these differing viewpoints of the design will always be right because the model forces that consistency. Design verification happens on the fly, with inter-discipline reviews occurring each time that party updates their portion of the federated model. Additionally, quantity takeoffs and cost estimates become much more accurate when they are automatically and directly derived from the model, versus being computed or taken off manually from the plans.

How about another analogy?

This truly is a transformative experience and a major change to business as usual. I’ve heard users make the comparison between CADD and BIM as being similar to moving from typewriters to Google Docs. The metaphor holds up well, because each typewritten page needs to be assembled to create a book. However, modern publishing is much different - the authors and editors can collaborate sentence to sentence and page to page in realtime while maintaining a version history of each draft and each edit that was made. Milestone points can be set in the document and the working version can be rolled back to a previous snapshot. What’s more, an author can explore multiple plot twists and work through “what if” exercises that provide a more informed outlook on the best way to get to the protagonist’s victory. Then once the book is final, the data can be re-rendered instantly into multiple formats for distribution - one format goes to the printer for generating physical books. Another format will go to Amazon for distribution on Kindle. Other options might include PDF and HTML. End-users with electronic versions can instantly get errata corrections over the air - or even see books update to a newer version when available.

… for the low, low price of ZERO dollars!!!!

Other benefits of model-based design are the additional things you get “for free”. Visualization is a great example of this - by creating an intelligent federated model of elements that include material information (e.g. concrete, steel, asphalt, wood), the geometry of those elements can be realistically rendered for static pictures, video animations, and even VR experiences. Stakeholders can immerse themselves in the model and quickly understand nuances of the work. A picture is worth a thousand words, and a view of the model is worth a thousand plan sheets.

Additionally, pay item information that is incorporated into the model can then flow easily into construction and reduce effort in the field while simultaneously improving the quality of material records and inspection documentation. Plus, the best time to inventory an assist is when it is going in the ground or otherwise being incorporated into the work. Pictures, measurements, and survey data during construction inspection can then flow through to the enterprise asset management system.

No silver bullet

What’s the catch? This type of paradigm shift is challenging. It will take time. There will be some pain and agony along the way. But the benefits greatly outweigh the short-term growing pains. Plus, you don’t have to swing for the fences right out of the gate. A handful of solid base hits will quickly score runs. Implementation can happen in stages - one project might be better suited to a focus on visualization, whereas another may be a good candidate for asset capture during construction.

Summary

Come on in, the water’s fine! I personally was fortunate to work in a model-based design environment early in my career with a number of very talented individuals. I have also spent multiple years in the more traditional “2D of this plus 2D of that plus some Excel gets us kinda there”. I know which environment I would rather operate in, and if you’re being honest with yourself, I tend to think you’ll probably agree.

A VDC Manifesto

2019-04-07T00:00:00-04:00

Table of Contents

Manifesto for Virtual Design and Construction of Civil Works
1. Safety
2. Dynamic Views
3. Shared Databases
4. Metadata - Use Tags, Not Folders
- Case Study - Email Discovery During Litigation
5. Open Data Formats
6. Platform Independence
Conclusion

In 2001, a number of software developers met to discuss various methods of developing software. Together, they created a Manifesto for Agile Software Development. The release of this document added momentum to various lightweight software development processes that had started to appear as incremental improvements on a traditional waterfall development process.

In that same vein, I proposed the following manifesto for Virtual Design and Construction of Civil Works as a list of guiding principles in applying technology to dramatically improve the start of the art in design and construction civil engineering projects.

Manifesto for Virtual Design and Construction of Civil Works

Safety over all other competing priorities.
Dynamic views over static output.
Shared databases over discrete file transfer.
Metadata over folder hierarchy
Open data formats over proprietary ones.
Platform independence over specific hardware.

Similar to the Agile Manifesto, I posit that values on the left are preferred over those on the right.

1. Safety

Canon 1 of the Code of Ethics for the American Society of Civil Engineers reads as follows:

Engineers shall hold paramount the safety, health and welfare of the public… in the performance of their professional duties.

In the face of competing factors in weighing a decision, the considerations of safety trump all other concerns. VDC can help improve jobsite safety by providing a digital walk-through of a proposed construction sequence or activity before it is actually to be undertaken in the field. Skilled tradespeople that will actually be performing the work can view the simulation in advance and offer suggestions for improvement that would otherwise have not been identified until those activities are actually taking place.

Additionally, time-enabled modeling of construction activities such as picking and placing a structural beam can identify conflicts with overhead utilities and or adjacent public areas that would be left vulnerable if a failure of some sort were to occur.

2. Dynamic Views

Currently, one of the most-often heard phrases in the design office or job trailer is something along the lines of “Is this plan sheet the latest and greatest?” True, we have moved from physical paper and mylar to electronic sheets, but the pain point is still present. As soon as an electronic sheet is generated, it is limited as a view of the model at that specific point in time. With a shared data model that can propagate updates to all views (plan sheets) in a matter of seconds, VDC eliminates this source of friction.

As in all things, it is possible to have too much of a good thing in this regard. It is understandable and necessary for the certifying professionals to control the change management process in regards to releasing revisions for fabrication. However, changes can be managed in a multi-state fashion as opposed to the current state of affairs.

Today, revisions are often limited to a binary state - invisible or released. With a robust change order management system, updates proposed in design can be released to the entire project team as draft, pending or the like. The ease and speed of publishing revisions once they are approved also adds transparency to the “submit, review, and revise” cycle. A superintendent in the field could pull up the proposed changes on their device of choice and overlay the new information on the currently released version of that part of the work. This workflow helps break down the silos of information and communication that often form between the design team and the construction staff.

3. Shared Databases

In a similar way, a shared data model that functions as the “single source of truth” allows for greater collaboration and reduces cognitive load in managing versions of data as a project progresses. Before computer networking was common, users would share information by passing around a floppy disk containing new or revised data with some type of file naming pattern that functions as version management by proxy. How many times have you come across a folder like this:

Plan.dwg
Plan2.dwg
Plan-old.dwg
Plan-new.dwg
Plan-junk-john.smith.dwg
zzz-KEEP-Plan2-new-junk-old.dwg

(Don’t do this. Just don’t.)

Modern data environments such as Bentley’s ProjectWise theoretically can support a single shared data model, but are typically used in a manner much closer to a glorified FTP site. Which, when you think about it, is just a glorified version of passing around floppy disks. An easy way to remember this concept is to recall a guiding principle I received from a mentor years ago: Share Data, Not Drawings.

4. Metadata - Use Tags, Not Folders

This tenet builds on previous ones in that it advocates a workflow recalibration. Again, rather than digitizing a legacy paper-based workflow, maximize differentiating features of technology that let the magic happen.

In the paper-based design office, documents were physical sheets of paper (or vellum or mylar). When it was time to store these physical media, they were ‘filed’ into ‘folders’ in a large cabinet or drawer. Although we are much less reliant on physical media in our design workflows, we still carry around these legacy paradigms. This is keenly apparent in the network file share. We have multiple drives (cabinets) full of hundreds or thousands of folders containing files of our design data and drawings. In this setting, many pounds of blood, sweat, and tears have been shed in order to arrive at the “correct” folder structure to be used on each project. This structure somehow has to meet a Goldilocks ideal - not too big, not too small. In practice, you end up with something much closer to Frankenstein with hundreds of folders that are empty or containing no more than a handful of documents each.

Browsing through these folders day after day builds up neural pathways and mental connections that are not easy to re-wire. Good luck if you recently came on board from another firm that organized data in a similar, but slightly different manner.

The answer here is to use metadata - tags and other information - to allow users to group and locate data based on whatever types of associations are most useful to them as an individual. Technology then allows you to create virtual folders that work like “magic”. Not only does this eliminate the need for one-size-fits-all across all users and all projects, but it allows for functionality not otherwise possible with traditional data management.

Case Study - Email Discovery During Litigation

I have had the unfortunate experience of dealing with litigation on more than one project. In both cases, the discovery phase required extensive effort in poring over past project communication - the vast majority of which was stored in email. In both cases, we were provided a list of items that were to be discussed as part of one party’s claim for damages.

As you might expect, there were abundant instances where a given email discussion was not isolated to a single claim issue. Heavy Civil construction projects are complex, and problems do not typically manifest themselves in a vacuum.

In both these instances, we ended up having to make a number of decisions on an individual basis as to which issue was most specifically related to a given email chain. That correspondence was then filed into an appropriate folder. In the legacy workflow, a document can only be in one folder at a time because that’s how it works with paper and manila folders hanging in your desk drawer.

However, we could have created categories for the damage claim issues, then associate one or more of these categories to each email thread. Then, Search Folders in Outlook would provide the same grouping that we would see with traditional, hard-coded folders. However, we would have gained a killer feature by being able to “store” a thread in two or more (virtual) locations based upon the categories we had assigned.

5. Open Data Formats

Open, well-documented data formats are more valuable than those that hold the user hostage. In this case, we are focused on “free as in freedom” - freedom to use the tool that you like, freedom to inspect the data being generated by or shared with a user, and freedom to archive information knowing that you can retrieve it again in ten years without worrying about whether you still have a working copy of a specific package that is proprietary to a company that might have gone out of business in the meantime.

6. Platform Independence

Just as the Industrial Age was kick-started by the invention of the automobile, the Information Age correlates to the arrival of the Internet. Over time, computing becomes more and more platform-agnostic. Indeed, the success of ChromeBooks confirms that the browser is the new Operating System. Likewise, VDC should not be restricted to a particular manufacturer, platform, device, or delivery mechanism.

Conclusion

Disruption is complex and messy. By focusing on a half-dozen key tenets and guiding principles, we can prioritize effort towards initiatives that provide value and avoid spinning our wheels unnecessarily.

Why You Don’t Need to Wait for IFC5 for Bridge Modeling

2019-01-12T00:00:00-05:00

Table of Contents

General - Goals of IFC5 Bridge and Road
Geometric Positioning
Linear Geometry in IFC
But why would I give up all our Intellectual Property?
Design to Fabrication workflow
Summary

I’ve been following Industry Foundation Classes (IFC) for a number of years. IFC is a neutral file format based that is free as in speech and as in beer. In particular, I have been evaluating IFC as a base schema for modeling civil works such as roads and bridges. The general consensus at present from users and software vendors is that IFC for Bridge Information Modeling (BrIM) won’t really be possible until the Bridge-specific data types and extensions planned for IFC version 5 have been finalized in the spec and implemented by software vendors. The truth of the matter is that IFC is ready for BrIM now.

General - Goals of IFC5 Bridge and Road

Presently there is a group hard at work on the next major version of IFC which will be focused primarily on linear civil works such as highways, tunnels, and bridges. This work is intended to address perceived shortfalls in the general suitability of IFC as a data interchange format for these types of projects.

While I am certainly in favor of this effort, I take exception at the notion that IFC can’t really be used for heavy civil work - particularly BrIM - in the meantime. One software vendor in particular seems to be using the draft status of IFC5 as an excuse for dragging their feet on incorporating IFC compatibility into their flagship BrIM program.

In fairness, if their users aren’t requesting this functionality then I can understand if they aren’t making it a priority for development. But if that’s the case, own up to it. Don’t hide behind the timeline of a spec that is developed by a non-profit primarily under volunteer effort. Be straightforward with the user base - they will understand that software development is typically prioritized according to their needs and specific requests for functionality.

Geometric Positioning

In general, integrated 3D modeling as a whole has been much slower to catch on in heavy civil when compared to sectors more focused on vertical construction such as buildings, schools, and general commercial development. The primary explanation typically revolves around a fundamental difference in geometry positioning. In a building, columns, walls, and the like are typically referenced from a grid with letters increasing along one of the XY axes and numbers increasing along the other.

Horizontal Alignment

For linear work such as roads and bridges, positioning is primarily based upon a linear distance along one or more alignment baselines. Each baseline is typically comprised of a series of lines, arcs, and spirals. Elements such as the centerline of a pier are located by distance along (station), distance left or right from (offset), and angle relative to (skew) a horizontal alignment baseline.

Vertical Alignment

Elevations along the baseline are described by vertical tangent grades and parabolic transition curves between the tangent grades. The resulting 3D alignment baseline does not neatly obey any basic mathematical primitives and is often approximated with chorded linestrings. BSpline curves are often sometimes used for this purpose, although that is typically rare.

Typical Section

Horizontal and Vertical alignment combine to describe the centerline or baseline of a roadway. However, a roadway needs to have one or more lanes for it to actually be useful as a means of transportation. The arrangement and geometry of lanes on a roadway is referred to as the typical section. Oftentimes, the typical section across a given stretch of road or bridge is constant. However, if we are at a roadway intersection or interchange and need to accommodate turn movements or ramp entrances and exits, the typical section is going to vary.

In other words, one end of the bridge may have two through lanes and an auxiliary lane. The auxiliary lane could very well be a different width or even dropped completely by the time we get to the other end of the bridge. This may be described directly by a list of dimensions at various distances along a single alignment baseline.

Or, it could be described indirectly by the addition of a second alignment baseline that does not parallel the other. All of these factors combine to complicate the process of describing and locating the geometry of bridge model elements.

Superelevation

Not only can the lane widths vary, but these variations often involve changes in superelevation. Superelevation is the roadway engineer’s term for the banking that is applied to the pavement in horizontal curves. This concept is seen most profoundly on a race track, where extreme banking is necessary to keep the cars on the track at high rates of speed. In the same way, the outside edge of a highway curve is superelevated (raised above) the inside edge. This is typically described in terms of percentage cross slope, or rise over run, across the width of a travel lane.

Putting it all together

So, if we are going to properly model a bridge deck, which is the portion that you actually travel upon, we need to be able to locate elements correctly using one or more horizontal and vertical alignments while also accommodating typical section and superelevation. This is indeed very complex, and most current BIM offerings such as Revit do not include tools out of the box that excel at generating and managing this geometry in a parametric manner.

Linear Geometry in IFC

Horizontal and vertical geometry were added to the IFC standard with version 4. Geometry plus sectioned spine covers the rest, as outlined in Volume III of BrIM protocols proposed by the FHWA.

But why would I give up all our Intellectual Property?

First off, the question of IP is going to vary from client to client and from contract to contract. My hunch is that there isn’t quite as much patentable, unique IP on a typical design project as some might suspect. But ignoring that issue, let’s focus on a specific part of the infrastructure circle of life - Design to Fabrication.

Design to Fabrication workflow

This step involves the handover of contract documents that include the technical drawings (plans) and specific requirements (specifications) for the materials and methods to be used in the work. The information is transacted from the designer to the construction contractor, who will typically then involve numerous subcontractors and other third parties to complete the work.

Often times, the contract documents go to a fabricator for specific parts of the work. For example, the structural beams will be sent to a supplier who will develop specific fabrication details (shop drawings), then fabricate and provide the beams that are needed for the bridge.

This process is labor-intensive, time-consuming, and prone to error as another party recreates and re-establishes data from the design information. This specific exchange of information is a great place to provide data from the BrIM in an open format that can easily be exchanged.

At this point, the design is complete. Any complexity in terms of parametric geometry or other supposed intellectual property is of lesser value when compared to the process quality improvements when working from a shared BrIM.

You can think of the BrIM as more of a PDF than a CAD file. Indeed, this specific exchange and Model View Definition (MVD) is addressed in the FHWA study previously discussed. In summary, that study found that IFC 4 is suitable for the Design to Fabrication workflow for the two sampled bridges.

Summary

In closing, there’s no need to wait for IFC5 before incorporating IFC into BrIM workflows. New standards take time to be incorporated into software and further refined by lessons learned on actual live project data. IFC is well established and has been in use for nearly two decades. It is robust, extensive, flexible, and best of all - free and open. Why lock in to a proprietary format when a superior open source alternative is available?

Web Development and Imposter Syndrome

2018-08-31T00:00:00-04:00

Table of Contents

The Deliverable
The Solution
The Analogy

I reached a milestone in my career this month when our team delivered a single-page app to one of our clients. This is exciting and nerve-wracking at the same time, but I feel better when thinking back to a similar time in my career.

The Deliverable

My role on particular project primarily involved work in Microstation, an environment that I am very comfortable with. We were performing a virtual analysis of clashing proposed geometry against the existing state as captured in a LiDAR point cloud. As we moved from analysis to documentation, we took an opportunity to develop a more modern approach for delivery than what we had done in the past.

Our delivery included approximately 300 cross section sheets with two frames on each sheet. The left frame included the proposed geometry and highlighted the clashes that we had found with the existing point cloud. The right frame included the same view but overlaid the point cloud onto an image at the clash location. Typically, we would deliver this information as a single PDF, which would run to the high 10s of MB. Additionally, this single document is not well-suited to quickly locating a particular point of interest.

The Solution

The first step was to keep each cross section as a single PDF, named by the corresponding alignment and station. Rather than delivering via file share such as FTP or USB media, we decided to build a geospatial SPA to share this information with our client. Each cross section was located along the alignment by lat/long and color coded based upon the severity of the clash at that location. Additionally, we built some UI controls to allow for filtering by the four alignments.

Our Microstation data was set with control to the local State Plane coordinate system, which allowed for easy translation to lat/long. I built the app prototype in a Jupyter Notebook, using the excellent folium library, which is a python wrapper around Leaflet.js. With a lot of deployment help from a senior developer, we were soon up and running with an interactive delivery secured by standard user authentication protocols.

The Analogy

In March of 2002, I had been out in industry for a few short months. I had learned AutoCAD in college and had taken a position as a bridge engineer with a local firm. I was tasked with developing the deck rebar layout for a bridge rehab project we were working on. It was easiest for me to draw the spacing and locations of the bars directly in AutoCAD as I worked through that process. Once the design had been checked, it was time to stick a titleblock on it and add it to the “official” plan set. I remember being very nervous the first time that Ctrl-S pointed to the project location on the network share, versus my private network working directory. It was official - something I had designed was now going to be used to construct an actual piece of infrastructure. Things got real when those bits and bytes were lined up to become concrete and steel.

Sixteen years later, I have been through that same process thousands of times in the world of plan production for construction projects. I’m hopeful that in the years to come, I will develop a similar comfort level when pushing web pages and mobile apps out to production.

Goodbye Excel, Hello Pandas

2018-08-18T00:00:00-04:00

Table of Contents

Basics of software architecture
Pandas to the rescue
- Import / Export CSV
Summary

I’ve heard a lot of positive buzz around the Pandas library for basic data processing with Python. After a few weeks of use, I can confidently say that it is a dramatically better tool than Excel for data driven tasks and workflows.

Basics of software architecture

Typically when building a software solution, it is important to have separation of concerns. This makes the solution more modular and flexible to adapt to changing inputs and requirements. Typically, this separation results in divisions between the Data Layer, Business Logic, and User Interface. In Excel, these layers are all mushed together and it is difficult to isolate particular functionality - particularly the processing and business rules that are heaviest in terms of formulas, functions and algorithms.

One major source of errors when working in Excel is in the application of formulas for calculated columns. A typical worksheet usually ends up in a table-like structure, with columns of data alternating with calculated fields based upon that data. However, the worksheet interface makes it very difficult to tell which values are data and which ones are calculated. Even worse, formulas that are almost always applied uniformly to each row of the table need to be updated and managed manually by the user. More often than not, Excel workbooks that I’ve found in the wild had multiple issues where a formula was updated for a subset of the table but not copied uniformly throughout the entire table. Even worse, it’s very common for a user to unknowingly overwrite a calculated field with discrete data. Again, unless the end user checks the formula for each cell, there’s no way to catch this sort of critical error hiding in the weeds.

This issue can be mitigated in part by writing custom user functions. This helps in that the formula is now in a single spot and only needs to be changed in a single location in order to update all calculated instances to the correct algorithm. However, the other issue still remains in that this calculation can easily be overwritten as data, with no help from the user interface. Even worse, writing a custom function means that logic is now moving from a spreadsheet formula into a VBA macro. As soon as you make this leap, you are giving up on the primary benefit of Excel - namely its ubiquitous install base and user familiarity. In other words, the vast majority of users are comfortable and able to audit and update formulas in cell calculations. However, that percentage of users likely drops to single digits when we are talk about a similar comfort level with reviewing and updating VBA code in macros.

Additionally, VBA macro code stays with the workbook that it was written in. It is very challenging to manage VBA code in a central location and deploy it across multiple installations of the “software”. This has been a constant Achilles heel in my experience - A macro or series of macros is developed in a “template”, then tested and rolled out for widespread use. Invariably, a bug or request for additional functionality is raised. This code change then needs to be rolled out uniformly across all of the workbooks that are currently in production. But this is much easier said than done, because those *.xlsm workbooks are now spread across multiple network share folders, email attachments, FTP sites, …

Pandas to the rescue

The name “Pandas” is an abbreviation of sorts for Panel Data. The concept of panel data will be very familiar to Excel users - it is essentially a table of rows and columns. By default, the rows are numbered (starting at 0) and the columns are labeled with headings or field names. Let’s take a look at common tasks in Excel and see how these are easier and more robust when using Pandas.

Import / Export CSV

Comma separated values (CSV) are very common for exchanging data between systems or applications. A very common use case in my industry is topographical survey data. When a project is being planned, a field crew will visit the site and use precision measurement tools to record the existing condition of the site. This data is then typically delivered as a plain text file with comma-separated values. Typically the data will come as hundreds or thousands of data points with field headings such as point number, northing, easting, elevation, point code, and description. This corresponds to attributed 3D geometry that can then be used to develop an existing map of the topography, as well as build a Digital Terrain Model (DTM) that can be used for analysis and design.

When processing CSV in Excel, my observation is that users will typically copy and paste from the text file directly into Excel - because this is easiest. But this represents the first deviation from the desired architecture with separation of concerns. The data is coming from an external source, and Murphy’s Law dictates that it will change and/or be appended in the future. Managing updates, particularly when the size (number of rows) changes, should be the responsibility of the software, not the user. I’ve never had much luck setting this up in Excel. My understanding is that this process would be done via links that auto-update, and formulas would (maybe?) auto-update and copy down for new rows as needed. Again, this might be user error on my part, but I tend to be fairly tech-savvy and I’ve never had much luck getting this set up correctly. With Pandas, it’s super-simple and is oriented towards maximizing the possibility of falling into success. In other words, the easiest way is the “right” way:

Point to a file, configure a header row, change headers if necessary, drop columns on input, drop rows if values are missing.

Summary

What a breath of fresh air! Many design engineers spend the majority of their time poring over spreadsheets, chasing stale links, or bemoaning formulas that didn’t get copied correctly across all rows in a given column. I’ve certainly had my share of those moments and that shared pain. I know lots of folks that have a warm place in their heart for Excel, and I can understand that. For me personally, there’s no looking back.

Why VDC?

2018-02-20T22:00:00-05:00

Table of Contents

Terminology
Benefits During Design
Benefits During Construction
Operations and Maintenance
- Asset Management
Drawbacks of VDC
Conclusion

The AEC industry is slowly becoming more and more digitized as technology is incorporated into day-to-day operations. A primary example of this technology uptake is in the adoption of Building Information Modeling (BIM). The purpose of this post is to solidify the business case and value-add that results from moving to information modeling as the linchpin of modern architecture, engineering, and construction.

Terminology

Before diving in too deep, let’s take a moment to discuss terminology. When I think of BIM, what first comes to mind is vertical construction - schools, hospitals, commercial buildings, and the like. This segment of the industry has led the charge in adoption of BIM, and at present, practitioners founded on 2D drafting as the primary means of document production are squarely in the minority. However, on the heavy civil / infrastructure side, BIM adoption has been very slow. There is also much less consensus on terminology. There aren’t many physical buildings on a typical highway or bridge construction site, so BIM seems to be a poor fit. Some acronyms currently in use include:

BrIM - Bridge Information Modeling
CIM - Civil Information Modeling
Horizontal BIM - (pretty self explanatory)
VDC - Virtual Design and Construction

I tend to use VDC as my acronym of choice, hence the title of this post.

In my experience, there is plenty of reference to modeling as an integral part of the design process. When used in this context, modeling typically refers to surface modeling. This usually involves the creation of templates or assemblies that represent the cross-sectional geometry of the roadway section. Templates are ‘pushed’ along a path defined by a horizontal and vertical alignment to create a digital terrain model (DTM) or triangulated irregular network (TIN). These surfaces are thin meshes that cannot accommodate two points with the same (X,Y) coordinates but with differing Z (elevations). In other words, these surfaces cannot contain vertical faces. Additionally, DTMs cannot “double back” on themselves to accommodate a feature such as the 45 degree edge of an asphalt pavement section. Perhaps because of these limitations, and / or the general conservative nature of the industry, typically these models are used only for the production of traditional 2D paper or PDF deliverables. Additionally, these surfaces are typically geometric only, with little to no semantic information associated to the data. When I make the case for VDC, I am speaking of 3D solid models in addition to the surface models that are already commonplace. Additionally, these models are much more valuable when they contain semantic metadata that further enriches the information. For example, a VDC model of a bridge might associate material properties such as the density and design compressive strength of the concrete that is prescribed for a 3D cylinder representing a pier column.

So, with this initial foundation of terminology having been laid down, let’s move into the benefits of adopting VDC as an integral component of capital infrastructure projects.

Benefits During Design

Visualization - external audience

Public works projects are, by definition, meant to enhance the general public’s quality of life. This means that it is important to share and describe the proposed work with those who will be most affected during and after construction. However, the proposed work is nearly always communicated via traditional engineering documents or hatched and colored 2D graphics overlaid on an aerial image. A majority of stakeholders struggle to understand these documents due to their highly technical nature.

On larger projects, it is not uncommon for 3D renderings and fly-through videos to be prepared to address this issue. However, this visualization effort often involves unproductive rework due to a lack of overlap in skill sets between the design team and the graphic artists.

When utilizing VDC, the design team gets the visualization “for free” due to the 3D geometry and associated non-graphic information such as material properties. A properly-configured modeling environment is typically able to produce renderings, animations, and even AR/VR datasets with minimal additional effort.

Visualization - internal audience

Integrated visualization is also helpful to the design team. One vendor touts the design-time visualization capabilities of its software as a means of reducing Errors and Omissions during design. find errors more quickly. One example that I am familiar with is the tie-in between the roadway section and concrete approach slab at the ends of a bridge. For constructability reasons, the cross section of a bridge deck is typically made as planar as possible.

The approach roadway might use 2% cross slopes in the travel lanes and 4% cross slopes on the shoulders. The shoulder cross slope needs to be transitioned to match the bridge section where both the travel lanes and shoulders are at a constant 2%. This detail is often overlooked, or buried in plan details like a piece of text on the typical section sheets or a handful of lines added to a superelevation diagram. This potential land mine becomes clearly visible when the 3D model of the bridge gutter is hanging above the adjacent roadway shoulder.

Additionally, conflict checking and clash resolution tools common in VDC software help disciplines stay out each others’ way and avoid construction delays when a storm sewer and bridge foundation are fighting to occupy the same space.

Constructability

Most people are familiar with 3D modeling, where the x and y (or northing and easting, if you rather) of plan view are combined with elevation - the pesky Z. 4D models that capture the time scale in addition to the XYZ of geometry can prove to be invaluable. During design, the primary answer provided by a 4D model is whether the design can actually be constructed in the sequence imagined.

Estimating

Adding cost to a model bumps it into 5D territory and provides even more benefits during design. Like visualization, quantity and cost estimating are mostly accomplished “for free” by the use of common, standardized part libraries with associated historical bid histories. A designer may only do major quantity take-offs and cost estimates once every few months, which make it difficult to gain efficiency with these tasks. Also, many designers take on this type of work begrudgingly due to the primarily repetitive and manual processes involved. By incorporating smart, parametric elements pre-populated with cost data, these tasks can be accomplished quicker and with fewer errors and omissions.

Benefits During Construction

Thus far, I have spoken primary of VDC benefits during design. The majority of my career has been spent in the design office, so that’s where my initial focus lands. During construction, the primary benefit of VDC is that it allows you to build the project virtually so that you know how to build it physically. This practice is often seen in industrial settings, where prototypes of new assembly lines might be built in a pilot plant.

By first constructing in a virtual dry run, the builder can confirm the critical path and uncover challenges that otherwise would not have become apparent until in the field. Additionally, the combination of the geometry, estimate, and schedule into a single model will illuminate issues or mistakes that otherwise would have been missed when looking at these items in isolation.

Additionally, the VDC model can be used during construction to monitor earned value and completion progress against the baseline schedule and estimate. Mobile and static lidar scans of the construction site can provide an accurate assessment of project completion at regular intervals.

This data can also be used for quality purposes such as confirming that the finished work is within the specified tolerances. These progress scans can also be used to record the “as-built” state of the project.

Operations and Maintenance

This is a bonus section, as thus far we have been talking primarily in regards to design and construction phases of the asset lifecycle. VDCOM is a bit of a mouthful, but there are undeniable benefits realized by continuing to update the VDC model as a construction project is completed and the infrastructure is put into service.

Asset Management

A geospatially-referenced VDC model of the as-built project naturally flows into an agency’s asset management systems. This centralized data store consolidates a wealth of information that otherwise would be spread across plan sheet markups, scanned PDFs of material tickets and test results, data dumps of progress estimates and change orders, random spreadsheets of non-conformance reports, … ad infinitum. In particular, civil works developed under a P3 delivery process often have long-term contractual requirements such as warranty periods that affect the availability payment mechanisms that allow a developer to recoup initial capital investments. Capturing this information during construction and spatially associating it to the warrantied assets make it much easier for an agency to avoid any costs associated with substandard craftsmanship or materials.

Most importantly, VDC functions as a “killer app” in that it provides x-ray vision for subsequent inspections of a facility. For underground utilities and infrastructure, the best time to measure, photograph, and record the location and properties of an asset are when the trench is open during construction.

Drawbacks of VDC

To be fair, VDC implementation isn’t all sunshine and roses. There is a significant investment required for most practitioners to make the leap from 2D drafting to multi-dimension digital modeling. And, there is a certain threshold in terms of project size and complexity below which there is less benefit derived from VDC. For example, a contract to add HDPE linings to a series of existing corrugated metal drainage pipes is not necessarily a good candidate for these processes. However, in the majority of cases, the benefits of VDC far outweigh the initial challenges and learning curve during implementation.

Conclusion

Most of those who have already moved to BIM or VDC could never imagine going back to 2D drafting. The difference between environments has been likened to using a typewriter versus Google docs. In some ways, this transition is actually more disruptive than the leap from the drafting table to the CAD workstation. That change primarily involved a change in tooling to perform (mostly) the same processes.

BIM and VDC, on the other hand, are a fundamental re-thinking of how we work and collaborate with our partners and stakeholders. It is an immense challenge, no doubt. But it must be met if we are to pull the industry out of the rut of low productivity, blown budgets and missed schedules that far too often characterize the AEC sector.

Greg Brady, Architect

2017-12-19T20:00:00-05:00

Table of Contents

Early career aspirations
Historic context
- IBM System/370 Model 145
Conclusion

This week, I found myself at another consultant’s office for a project kickoff meeting. As we were waiting for others to arrive, I noticed that the reception desk contained a rolled up set of paper plans on top of a transmittal letter. I was struck by this, as it reminded me of how little our industry has changed in the last half century.

Early career aspirations

From my earliest memories as a young elementary school lad, I had always wanted to be an architect when I grew up. In some ways, this may have been influenced, or at least reinforced by the character of Mike Brady. In middle school, after learning more about the distinction between architecture and engineering, it quickly became apparent that my calling actually lay in the latter. But, that is a story for another time…

One specific episode that I remember involves Greg Brady getting a part time job helping his father in the office. The plot thickens when he is entrusted with delivering a set of blueprints to an important client. Seeing the plans on that desk reminded me that although we have traded drafting boards for CAD workstations, the primary medium for information exchange in our industry is still paper. I was curious to do some research and see how long it had actually been since that episode first aired.

Historic context

Although my memories of this episode date from the late ‘80s, it originally aired nearly 20 years earlier, on October 30, 1970. That means that we are a few short years from this cultural reference dating from a half century ago.

So, approximately 50 years of progress in AEC took us from board drafting and delivering a roll of paper plans to… computer-aided-drafting and delivering a roll of paper plans. Let’s take a look at how things have changed in computing over the same time frame:

IBM System/370 Model 145

About a month before this episode aired, IBM announced its latest iteration on the System/370 line of mainframe computers.

This model was notable as the first machine released by IBM to use ‘monolithic’ silicon memory chips, as opposed to the metallic rope core memory of its predecessors. A basic configuration of this machine would include 400 kb of RAM and 233 Mb of hard drive storage. This breathtaking amount of computing power could be rented monthly for upwards of $15k or purchased outright for a sum of $705k or more (1970 dollars). Needless to say, we’ve come a long way since then.

Conclusion

Perhaps it’s unfair to compare the civil engineering industry to the tech sector. After all, we have been crafting with concrete for millenia, while silicon transistors have appeared just in the last century. Just as concrete strength gains level off after a relatively short period, so too would we expect technology to stabilize as it matures over the coming decades. Still, I can’t help but be frustrated by the pace of progress in technology versus the general stagnation of the AEC industry over a similar time period.

A Weekend on the Command Line

2017-10-28T23:00:00-04:00

Table of Contents

Bash on Ubuntu on Windows
Windows Subsystem for Linux - Fall Creator’s Update
- Why WSL?
Jedi-Vim
tmux
cmake and make
Github
Conclusion

Recently, I spent the better part of a weekend installing Linux under Windows and building some command line muscle memory. Not only did this re-kindle some fond memories from my early days of computing, but it also began to set the stage for effective python development.

Bash on Ubuntu on Windows

Recently, Windows 10 gained a beta feature that allows you to install and use the bash shell as if you were working on Linux. As I understand it, the Unix system calls are translated on the fly for execution in the NT kernel. In other words, it’s not a dual boot setup or a virtual machine layer. Rather, it’s a very interesting mix of the typical Windows experience with a tasty smattering of the command line for those who are so inclined.

Windows Subsystem for Linux - Fall Creator’s Update

The recently-released Fall Creator’s Update for Windows 10 included a host of improvements and added functionality. Of personal interest to me, WSL no longer requires developer mode and is no longer considered to be a beta feature. This development, in particular, appealed to me as it seems to indicate that this hybrid workmode is less of a niche target and more of a mainstream use case that Microsoft is serious about supporting and improving. Additionally, being able to install multiple distributions from the Windows App Store indicates a serious level of collaboration with the individual Linux distributions.

Why WSL?

Since I started hobby programming in python roughly a year ago, I’ve noticed that most documentation and examples seem to assume that the user is working in macOS or Linux. At times, Windows seems to be a second-class citizen. When documentation for Windows is included, Visual Studio typically figures prominently as an integral component of the build toolchain. I’m curious to see if the sledding gets any easier in an environment where compilers and other dev tools are installed by default.

A second motivator for installing WSL is the fact that it is pretty effectively sandboxed from the important bits of Windows. Both of the machines that I have been hacking on are shared with the rest of my household, and I am careful to keep in mind that any software installs or registry hacks will at least cause some clutter in the \Program Files directory, and at worst could cause serious deterioration or even destruction of an appliance that is critical to our day-to-day adult responsibilities.

Lastly, I was a full-time user of Ubuntu in the Dapper-Edgy-Feisty timeframe, and am curious to see how far the state of the art in Linux has come since then.

Jedi-Vim

One issue that has been an on-going frustration with python is getting autocompletion to work consistently. I was never successful in installing and configuring the plugins necessary for Python auto-completing to work in Atom. Later, I had some success with Visual Studio Code, but found that the hardware requirements were a little steep for our primary desktop PC.

Since reading that Vim and python are a match made in heaven, I took the plunge with vimtutor a few weeks ago and have started to crest the first hill in the learning curve. This past weekend, one of my first victories was the successful integration of jedi-vim and YouCompleteMe. This has definitely made python development much more enjoyable, as I can preview function signatures and module documentation directly in the editor. I have gotten very accustomed to Intellisense and library browsing in the VBA IDE, and am happy that my user experience in python will be very similar from a usability standpoint.

tmux

tmux is a terminal multiplexer that allows for an “editor over shell” configuration similar to the default layout in Visual Studio Code. This setup makes it very easy to drop into git and other cli tools while keeping context with the file(s) currently being edited in vim. This was an easy install, and with the help of a basic cheat sheet I am now splitting and spawning multiple terminal windows on a single screen with nary a hint of input from the mouse.

cmake and make

With tmux and auto-complete now up and running, I set my sights on building some software from source cloned from github. This brought back some memories from my initial forays into Linux circa 2003 that involved multiple failed attempts at installing software from source. This time around there were a few hurdles to clear, but eventually the package built successfully and the module imported with no complaints.

Speaking of github…

Github

This weekend also marked some of my first forays into code collaboration on Github. For one, I created a feature branch in a repository that I had cloned a few weeks earlier. I made some changes and did some testing to confirm that everything was working as expected. Then, I noticed that the upstream master branch had been updated since the time of my fork. So, I had to merge my master branch with the new changes from upstream, then merge my feature branch and push back to my remote fork. Of course, I didn’t get all of that right the first time through. After a few newb mistakes, I eventually got to a pretty messy state of affairs. With some help from Google, I ended up issuing a reset --hard as a sort of mini ‘nuke and pave’.

Conclusion

Although it took longer than initially planned, I ended up re-connecting some cli neurons and getting reasonably proficient in bash, git, and vim. These new workflows remind me of the many hours spent in front of a monochrome green terminal connected to an Apple ][e or IBM PC/XT. In some use cases, I actually much prefer the command line to wrangling a mouse through menus, windows, and inconsistent form controls. It definitely feels like I have settled into an environment that will serve me well and fits my personal preferences on computer usage and software development — at least, with python.

Hello, Pelican!

2017-03-01T00:00:00-05:00

This is my first post in Pelican. Training wheels are on. I have been obsessing a bit over selecting a theme, but it’s probably best to put together more content before getting too crazy about UI tweaks.

I am quite enjoying building my chops with static site generation. This is much more appealing than hand-coding HTML in Notepad or that week I spent in FrontPage 97. Now, with a little CLI and python under my belt, I’m happy that those days are safely in the past.

Learn Python the Hard Way

2016-10-15T10:40:00-04:00

Table of Contents

My first steps with Python
Learn Python the Hard Way
Conclusion

My first steps with Python

I’ve been programming for a number of years now, with the majority of my time spent writing VBA tools for our CAD system. For my birthday this past spring, I got Javascript for Kids. I was introduced to No Starch Press recently at our local Barnes & Noble, and thought that this particular book might be useful for myself and (potentially in the future) for my kids.

In the process of researching no starch, I happened across a title Automate the Boring Stuff with Python. This concept resonated deeply with me, as the primary motivator of my programming work to date has been aimed at automating manual (and sometimes boring) workflows.

I’m still working through the javascript book, but have taken an interest in starting to learn python as well. In addition to the boring stuff book, there are a handful of other reasons for heading in this direction. For instance, I’ve noticed that python appears to play a starring role in the Raspberry Pi ecosystem.

I’ve crossed paths with python a number of times in the past, generally in the context of Linux. I seem to have absorbed some of the ‘just a scripting language’ bias in the past, but now I’m ready to take a fresh look.

Learn Python the Hard Way

So, here I am. I’ve surveyed the internet for resources providing instruction on getting started with python. I happened across Zed Shaw’s Learn Code the Hard Way series and have quickly dug into the python flavor.

I have been following the instructions to the letter and now have a working install of python, with Atom as my editor. As expected, it has been slow going as I start climbing the learning curve. I eventually worked my way up to this tidbit, which is example 5.

his_name = 'Zed A. Shaw'
his_age = 35 # not a lie
his_height = 74 # inches
his_weight = 180 # lbs
his_eyes = 'Blue'
his_teeth = 'White'
his_hair = 'Brown'

print "Let's talk about %s." % his_name
print "He's %d inches tall." % his_height
print "He's %d pounds heavy." % his_weight
print "Actually that's not too heavy."
print "He's got %s eyes and %s hair." % (his_eyes, his_hair)
print "His teeth are usually %s depending on the coffee." % his_teeth

# this line is tricky. Try to get it exactly right
print "If I add %d, %d, and %d I get %d." % (
    his_age, his_height, his_weight, his_age + his_height + his_weight)

Conclusion

So, I’m officially on my way to learning python. Things are going slow, of course, but I feel like I’m quickly acclimating to the whitespace. I really like this course! It seems to be very straightforward, and I like the philosophy of putting in your time and not looking for shortcuts. Of course, months from now, I will look back and marvel at the simplicity of the code I wrote in these first few days with the language.

Hello, World!

2016-06-01T00:00:00-04:00

10 CLS
20 ECHO "HELLO WORLD!"
30 GOTO 20

At present, a primary mission in my professional realm is to help move the AEC industry into the Internet Age. As the oldest of engineering disciplines, we have gotten comfortable with terminology and workflows honed in the nineteenth and twentieth centuries. The proliferation of always-on connectivity and instant access to information has dramatically reshaped society since the dawn of the new millenium. However, many segments of AEC still operate as if there is a large drafting room in the home office filled with rows of t-squares and electric erasers.

64-bit Public Infrastructure

As I wrapped up my Bachelor’s degree in 2001, mainstream computing lived in Windows NT or 2000. A few years into my career, I began to see glimpses of 64 bit computing and spent many productive years in the land of Windows XP, then Windows 7. In particular, the ability to address more than 2GB in RAM drastically improved the performance of our CAD platforms. Hence, the x64 suffix in my username is used to indicate a modern, contemporary vision for Civil Engineering.

Heavy Civil, Big Data

Calculating cant alignments in IFC

Conclusion¶

A Framework for Digital Delivery of Infrastructure Projects

Thesis

Current state

PDF - based delivery

Common, Open Format - IFC 4.3

Data must meet specific requirements

Information Delivery Specification

Specific Requirements for Project Information

Model as Legal Document

Conclusion

Automation investment and payoff

Pitfalls¶

Obligatory xkcd reference¶

The Problem Statement¶

The Solution¶

The Calculation¶

Conclusion¶

SmartHouse.__init__()

Prologue: Meross Smart Switch

Smart Thermostat

Hardware Installation and Unexpected Re-wiring

A new life for an old Pi

Initial thoughts

Next Steps

Book Review: Python Testing with Pytest

Python Testing With pytest

My Take

Strengths and Weaknesses

Pros

Cons

Conclusion

Throwback Thursday - TiddlyWiki

A New PM Needs to Get Things Done

Getting Things Done

Enter TiddlyWiki

This is Great!

This is Not So Great

Tweaking Outlook

OneNote, The One Notebook To Rule Them All

Finding A Place For TiddlyWiki

A Personal, Linear Notebook For Paper

A Paper List of What Needs To Get Done

Lessons Learned

Why You Need Civil BIM, not CAD

CAD is a graphics display system, BIM is a feature modeling platform

The Fallacy of Data Organization By Layer

Civil CAD is better, but still has gaps

Surface modeling is not BIM

How CAD is merging with BIM

Conclusion

Gemini Principles for Digital Twins

Purpose

Public Good

Value Creation

Insight

Trust

Security

Openness

Quality

Function

Federation

Curation

Evolution

Wrapping up

Miles per burrito

Background¶

Research¶

Settling on a Unit of Measurement and Running the Calcs¶

Caloric Expenditure¶

MPG Equivalence¶

Infrastructure As Code

DevOps trends

An interesting parallel to the physical world

Conclusion

In Praise of the WYCIWYG Editor

Starting out on WP 5.1 for DOS

Office 97 lands

SmartHouse.init()