Autodesk Utility Design 2013 Release 2: The New Phone Books Are Here! The New Phone Books Are Here!

Well, maybe not exactly…but in a new release of Autodesk Utility Design (AUD) that showed up on the Autodesk Subscription Site last Friday (for those who have the appropriate subscription – Infrastructure Design Suite Ultimate, or AUD itself) communication and fiber support have been added in all areas of functionality – industry model, configuration, layout and analysis.

In addition, so additional functionality has been added. Those who were working with 2013 AUD knew that when creating new overhead lines, a 3D view of the lines and pole was created (complete with sag in the lines). Now the same 3D visualization has been added to underground facilities.

There’s a new analysis tool for overhead clearance. It identifies clearance between wires as well as with the ground.

 For those who do design based on work locations and stake-outs, a new work location block allows you to associate material and feature information with that work location.

For design output, material lists in table form can be added to the drawings with the “Insert Table” button.

An exciting new feature for a lot of electric utilities will be the new schematic layout tools. Non-schematic lines with conductors and ducts can be converted to a schematic view and then back. The new commands AUDEXPAND and AUD COLLAPSE will generate the schematic and then back to non-schematic views.

Some new electric distribution tools have been added – neutral conductor can be automatically added with the layout of primary, new span tools (create equal spans and preferred span lengths in addition to maximum span lengths), double circuit networks for overhead 3 phase systems have analysis and visualization capability, a spreadsheet-like load calculator for calculating demand load at service points, pole heights are now set by a pole height above ground pole leveling and custom status.

When you open an existing drawing with the new release, you will see a model update, and the new tools will be available.

I’ve worked with AUD for several iterations, and the current implementation is pretty exciting. It is a great tool for doing utility design. It is very easy to integrate with GIS and provide the calculations that engineers and designers need to get quality designs out faster. Watch over the next several weeks, and I’ll add some new blogs on AUD and utility design. If you haven’t taken a look at it, now’s the time.

The Tyranny of GIS – Part One

As a professional in this industry for over twenty-five years, it’s difficult not to notice common threads. One of those is the cost of doing GIS. I run into many smaller organizations, such as utilities or municipalities where the cost of GIS is burdensome or a complete barrier to the technology. These organizations often miss out on the efficiencies that GIS can provide them (the fact that even many enterprises with very extensive systems miss out as well is a topic for another time). In many cases, these organizations don’t have the personnel knowledgeable to make effective decisions about the systems and so become hostage to consultants who have interests outside of their own (as a matter of fact, I’ve heard the word “hostage” used many times in reference to GIS). Now as one of those consultants, I’m not self-loathing here, but pointing out that in all cases, a consultant has a mission of increasing their sale of services to be successful – this is different than the needs of his customers. “Good” consultants will align their mission with the customer’s – selling the customer the services they need. In many cases, though, the consultant may not fully realize the needs of the customer, and the customer lacks the technical understanding to fully express their needs.

I noticed this particular effect in a very simple transaction recently – getting my hair cut. I’ve noticed a trend lately. Many of the new stylists (I have to not show my age and call them barbers) ask detailed questions about the methods – cutting with scissors or clippers, or the specific length at the top or sides. Quite honestly, I have no idea how to answer these questions. I want it to look a certain way, but I don’t know how to get there. That’s why I get my hair professionally cut (aside from the fact that trying to cut one’s own hair is challenging). Does this sound familiar? I’ve seen this interchange with service providers from all areas of business, and quite often in the GIS world.

This breakdown of communication is purely the result of different frames of references. The result is that there are several areas of the process that are not effectively “sized’ for the organization. Areas like data precision, system design, system architecture, training, data sharing are specific areas of an implementation (or operation) plan that can get a bit out of control and drive the cost of the GIS up. Getting a handle on these can often bring the cost of these way down. I’ll tackle several of these over my next several entries.

Precision –The first area I’ll cover is data precision. This raises the obligatory issue of precision vs quality. Precision is the standard of measure we use, such as survey grade GPS, consumer GPS, steel tape, measuring wheels or “calibrated eye” (I once saw a survey while on the east coast that used a distance of a cigarette – I assume he meant the time to smoke it – I wonder if it was a regular or long?). Quality is how well we did the measurement (was my tape level, did I read the angle correctly, etc). For the purposes of this discussion, we’re talking about precision, and I may use the word quality, but I mean precision.

The actual GIS data is probably the greatest expense for most GIS programs. Of course, that makes sense, when you consider that data really is the GIS. While there is the old adage that GIS is software, hardware, processes, people and data, the reality is that all of the other components are interchangeable while the data is the reason we do GIS. Because of this, it’s easy to place an undo importance on the precision (quality standard) of the data. Often this is because the actual business purpose of the specific GIS data is poorly developed or understood. It is the actual purpose of the data which should define the precision standard of the data, but in many cases, we start data capture without clearly defining the need, and so data capture is at a higher standard than necessary or prudent. When speaking of data creation, the cost generally rises with precision (often at an increased rate).

I can hear some folks cringe at the suggestion. After all, the standard engineering definition for GIS is “Get It Surveyed” to identify their inability to rely on GIS for design. I’m not sure that’s a bad thing. As a designer, I consider it due diligence to survey the site for every project, as things change and the records may not show it. Relying on existing records is a way to guarantee delays, change requests and other problems with a project. Knowing that, then “basis for design” may not be a valid business requirement for GIS data. Generating, for example, parcel data that is suitable for design can be quite expensive and completely justifiable for a construction project (as a percentage this would be a small part of the overall cost of most projects). Generating data of similar quality for an entire municipality where a very small portion may ever be involved in new construction can be prohibitive and become a major impediment to developing GIS capability. Using a lower quality data set may be perfectly effective for the planned use of the data. In the parcel example, maintaining a link to the actual record of survey may provide satisfactory access to higher precision data.

I’m not suggesting that data always be at the lowest possible quality, but I am suggesting that prior to determining the need for accuracy, that a reasonable expectation on the data and it’s uses (immediate and long term) be evaluated. In many cases, having a highly precise base map provides benefits that are perfectly justified for the organization. I know of some utilities that use highly precise data to model their systems and feel the cost of generation was completely justified. In many new systems, using highly precise as-built data from recent designs would be more cost effective than trying to generate a lesser quality dataset. I know of other utilities that have used a commercial street centerline data set and rubbersheeted old scanned system maps to fit. In usage, I’ve seen a range of precision levels used quite effectively.

The key is understanding the needs and understanding your data. The results from modeling a system based on lower quality data will give lower quality results. That isn’t bad, it just means that there is a greater margin of error. That may be just fine for the particular application.

In the next post, I’ll address some system design issues.

The Power of 10: Filtering Contours with Map

Property Alterations: Get Elevations from Layer Names

Street views in AutoCAD Map

OK, first off – a disclaimer. I’m blogging about a new software tool – I have no connection whatsoever with Earthmine – I just think it’s a cool new tool for AutoCAD Map users, so I’m sharing.

Thanks to Google Maps, Sketchup, and other visualization tools, there is a stronger interest in seeing things from a 3D and realistic perspective than ever before. There’s a new tool for AutoCAD Map 3D that shows some interesting promise for GIS users. Think of the streetview from Google Maps, and now incorporate that kind of view interactively into your GIS applications. Earthmine combines a new collection process collecting stereo photos as well as point cloud information to create a 3D photographic view of an area. Now take that view, and integrate it with your geospatial data in AutoCAD Map 3D.  You get to see the photo model, with your data right in the model.

I see some exciting applications for asset management – municipalities, utilities and campuses, as well as land developers.

So while you “look” around the street, your data shows up in real locations. So you can see the streetview with your valve or manhole location where it might not otherwise be visible in the photograph.

Or get an idea of the subsurface utilities under the street while looking at the model.

It’s a major step forward in the technology. The data itself is coming from Earthmine servers that either you can host, or have Earthmine host. It appears that you can license model information from their partners, or create your own photo models. Imagine making a 3D view of your new development or campus with your geospatial data superimposed. It brings to mind a number of possibilities. Aside from the tools for viewing the data in AutoCAD Map 3D, Earthmine also has a mobile 3D mapping system so you can have your own photo car to drive around (or pedal). I’m pretty excited to see how this technology advances in the coming months and years. I’m looking forward to getting a closer view of this new tool.

Time for an advantage…

Autodesk has released the new Subscription Advantage Packs for the 2011 products. These Packs give some additional tools to the software users and provide a little extra value to those users on subscription. Last year, we got cloud file capability with AutoCAD Map 3D.  The Packs are downloadable fromt he Scubscription Center, and are now available.

In addition to the added AutoCAD tools which my coworker Isaac Harper blogs about, one of the tools I’m looking forward to most is the new FDO providers. There is a new FDO Provider for ArcGIS allows direct, editable connections to Personal and File Geodatabases as well as SDE connections. These data stores are becoming increasingly popular, particularly as portable GIS data stores. The new FDO Provider opens up more possibilities for managing spatial data without conversion right inside AutoCAD Map 3D (or Civil 3D as well).

Creating AutoCAD Text from Geospatial Datasets

In this tip, I’m going to convert attribute information from a data set in ESRI Shape format, and create a text label from that attribute. There are a couple of techniques, but in this case, I’ll show a quick and easy technique to create AutoCAD text labels from a connected shape dataset using an FDO connection.

Start with a new drawing, assign the coordinate system and create a connection to the data set to label.

Add the data to the map, and the dataset will show up in my Display Manager.

Once the data connection established, create a style and hit the feature label option. This will open the Style Label dialog box where all the settings are to get the labels to look the way you want.

Going down the dialog box, set the Multiline Option (the Advanced Placement option will set the text to follow the line, which you may want for certain conditions, but the text can end up in separate text entities for every letter – that may be ok – it depends on what you are looking for).

The next item is to identify what property (attribute item) you want to use for your label. If you have an attribute that is exactly what you want, you can set it and be done. In some cases, you may want to modify the value or even combine several elements of the text. To add pipe sizes, take the size and add an inch symbol (the double quote) – so an 8” line will read 8”. You could add material so the label reads 12” PVC, or if doing street names, you might want to combine the street number, direction, name and type to get a complete street name. To modify this, select expression here. This will open the Map Expression Builder dialog box.

There are a lot of options for creating text labels from data, calculations, or other elements. In this case, I want to place some text elements together. The function to combine text is concatenate, which I can find under the text functions under Concat.

Selecting the function places the text and format in the expression window. The function works on the elements contained in the following parenthesis, and the bracketed text property are place holders for the text elements. To get the pipe size and combine it with an inch symbol, replace the first Text Property with the value for the size, and the second with the “ symbol (surrounded by single quotes to show it’s a text value – ‘”’). You can continue to add pieces of text until you get the desired label. To add attribute values, such as the size, use the Property menu and the list of attributes will be there sorted by the type of field. I’ll select the size, and complete the expression – Concat ( SIZE , ‘”‘ ). For street names, an expression might be concat( ST_DIR, “ “, ST_NAME, “ “, ST_TYPE) where the “ “ is used to add spaces between the fields.

You can use any combination of data fields and other elements to create a label with this expression process.

Once the expression is set, you can set the display parameters such as size, color and font. Remember with the size, Map Space is scale relative to the model and set the height to a specific value, while device space is relative to the monitor, or view, and will change the physical height based on the zoom scale.

Once the labels are set how we want them, we can go to the Display Manager and select Save Current Map to DWG from the Tools menu.

After saving to a new drawing file, we can open the file and the labels are now standard AutoCAD text items along with the roads. The process actually takes more to write about than to actually do.

Next post, I’ll go the other direction. I’ll show how to grab labels and turn them into attributes.

Getting Your Results – Using Network Topologies

One of the important tools that AutoCAD Map 3D includes is the capability to work with topologies. It can create and manage them, and more importantly, do analysis from these topologies. Basic training classes, and all of the books I’ve seen, show how to create, manage and perform analysis with topologies.  One thing that is absent, is how to take advantage of and access the results after the analysis – it’s the cause of a lot of questions from people I work with.

Before I get on to working with the results, I want to give some background of topologies, and how AutoCAD Map 3D manages them.

Topology is a technique for managing spatial relationships between objects. It is a particularly useful technique for managing networks of objects, such as utilities or streets. The relationships allow you to trace up- or downstream to answer questions, such as:

• What customers will be out of power if this line is down?
• What is the likely source of contaminant (common point upstream) when there are a series of sites that are showing high levels of contaminants?
• What is the shortest route between two locations?

The relationships are created by maintaining unique identifiers for each point or junction and line in data tables. These identifiers are called Primary Keys. The lines will also contain fields to include the identifiers of the points from each end of the line. These are called Foreign Keys. By searching for lines that have a specific point identifier, you can find all the lines that connect to a point, and go from point to associate line to associated point to associated line, etc, much like a monkey swinging from one vine to the next.

Consider the example below.  Here is a collection of lines and points. There is graphic representation of the network, and a snapshot of the data tables from the network. Look at point # 2364.  It appears to be the end of

that particular network stream. We look for all of the lines that have either a start point or end point value equal to 2364, and we find that line # 6888 has 2364 as an end point. We then look at the other point value and find it is 2359 (the start point and end point allow a topology to show direction as well as just connections), so we search the line table for other lines that have that value. We will find line #’s 6887 and 6921. We can then continue in this process until we get the results we are looking for.  For a best route, we will evaluate the potential sets of lines that provide a connection between the two points, and use some criteria to determine the best (usually the shortest lengths) set or route.

AutoCAD Map 3D provides tools and the data structures to create and manage topologies, as well as topology analysis tools. Within the Task-based Workspace, the topology commands can be found on the Object Map ribbon on the Topology panel.   The only thing required to create a network topology is a clean network of lines (clean means that the end of one line exactly meets another line – their coordinates should match). Map 3D will create points (called Nodes) at the junction points if they don’t exist in the data.

Once a network topology is created, Map 3D will create Object Data to manage to topological relationships. The nodes will get a table called TPMNODE_topologyname (where topologyname is the name of the topology). You can use the Edit Object Data command (The Object Data panel of the Object Map ribbon) to select a node and review the table information. The ID field is the critical element (the resistance values can be used to manipulate the analysis – for example, the resistance on a point could be a time factor for an average stop at a stop sign so route can be qualitatively compared including stops, turns, etc).

You can also see the values of the object data by selecting an object and reviewing the Properties (on newer versions of map 3D).

The lines (Map 3D calls them LINKS) will get their own set of Object Data, called TPMLINK_topologyname. The line (or link) object data will include additional fields. The critical elements are the Start_Node and End_Node fields, which are how Map 3D manages the topology relationships.

The purpose of creating and managing topology is to perform analysis, and the tools are also available on the Topology panel of the Object Map ribbon.  The three network analysis tools are Shortest Path, Best Route and Flood Trace.

When completing the analysis, you have the option of highlighting the results with a color (it will go away with the next redraw), or save to a topology. You will give the results a name (my example attached was a flood trace, and I named the resulting topology FloodResults).

Once the topology is created, I can always re-highlight the topology, but it goes away with each redraw. That makes it a bit difficult to use for further analysis or to even print the results.

The easy way to work with them at this point is to save the drawing with the result topology (or topologies) and open a new drawing. In the new drawing, use the Map 3D drawing attach tools to connect to the results drawing.

In the Map Explorer of the Task Pane, the topologies from the attached drawing will be visible in a greyed out icon. This means the topology hasn’t been loaded into the current drawing.

Right-click on the result topology and select the Load Topology option under the Administration menu.

This will open a dialog box with the option to Create objects when loaded. Selecting this will recreate the topology objects in the current drawing, essentially making a copy of the topology data from the original file.

You may have to Zoom Extents to see the newly created data.

At this point, we now have just the objects and topology from the result topology we created. We can save this and have it available for future applications and uses, such as printing route maps or performing buffers, etc. In addition, the objects will have not only the object data from the results topology, but it will also have the data from the original topology and any object data from the original objects.

Try Out a Map Book

On the Task Pane of AutoCAD Map 3D, there is a series of tabs. I spend a lot of time demonstrating and talking about the first two, but I don’t spend much time on the third one – Map Book. The Map Book tab is simply a set of tools designed to help users generate an easily reproducible set of printed maps. It will generate a standard AutoCAD Sheet Set, but will include some tools specifically designed to simplify setting up a grid-based Sheet Set, complete with a key map and navigation features.

You can also have a number of Map Books defined. You may want different books for different utilities, or you may have different scales needed for different map sets. Moving back and forth is just a matter of selecting the appropriate map book at the top of the task pane.

Before creating a new map set, you need to make some decisions. You will need a template drawing file with a layout defined with a title block, and if desired, the adjacent navigation blocks, main, key view and legend viewports. There are a number of templates to use as examples that come with Map. You can find them with the other templates in \Documents and Settings\\Local Settings\Application Data\Autodesk\ AutoCAD Map 3D 2010\R17.2\enu\Templates. These templates may net perfectly suit your specific application, but they can give you some ideas.

You can open one of these and use the Identify Template Placeholders command from the Tools button to look through the settings.

Once you have everything ready, you can use the New button to start a new map book. As with most things Map, you will get a workflow-based dialog box. Go through each section of the dialog before closing the dialog or hitting the generate button.

First, you’ll set a name in the Source. You will select Map Display if you’re plotting objects from the Display Manager (primarily FDO connections) and Model Space if you’re selecting AutoCAD Objects. Remember, you can add Objects from model space into the map display.

Your next step is to select the template drawing file, with the Layout and Title Block you’ve setup. At this point you can add the links to adjacent sheets. You will also set the scale here.

Next, you will set up your map grid using a Tiling Scheme. You can set the grid based on the size of the map (area), by the number of sheets you want, or by selecting a premade grid made from existing closed polylines. You can also set a map overlap, so that you can have a strip of surrounding area. You do that with a percentage.

If you want to create an overlap, and still keep scale maps (I’m not sure who wouldn’t), a friend of mine, Jeremiah McKnelly wrote a great tip for getting true scale map books with an overlap. Check it out here.

The next step is to set a naming scheme for your grid. You have several options based on numbers, letters, or you can even use data from the drawing using the expression builder.

Next you can define your key drawing. You can use premade keys, or generate one on the fly by selecting the layers you want to show.

You can then add a legend where map will setup the viewport if you’ve already created a legend, or you can set up a space to add one later.

Finally, you get to define where you’ll put your sheet set definition file (.DST).

After all this, your Map Book tab will now display all the sheets with their views. You can check the properties of one of the views to see the adhacent sheet information stored. You will also see all of your new layouts, all named and ready to go.

The process allows you to very quickly layout some simple sheet sets with very minimal effort. If your organization needs to plot map books or sets of printed maps, this may be the right tool for

Losing Your Religion: Interoperability with AutoCAD Map 3D and ESRI, Part 2

A little while ago I started this interoperability discussion by discussing the similarities between CAD and GIS, primarily Autodesk’s AutoCAD and ESRI’s ArcGIS. Now I’ll talk about the critical differences between them, or at least the data formats.  These differences are critical to understanding how to manage interoperability.
There are two primary areas of difference.  One is the data structure paradigm, and the other is the graphic representation. AutoCAD drawings and ESRI data sets store data in fundamentally different methods. They are both forms of databases that store information about the location, properties and appearances of the various objects, but because they have substantially different requirements, they have to organize the data differently.
 

Data Structure Paradigm
CAD is used for all types of drawing. The CAD drawing file is essentially an object-oriented database which stores objects sequentially (essentially as they’re drawn). Each row of data will represent an individual CAD primitive object.  The structure of the data and the number of elements is dependent upon the type of primitive. For example, a point is going to carry a single coordinate pair (X,Y) for its location while a line will store two coordinate pairs – a start point and an end point.  A curve will have a start point, an end point, and a bulge (or curve) factor. Along with that, there are additional data elements describing the color, line type, layer and other properties.

An ESRI GIS dataset, whether it is a shape file, geodatabase or personal geodatabase organizes the data into more formal structures, in the form of tables (this is simplified to a conceptual level – each of these data formats include several files or tables to complete the dataset, but are not really germane to the discussion). Different primitives, such as points, lines and polygons can’t reside in the same set of tables. In addition, the number of data elements in each row will be consistent with the dataset. Points representing valves will be in a different table than the lines representing the pipes they’re attached to. The tables will be divided based on some set of business rules to organize the data. In the ESRI terminology, this is essentially a Feature Class. For example, water, storm and sanitary sewer lines may all be in one table, or they may be divided into 3 or more tables. The division may be due to organization, or due to the different information needed for each group. Many times within each Feature Class, there will be a further subdivision of objects, such as high-voltage conductor and low-voltage conductor, called a Subclass. Typically the subclass will be the level of organization used to symbolize the objects. The result is a very structured organization of data.

The analogy that I typically use, and it seems to fit, is that the data sets are like a collection of coins. My AutoCAD file is like a pile of change and my ESRI data file is like the same group of coins all organized into paper tubes.

The take away from all of this is that an AutoCAD drawing will store multiple data types in a single drawing file, while the ESRI data sets will store multiple data types in multiple tables (and/or files). This is a critical point to managing interoperability.

Graphic Representation
The other major area of difference is with the graphic representation. The AutoCAD drawing includes information regarding the appearance of the objects. For example, a line will include the color, line type, and thickness. Each of these properties is inherent in the primitive object. These properties define how AutoCAD will display the file. If I pass the file to someone else, and they open it, it will look the same.
ESRI datasets are a different case. The datasets are not related to the appearance of the data. The appearance is left up to the application at the time of display. ArcGIS, for example, stores the appearance of a map in a Map document, which contains pointers to the data, describing what data to select (allowing a subset of the data through a query) and how to display it. It is the map document that contains the symbolization information, such as linetype, color and stylization.

The Result
These two differences present the primary difference to interoperability. Neither one nor the other is inherently better – they have different ways of achieving similar results.   The data organization presents some challenges when bringing CAD data into GIS tools, for example, when reading an AutoCAD DWG file in ArcGIS, it reads the data as if it were ESRI datasets, and groups objects by their primitive forms, such as lines. That’s different than the way CAD users think of and manage the data. Additionally, CAD drawings typically contain information that doesn’t fit into the GIS data model. The separation of the data from the symbolization is what allows GIS systems to display the same information is many ways depending on the view or analysis needed. This also makes it a little more difficult when transferring data between systems. There is no direct method within AutoCAD to read the Map document to get the shortcuts and symbolization and replicate an ArcGIS map without recreating the symbolization. In most cases this is not really an issue because the data is the important part, although it can be problematic when you want to reproduce the entire map.
There are some other differences between systems that are important to be aware of, such as shape files not dealing with arcs (causing arcs to be broken into many small lines), and single or double precision data differences. These are important to be aware of, but not as critical to the basic interoperability of the systems.

One of the reasons I like working with Autodesk’s AutoCAD Map 3D  product is that it provides me with both worlds. It is an AutoCAD drawing, and with the Feature Data Objects connectors, I can work with the ESRI datasets natively without having to make any changes in the way I work with these disparate data types.

In most cases, there are business issues that interfere with interoperability that have a much greater impact than these technical software elements. I’ll explore those in a future blog.