Lab 3 Part 1: Processing LiDAR Data

Due by Oct 28

Introduction

This lab has four separate tasks that will introduce you to the process of converting raw LiDAR returns to useable GIS data. Tasks 1 through 3 discuss the basic steps of processing LiDAR data in ArcGIS. Task 4 deals with 3D visualization of LiDAR data.

Spend some exploring the functionality introduced in this lab – it’s the best way to get acquainted with both the software and the data. Use the help system to learn about the different options. If you have a question, ask.

Type the answers to all the questions at the end of each task, attach any documents and maps that are requested.

The files used in Lab 3 (both Parts 1 & 2) are in the I:\Students\Instructors\Geoffrey_Duh\GEOG4593\Lab3 folder. Please copy lab data to your local workspace before you start the lab.

 

Task 1 – explore raw LiDAR las file

Instructions

The first task is to extract information and point data from a “raw” LiDAR las file. You will use several ArcGIS tools to complete this task. The las file (45122D7103.las) was derived from a 2007 LiDAR mission for the Portland metropolitan area by the Oregon Lidar Consortium. The size of the las file is around 370 MB and covers an area of about 0.56 square miles (145 hectares) near the Tualatin Hill Nature Park in Beaverton. The data were delivered to the Consortium in the Oregon HARN State Plane projection. The linear and elevation (Z) units are both in feet. The las file format is LAS Version 1.0.

1.      Extract the header information of LAS file. Use ArcGIS 10's search window to search information about "las." Spend a few minutes to read the information about the tools you find. Start the Point File Information tool. Make sure to browse for "Files" and select the 45122D7103.las as the input. If you have multiple las files, you can select them all as individual files. If you have lots of las files, which usually is the case, then you can put them in folders and select them their containing folders using the "Folders" option. Use 45122D7103_PointFileInfo.shp as the output feature class name and save it to your workspace. Make sure the input file format is set to "LAS." Click OK to accept the default values of other options and finish the extraction of point file information. Examine the attribute table of the output shapefile.

2.      Convert the las point data to GIS format. Start the LAS to Multipoint tool. Set 45122D7103.las as the input file. Specify your output file as 45122D7103_FirstMPoints.shp. Use a value of 1 as the average point spacing and check 1 as the return value (uncheck all other values). Specify NAD 1983 HARN StatePlane Oregon North FIPS 3601 (Intl Feet) as the input coordinate system. Click OK to accept the default values of other options and finish the conversion. Depending on the hardware configuration, this process might take a long time to finish. Repeat the same procedures to create a shapefile of the Lidar last return data and name it 45122D7103_LastMPoints.shp. This time, specify "Last Return" as the return value. Spend a few minutes to examine the maps and attribute tables of the shapefiles you created.

3.      Visualize the Lidar first and last return data in ArcScene. Start ArcScene and add both 45122D7103_FirstMPoints.shp and 45122D7103_LastMPoints.shp to the scene. You will see the LiDAR point clouds of the first and last return layers.

4.      Convert multipoints to single point features. Start the Multipart to Singlepart tool. Set 45122D7103_LastMPoints.shp as the input feature layer and 45122D7103_Last.shp as the output feature class. Click OK to start the conversion. Depending on the hardware configuration, this process might take a VERY VERY long... time to finish.

5.      Additional visualization tasks (optional). You can use the Add XY Coordinates tool to add the Z values (as well as the X and Y coordinates) of the LiDAR points to the singlepart shapefile's attribute table. You can then use the Z values to symbolize the point data in ArcScene and create a more informative visual rendering of the LiDAR point data.

Questions

1. Based on the attribute information of the output of the Point File Information Tool, what is the point density (i.e., points per square feet) of the LAS file? What is the point spacing (in feet)? Use your own calculation to calculation the point spacing and show your equation and result.

2. How many points are there in the 45122D7103_Last.shp? Based on the information, what might be the highest DEM cell resolution (in feet) if you want to convert the LiDAR data to a DEM? Please explain.

Task 2 – import preprocessed LiDAR data

Instructions

This task is to import a preprocessed (i.e., filtered) LiDAR data points into a GIS dataset that you can manipulate in your GIS application (in this case, ArcGIS). The goal of this process is to generate digital elevation models stored as raster grids. The dataset used in this task covers a different area from the dataset used in Task 1. Also, the data have been converted to x, y, z values stored in Dbase dbf format. This is an example of how you would import LiDAR points into ArcGIS.

1. Add the “LiDAR_Returns_Bare_Earth_0503.dbf” file to ArcMap and view its content. These are the “bare earth” (surface) LiDAR returns for a small area of Portland. The table contains the X/Y/Z coordinates of the returns in Stateplane feet. The X attribute is X_COORD, the Y attribute is Y_COORD, and the Z value (or elevation) is Z_VALUE.

2. Open ArcMap. Go to File/Add data – Add XY data. Browse to the bare earth dbf. Make sure the specified X Field, Y Field and Z Field are correct. You will also need to define the projection of the data. Click the Edit button, click Select to select a predefined coordinate system, and browse to the correct “projected” coordinate system. Here’s the details:

 

Stateplane projection: NAD 1983 HARN (Feet, Intl and US), Oregon North FIPS Zone 3601, International feet

 

3. Click the OK button. The points will be added to the map (this may take a few seconds). Turn on the layer (it will take a minute or two to draw). This is an ArcGIS “event table” layer. Zoom in to the layer and use the Identify tool to click on a few points. An event table layer is a temporary, non-permanent representation of geographic data from coordinates defined in a table. It will persist with the MXD document (meaning if you save the ArcMap document you will save the layer in it). However, it is tied to the original DBASE table, so you cannot change the location of or remove the original table. You can create permanent GIS data from an event table layer by right-clicking the layer and selecting Data – Export Data. You don’t need to use permanent point layers for this task, but you will use them in a later task. Follow the steps above to create permanent layer files for both LiDAR dataset.

4. Repeat steps 1 through 3 for the “first return” LiDAR coordinates in the “LiDAR_Returns_First_Return_0503.dbf” in the same directory as the bare earth returns. Note: there are more points in this dataset – it will take longer to draw.

Questions

1. How many points are in the “first return” dataset? What is the estimated average number of points per square meter? (Hint: The area is rectangular. Be aware of your units.)

2. Explain some possible reasons that there are areas in the bare earth dataset that have no returns (Hint: Use the aerial photos in the 2005_Aerials to find clues).

3. Zoom in to a relatively small area and compare the bare earth returns with the first returns. Many of the bare earth returns are also in the first return dataset. Why?

 

Task 3 – create a TIN and DEM from the LiDAR returns

Instructions

The second task is to convert the LiDAR returns into a triangulate irregular network (TIN) model and then into a raster “digital elevation model” (DEM) of the surface using the bare earth LiDAR returns. The DEM is usually the final representation of the surface derived from LiDAR points. The raster format is a much more efficient way of storing the large, complex elevation models derived from LiDAR data (the TIN is most often discarded after the DEM is created). Again, there are many ways to do this, and some offer you much more control over the resulting triangulation than others. This is an example of how you would create a TIN and DEM with the standard ArcGIS 3D Analyst Tools. Note that the next few tasks will be performed on the bare earth returns only.

1. Open ArcMap if is not already open. Make sure the 3D Analyst extension is turned on.

2. Turn on the 3DAnalyst toolbar.

3. Start the Create TIN tool. Set the “LiDAR_Returns_Bare_Earth_0503 Events” as the inpute feature class.

4. Set the Height source to Z_Value. The Z_Value is the elevation of each point above-sea-level. We are creating this TIN from “mass points”. Leave the Tag value field as “<None>”.

5. Specify a file location and name for the output TIN. Because the output is in ArcInfo coverage format, the name must be 13 characters or less. You may want to indicate that this is the bare earth TIN in the file name (e.g., tin_be). Set the spatial reference the same as the input LiDAR data.

6. Click OK to create the TIN. This WILL take a while. Zoom in to a smaller area and explore the model (smaller areas draw much faster). Right click on the layer and select Properties. Go to the Symbology tab, and press the Add button. Add a few different renderers to explore the different TIN symbolization options.

7. Start the TIN to Raster tool. Specify the bare earth TIN you created as the Input TIN. Set output data type as FLOAT, method as LINEAR, and sampling distance to "CELLSIZE 5." Leave Z Factor as 1 (this is the amount of vertical exaggeration). Specify a file location and name for the output raster image. Because the output is in ArcInfo GRID format, the name must be 13 characters or less. You may want to indicate that this is the bare earth DEM in the file name (e.g., dem_be).

8. Click OK to create the DEM. When complete, zoom out to the full extent of the DEM. Right click on the layer and select Properties. Go to the Symbology tab, and explore the different symbolization and classification options.

9. Repeat steps in Tasks 2 and 3 to convert the LiDAR_Returns_First_Return _0503.dbf to a DSM (e.g., dsm_fr).

Questions

1. What are the maximum and minimum elevation values in the DEM? Why are they different from the TIN model?

2. You specified a cell size of “5” when creating the DEM and DSM. What does this value represent? Why is “5” an appropriate value for this LiDAR data? (Hint: You need to know the point spacing values for the first and last return data.)

3. Visually compare the bare earth DEM and the first return DSM. What is the height of the “domed” structure just west of the storage tanks?

 

Task 4 – visualizing the data in 3D

Instructions

This task is to visualize the data you have created so far in 3D. We will use ArcGIS’s ArcScene for this task. It’s not the best 3D visualization software, but it (usually) gets the job done.

1. Save your MXD and close ArcMap.

2. Open ArcScene (in the Windows Start menu under ArcGIS).

3. Add the bare-earth DEM created in Task 2.

4. Right-click on the layer, select Properties, then select the Base Heights tab. Click on the Floating on a custom surface option, and select the DEM from the drop-down menu. This will render the DEM in 3D using the elevation values from the raster pixels.

5. Click Raster Resolution and enter 10 into Cellsize X and Cellsize Y. This is the resolution to which ArcScene will generalize the DEM (to make it draw faster). If you wanted the DEM to render at its full resolution, you would enter 5 for each value.

6. Click on the Rendering tab. Under Effects, check on Shade areal features relative to scene’s light position. Also, drag the Quality enhancement slider bar a bit to the right, so that the slider sits about 2/3 of the way to “high” (to improve the quality of the rendering).

7. Click the Symbology tab. Select a more colorful Color Ramp.

8. Click OK.

9. Repeat steps 1 through 8 with the first return DSM.

Questions

1. Note that the model has some irregularities on the southern edge. What are some possible ways to remove these kinds of edge issues when generating an elevation model?

2. Play around with the Raster Resolution layer property (in the Base Heights tab). Set it to the maximum resolution (5). For which model does it make the most difference visually? Why?

3. Where is this area in Portland? (Hint: Use RLIS data on the I drive to figure it out.)

Export a view of the full first return DSM from ArcScene (File – Export Scene – 2D). Print out a copy and turn it in with your lab.

Optional Task – creating a Terrain dataset in ArcGIS

Instructions

This task is for those who are interested in using the Terrain dataset in ArcGIS. Terrain dataset is a new data organization tool in ArcGIS for terrain data. It allows users to put all terrain related data such as, Lidar mass points, GPS points, breaklines, polygons, contours, etc, of a project in one place, i.e., a terrain dataset. A terrain dataset can only be created within a feature dataset. That means all the terrain data have to be in the same projection. Use Arccatalog to complete the following steps and examine the final product in ArcScene.

1.      Create a file geodatabase (GDB) in your workspace.

2.      Create a new feature dataset in the GDB, specify the Oregon HARN Stateplane that you used in Task 1 as the projection.

3.      To reduce the process time for this optional task, you can use the Subset_Polygon shapefile to select (i.e., select by location) the LiDAR mass points from the bare earth and the first return XY data and export the data points to the feature dataset as featureclasses.

4.      Import the Contour_55.shp to the feature dataset. This polyline featureclass will be used as a breakline layer. Use 3D Analyst to convert this featureclass into a 3D featureclass, so that the nodes and vertices of the polylines have elevation data (i.e., 55 feet).

5.      You could convert the LiDAR mass point featureclasses you imported into 3D featureclasses, so that you can view them in 3D in ArcScene. This step is not required for the completion of this task.

6.      Now you can use the Bare Earth points and the contour layer to create a Bare_Earth terrain dataset. Right-click the feature dataset and select New – Terrain. Type a name for the Terrain dataset and select the LiDAR Bare Earth and the contour layers to participate in the terrain. Type 5 as the average distance between points and click Next. In the next dialog, set the Height Source of the Bare Earth layer to Z_VALUE and set the SFType of the contour as soft line. Click Next, accept the default pyramid type and click Next. In the next dialog, click Add to add the first window size. Then click Next to review the setup. Use the back button to change the setting or click Finish to complete.

7.      You can use the tools in the Terrain toolset in the 3D Analyst toolbox to modify the terrain dataset. Once the terrain is modified, use the Build Terrain tool to update the terrain.

8.      Use “terrain” as the keyword to search ArcMap and see what tools are available for processing terrain datasets.