Processing Multi-Spectral UAS Imagery

Introduction

Methods

Results/Discussion

Conclusion

to be continued...

Processing Pix4D Imagery with GCPs

Introduction

In a previous activity, students used the software program Pix4D to process a set of UAS imagery. This imagery did not contain GCPs. For this exercise, students will process the same imagery using Pix4D, however, this time the imagery will include GCPs. GCPs, or Ground Control Points, are characteristic points whose coordinates are known and are used to georeference a project and reduce the noise. The maps that are generated in this exercise will be qualitatively compared to the maps that were produced in the previous Pix4D exercise.

Methods

The methods for this exercise are very similar to the methods of the Processing Imagery Using Pix4D activity up through Step 8: Processing (Part 1). It is from here that the methods will be more thoroughly documented for this exercise and the steps leading up to this point will simply be skimmed through. 

This exercise uses the same imagery -- Litchfield 1 and Litchfield 2 -- for processing, and the initial setup of the project is identical to that of the previous Pix4D project with the exception that the project is given a different name and saved to a different folder. The steps for the initial setup of the project includes the following:

• Naming the project and selecting a folder to save it in
• Adding images (Litchfield 1 and Litchfield 2)
• Editting the camera model (switching the 'Shutter Model' from 'Global Shutter' to 'Linear Rolling Shutter')
• Selecting the template ('3D Maps' option)
• Unchecking step 2 and 3 in the map view (only step 1. Initial Processing should be checked)

Now it is time to begin working with the GCPs.

Step 1: Open the GCP Manager window

Figure 1: GCP/MTP Manager window

Click on 'Project' in the upper left corner and select 'GCP/MTP Manager...'

This will open the GCP/MTP Manager window (Figure 1)

Step 2: Import the GCPs

Figure 2: Import Ground Control Points window

Click on 'Import GCPs...' (upper right side of figure 1)

Next, referring to figure 2, click on the tab next to 'Coordinates Order' and select the 'Y, X, Z' option

Then, click on 'Browse...' and find the GCP file associated with the Litchfield imagery and select it.

Finally, click 'OK' and if all went well, blue plus signs should appear on the map view in the correct area (these are the GCPs)

Step 3: Run initial processing

Figure 3: Initial processing

Once the GCPs have been imported, initial processing can be started after unchecking steps 2 and 3 in the lower left corner (Figure 3)






Step 4: Edit GCPs using the Ray Cloud Editor

Figure 4: Ray Cloud Editor

After initial processing is complete, the GCPs can now be edited using the Ray Cloud Editor. Click on the 'rayCloud' tab on the left side of the screen and uncheck 'Cameras' under the 'Layers' section (Figure 4)












As can be seen in figure 5 under 'Display Properties', there are 20 GCPs that are available to edit. Start with GCP number '0' by clicking on it which will bring up the 'Selection' and 'Images' tabs on the right side of the screen (figure 6). The GCPs must now be found in each image (the squares with

Figure 6: Selection and Images
tabs prior to editing

Figure 5: GCPs of image

black and white triangles opposite of each other). Under the 'Image' tab, select an image by clicking on it and locate the GCP within the image. Once located, click on the center of the GCP where the white and black triangles meet then under the 'Selection' tab, click on apply. Repeat this process for another image, however this time, after clicking on the center of the GCP, the 'Automatic Marking' button under the 'Selection' tab should become available to select. Click on 'Automatic Marking' and the remaining images under the 'Images' tab should automatically select the GCPs that remain. Check each image to make sure that this has happened. The GCP in each image should have a green 'X' in the center with a yellow circle around the outside of it (figure 7). Repeat this process for the remainding 19 GCPs.

Figure 7: Marked GCP

Step 5: Complete the Processing

Figure 8: Processing steps 2 and 3

After the GCPs have been edited using the Ray Cloud Editor, the remaining processing can be run. Select processing options 2 and 3 in the lower left side of the screen and click 'Start' (figure 8). After processing is complete, the resulting orthomosaic and and DSM should be of high quality which can then be used to make maps in ArcMap.


Results/Discussion

Figure 9: Litchfield Sand Mine with GCPs

Figure 10: Litchfield Sand Mine without GCPs

Comparing the map of the mine with GCPs (Figure 9) to the one created in the previous lab that does not contain GCPs (Figure 10) it is fairly clear to see that the images in figure 9 are sharper than those in figure 10. In both maps, on the left side is a hillshaded DSM with a 30% transparency value overlaid on top of the DSM, and on the right side is an orthomosaic. On the DSMs, green areas represent lower shade while red areas represent higher shade. Figure 9 is much smoother and less "gritty" compared to figure 10. The data quality overall appears to be much better in figure 9.

Figure 11: Quality check with GCPs

Figure 12: Quality check without GCPs

Another way to compare the quality between the imagery is by looking at the 'Quality Check' produced by the Quality Report of each project (figures 11 and 12). The 'Georeferencing' section receives a better check in the report with GCPs compared to the report without GCPs while the 'Camera Optimization' also receives a better score by 1.06%.

Finally, looking at the 'Ground Control Points' section of the report with GCPs, the RMS Error for the X, Y, and Z values are no higher than 0.038 which is very efficient.  

Conclusion

It's clear to see that including Ground Control Points while processing UAS imagery and using the Ray Cloud Editor in Pix4D produces much better results as opposed to not including the GCPs. While the process of including the GCPs and editing them can be somewhat tedious, the extra time and effort of working with them is worth it in the end for the high quality orthomosaics and DSMs that Pix4D produces. 

Using ArcGIS Pro to Engage in Value Added Data Analysis

Introduction

For this exercise, student's accessed on online tutorial made available by ESRI titled Calculate Impervious Surfaces from Spectral Imagery. Using ArcGIS Pro, students will determine the amount of impervious surfaces of a set of parcels in a neighborhood in Louisville, Kentucky. Impervious surfaces are ground surfaces that are impenetrable to water and can present serious environmental issues such as storm water runoff contaminating rivers and lakes. Many local governments will enact fees on land parcels that contain a high amount of impervious surfaces. From this exercise, students will build skills in the following areas:

• Following a workflow with an ArcGIS Pro task
• Performing a supervised classification
• Assessing classification accuracy
• Calculating land-use area per feature

Once the tutorial is complete, students are to create a map with proper elements of the impervious surfaces.

Methods

The online tutorial was broken up into three short lessons:

1. Segment the imagery
2. Classify the imagery
3. Calculate impervious surface area

The outline of the methods for each lesson will be featured below.

Segment the imagery

• Download and open the project: Download data supplied by the local government of Louisville, Kentucky that includes imagery of the study area and land parcel features. Open an ArcGIS Pro task to go through the workflow step by step.
• Extract spectral bands: The multiband imagery of the neighborhood currently uses the natural color band combination. Change the band combination to better distinguish urban and natural features. 
• Segment the image: Group adjacent pixels with similar spectral characteristics into segments which will generalize the image and make it easier to classify. 

Classify the imagery

• Create training samples: Training samples are polygons that represent distinct sample areas of the different land-cover types in the imagery. They signify that segments with certain spectral characteristics should be classified together to represent the same land-use type. The training sample will then be saved as a shapefile.
• Train the classifier: Use the created training samples to create an ESRI classifier definition file (.ecd) which can then be used during the classification to ensure that features are classified accordingly. 
• Classify the imagery: Use the classifier definition file (.ecd) to classify the imagery into the seven land-use classes designated with the creating of the training samples. Then, reclassify that image to have only two classes: pervious and impervious. 

Calculate impervious surface area

• Create accuracy assessment points: Create randomly generated accuracy assessment points throughout the image and then compare the classified value of the image at the location of each point with the actual land-use type of the original image.
• Compute a confusion matrix: A confusion matrix is a table that compares the Classified and GrndTruth attributes of accuracy assessment points and determines the percentage of accuracy between them. 
• Tabulate the area: Determine the area of impervious surfaces within each parcel of land in the neighborhood. First calculate the area and store the results in a stand-alone table; then join the table to the Parcels layer
• Symbolize the parcels: Replace the field names with shorter aliases and symbolize the parcels by impervious surface area to depict the area attribute on the map.

Results/Discussion

Figure 1: Map of impervious surfaces of parcels

Figure 1 illustrates the impervious surfaces that were the result from the tutorial. Yellow surfaces represent the lowest amount of impervious surface area while the red surfaces represent the highest amount of impervious surface area. The parcels are broken up into seven classes where the lowest begins at 0 square feet and the highest ends at 99345 square feet. Many of the road sections are highly impervious, which makes sense since roads are made of of concrete and asphalt. The area with a very low amount of impervious surfaces are those that made up of mostly or all grass/vegetation. However, most parcels fall in between the low and high extremes due to there being houses with driveways (impervious surfaces) that make up most of the parcel while there is only a small amount of yard (pervious surface) within the same parcel. 


Conclusion

Analyzing UAS data in the way that is was in the online tutorial Calculate Impervious Surfaces from Spectral Imagery has the potential to add value to any set of UAS data. By analyzing the imagery of the Louisville, KY neighborhood step by step using the tasks that was set up in ArcGIS Pro, students were able to make a map that illustrated impervious surfaces of parcels that local government officials can then actually use to enact fees on said parcels. This type of approach could also be used for a number of other applications including perhaps comparing the growth of impervious surfaces temporally or analyzing changes in land-use over a certain period of time. 


Sources

Information from the online tutorial: https://learn.arcgis.com/en/projects/calculate-impervious-surfaces-from-spectral-imagery/ 

Processing Imagery Using Pix4D

Overview of the Software

• What is the overlap needed for Pix4D to process imagery?
• A high overlap between images is required in order to get high accuracy results automatically.
• What if the user is flying over sand/snow, or uniform fields?
• Use a high overlap with at least 85% frontal overlap and at least 70% side overlap
• Set exposure settings accordingly to get as much contrast as possible in each image
• What is Rapid Check?
• Rapid check is used in the field to get a quick preview of the outputs
• Can Pix4D process multiple flights? What does the pilot need to maintain if so?
• Yes, Pix4d can process multiple flights, and to do so, the flight height should not vary much between the flights.
• Can Pix4D process oblique images? What type of data do you need if so?
• Yes, Pix4D can process oblique images, and to do so, it is strongly recommended to use GCPs or Manual Tie Points.
• Are GCPs necessary for Pix4D? When are they highly recommended?
• Adding GCPs are optional, but are highly recommended when processing images without image geolocation
• What is the quality report?
• The quality report displays how well the imagery was processed

Walking Through the Software

Professor Hupy provided two data sets for students to process in Pix4D: 'Litchfield Flight 1' and 'Litchfield Flight 2'. The imagery did not contain ground control points (GCPs). 

Processing the Imagery

Step 1: Start a new project

Figure 1: Starting a new project on the home screen

Open Pix4Dmapper and click on 'Projects' > 'New Project...' (Figure 1)

Step 2: Name project and save it in a folder

Figure 2: Naming a project and choosing a folder to save it in

It is recommended that when naming a project to include the following within the name:

• Date
• Site
• Platform/Sensor
• Altitude

Enter the name of the project into the 'Name:' bar and select the folder where it will be saved by clicking the 'Browse...' button to the right of the 'Create In:' bar and locating the correct folder (Figure 2). For this project, it should be saved in the student's personal folder.

Click 'Next'

Step 3: Adding images

Figure 3: Adding images

Add the images that will be used for processing by clicking 'Add Images...', locating the folder(s) containing the images to be used, and selecting the desired images and adding them to the image menu (Figure 3).

Images for this exercise are located in the 'Litchfield Flight 1' and 'Litchfield Flight 2' folders.

At least three images must be selected and must also be in either TIFF or JPG format.

Click 'Next'

Step 4: Image Properties

Figure 4: Image properties 

Once the images are added, Pix4D will check to see if the images are geotagged and what coordinate system is associated with them (most UAS data defaults to WGS 84 decimal degrees)(Figure 4).

Pix4D also detects the camera model and associated info, but these defaults are not to be trusted!

To edit the camera model, click 'Edit...' under the 'Selected Camera Model' section.

Step 5: Edit Camera Model 

Figure 5: Edit camera model

Look under 'Camera Model Parameters' and notice next to 'Shutter Model' it is defaulted as 'Global Shutter or Fast Readout' -- this is incorrect.

Click on the tab next to 'Shutter Model' and select 'Linear Rolling Shutter' as this will be important for processing (Figure 5).

Click 'OK' 

This will lead back to the image properties menu (figure 4). Click 'Next'

Step 6: Select Output Coordinate System

Figure 6: Select output coordinate system 

Leave in defaulted selection (Figure 6). The imagery can be reprojected later in a GIS program if need be. 

Click 'Next'

Step 7: Processing Options Template

Figure 7: Processing options template

Pix4D offers many processing templates to choose from for processing imagery (Figure 7).

For this project, select '3D Maps'

Click 'Finish'

Step 8: Processing (Part 1) 

Figure 8: Processing and the map view screen

The map view will appear on the screen. Uncheck processing steps 2 and 3 right away (Figure 8), then click on 'Processing Options' in the lower left corner

Step 9: Processing Options

Figure 9: Processing options

Click on '3. DSM, Orthomosaic, and Index' and under the 'Raster DSM' section select 'Triangulation' next to 'Method' (Figure 9)

Click 'OK'

This will return to the map view screen 

Step 10: Processing (Part 2)

Figure 10: Processing steps 1, 2, and 3

Click 'Start' and this will begin the Initial Processing. After Initial Processing is completed, a quality report will be generated. If the report is is good, continue on to processing steps 2 and 3 by unchecking step 1, checking steps 2 and 3, and clicking 'Start' once again (Figure 10). Once processing is complete, all steps will become green.

Quality Report

Figure 11: Quality report summary

Figure 12: Overlap section of quality report

The quality report provides extensive feedback on how well the imagery was processed. The report is generated after Initial Processing is complete. The 'Summary' (Figure 11) provides information such as the date and time of processing, camera model name, and the area covered. According to the 'Quality Check' section that appears just after the 'Summary', all 155 images that were selected for processing were utilized, a 100% image calibration. Further down the quality report is the 'Overlap' section which shows the number of overlapping images processed for each pixel of the orthomosaic. Green areas represent high overlap while red and yellow areas represent low overlap and can lead to poorer results in these areas. Looking at the overlap for this project (Figure 12), there appears to be good overlap overall except for some areas on the edge of the processing area, particularly on the south and north edges. This may be due to there not being as many images that can be overlapped with each other at the edges as compared to areas towards the middle of the orthomosaic. 

Fly-Through Animation

The final part to be completed in Pix4D is to create an animation that flies through the orthomosaic. This can be done through the following steps:

• Click on 'rayCloud' on the left side underneath 'Map View'
• Click on the 'New Video Animation' button (the one that looks like a video camera) underneath the 'Create' tab
• This will bring up the 'Video Animation Trajectory Creation' screen; accept the default and click 'Next'
• This will bring up the 'Record Video Animation Trajectory Waypoints' screen; from here select the desired camera angle and orientation and click 'Record Trajectory Waypoint' and keep doing this until all desired waypoints have been added
• Click 'Next' to continue to 'Rendering Options'
• Select the desired length of time of the animation, give it a name, and click 'Finish'
• The animation can be played back and the duration/speed of the animation can be adjusted accordingly; when the animation is suitable, click 'Render' under the 'Video Rendering' section and save it to the appropriate folder 
• The animation for this project can be viewed in the 'Results/Discussion' section below

Results/Discussion

Figure 13: Final maps including hillshaded DSM (left)
and orthomosaic (right)

Pix4D created a DSM and an orthomosaic which could then be used in ArcMap for further analyses and map-making. Figure 13 features the DSM with hillshade on the left side while the orthomosaic is on the right. A hillshade was ran on the DSM, then given a diverging color symbology and set over the top of the DSM with a translucent effect of 35%. Green areas represent low shade while red areas represent high shade. When comparing these areas of extreme low to high shaded relief to the orthomosaic, many of them represent large piles of sand. In the orthomosaic, the grey/golden areas are the mine and the sand itself. A road running north/south on the right side of the image is clearly visible as it turns into the mine in two different places. A aqua blue body of water just barely appears on the northwestern edge of the 

image while a group of thick trees appears in the middle of the southern edge of the image and just barely on the east side of the image as well. 

 Here is the animation created in Pix4D of the processed Litchfield Flight data which shows the 3D imagery very well by capturing the image from multiple angles at different orientations.


Conclusion

Based simply off of this one activity, which just introduces students to the software without getting into "very advanced" territory, Pix4D seems like a pretty easy program to use and it runs and processes smoothly. All of the steps in setting up the project seem pretty straight forward, although if Dr. Hupy hadn't assisted students in a couple of sections with tips (such as changing the 'Raster DSM' 'Method' to 'Triangulation' in 'Processing Options') the project may not have gone as smoothly. If a user does get tripped up, the program offers many resources for assisting those who are in need of help along the way. I was surprised how easy it was to create the animation of the orthomosaic at the end of the project. Overall Pix4D seems like a good program to use for processing UAS imagery.