MEGA data processing errors #52
Description
There are a few errors that pop up when processing MEGA data, not sure if its only for this transducer set-up.
The first is that the horizontal distance of the output processed data is much less than actual. For instance the right bank is ~21m from the boat track in the attached file, where as PyHum is reading it as ~7m. 
. This does not happen if I set the transducer to the incorrect "1199" (instead of "'mega'"), whereby the correct horizontal distance is returned.
Shadow removal does seem to remove anything beyond about 10m from the transducer where as the data was collected out to 30m. Contouring only returns a thin strip at the edge of the data area (see attached).
The second issue is that I get a memory error in the gridding process where by it keeps trying larger and larger resolutions until infinity, regardless of having "99" or "1" in the res setting. I have to hit exit/enter in the CMD terminal to get it to stop. This also happens when I used the (incorrect) 1199 transducer type in the script.
I dont know if its relevant but I get the "memory mapping failed in sliding window - trying memory intensive version" in the initial read module. It seems to continue from this point OK though. Using a Windows 10 machine, i7-8550 with 16GB RAM.
Does anyone know the correct transducer array length for the MEGA units (XM 9 20 MSI T)? It physically measures about 16-18cm but I cant find details about the internal array length.
Script below:
-- coding: utf-8 --
syntax for script without file path included: (pyhum) C:\Users\philw>python bhp_pyhum_pw1.py -i C:\Users\philw\BHPsonar\R00170.DAT -s C:\Users\philw\BHPsonar\R00170\
"""
Spyder Editor
This is a temporary script file.
"""
import sys, getopt
from Tkinter import Tk
from tkFileDialog import askopenfilename, askdirectory
import PyHum
import os
if name == 'main':
argv = sys.argv[1:]
humfile = ''; sonpath = ''
# parse inputs to variables
try:
opts, args = getopt.getopt(argv,"hi:s:")
except getopt.GetoptError:
print 'error'
sys.exit(2)
for opt, arg in opts:
if opt == '-h':
print 'help'
sys.exit()
elif opt in ("-i"):
humfile = arg
elif opt in ("-s"):
sonpath = arg
# prompt user to supply file if no input file given
if not humfile:
print 'An input file is required!!!!!!'
Tk().withdraw() # we don't want a full GUI, so keep the root window from appearing
humfile = askopenfilename(filetypes=[("DAT files","*.DAT")])
# prompt user to supply directory if no input sonpath is given
if not sonpath:
print 'A *.SON directory is required!!!!!!'
Tk().withdraw() # we don't want a full GUI, so keep the root window from appearing
sonpath = askdirectory()
# print given arguments to screen and convert data type where necessary
if humfile:
print 'Input file is %s' % (humfile)
if sonpath:
print 'Son files are in %s' % (sonpath)
doplot = 1 #yes
# reading specific settings
cs2cs_args = "epsg:4326" #WGS84
bedpick = 1 # auto bed pick
c = 1450 # speed of sound fresh water
t = 0.108 # length of transducer for MEGA unit not confirmed
draft = 0.3 # draft in metres
flip_lr = 0 # flip port and starboard
model = 'mega' # humminbird model
calc_bearing = 0 #no
filt_bearing = 0 #no
chunk = 'd100' # distance, 100m
#chunk = 'p1000' # pings, 1000
#chunk = 'h10' # heading deviation, 10 deg
# correction specific settings
maxW = 1000 # rms output wattage for MEGA unit
dofilt = 0 # 1 = apply a phase preserving filter (WARNING!! takes a very long time for large scans)
correct_withwater = 0 # don't retain water column in radiometric correction (1 = retains water column for radiomatric corrections)
ph = 7.0 # acidity on the pH scale
temp = 26.0 # water temperature in degrees Celsius 20 at S4 dike and 27 at Mascarenhas in Oct 2017
salinity = 0.0 #salinity = 0.0
# for shadow removal
shadowmask = 0 #auto shadow removal
win =10
# for mapping
res = 99 # grid resolution in metres
# if res==99, the program will automatically calc res from the spatial res of the scans
mode = 1 # gridding mode (simple nearest neighbour)
#mode = 2 # gridding mode (inverse distance weighted nearest neighbour)
#mode = 3 # gridding mode (gaussian weighted nearest neighbour)
dowrite = 0 #disable writing of point cloud data to file
scalemax = 60 # max color scale value (60 is a good place to start)
## read data in SON files into PyHum memory mapped format (.dat)
PyHum.read(humfile, sonpath, cs2cs_args, c, draft, doplot, t, bedpick, flip_lr, model, calc_bearing, filt_bearing, chunk) #cog
## correct scans and remove water column
PyHum.correct(humfile, sonpath, maxW, doplot, dofilt, correct_withwater, ph, temp, salinity)
## remove acoustic shadows (caused by distal acoustic attenuation or sound hitting shallows or shoreline)
PyHum.rmshadows(humfile, sonpath, win, shadowmask, doplot)
## Calculate texture lengthscale maps using the method of Buscombe et al. (2015)
win = 10
numclasses = 4
PyHum.texture2(humfile, sonpath, win, doplot, numclasses)
## grid and map the scans
cs2cs_args = "epsg:4326"
res = 99
mode = 1
nn = 64
numstdevs = 4
use_uncorrected = 0
scalemax = 60
PyHum.map(humfile, sonpath, cs2cs_args, res, mode, nn, numstdevs, use_uncorrected, scalemax) #dowrite,