gmane.comp.gis.gmt.user

Re: need help using xyz2grd to convert .bil (ned) to grd

Joaquim Luis — 2012-05-24T02:50:33

What happens if you do?


grdinfo NED_74828757.bil=gd




To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

Paul Wessel — 2012-05-24T02:45:51

OK, I think you are selecting -Rs that do not match the original file?  If you wish to extract a subset from these kinds of files you must first create the full file, then use grdcut on the grid.

Paul Wessel, Professor
Dept. of Geology & Geophysics
SOEST, University of Hawaii at Manoa
1680 East-West Rd, Honolulu, HI 96822 USA
1-808-956-4778(phone)/5154(fax)

On May 23, 2012, at 4:38 PM, Vusi Tora wrote:



To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

Vusi Tora — 2012-05-24T02:38:53

I performed another example this time making sure the region was correct
but still got this error:

the size of my file is 79209304  i.e 76MB

xyz2grd NED_58983517\\NED_58983517.bil
-R-91.56439/-89.47077/35.55078/37.00997 -I2c -Gtest.grd -F -V -ZTLhw -N-9999
xyz2grd: Given domain implies x_inc = 0.000555484
xyz2grd: Given domain implies y_inc = 0.000555459
xyz2grd: nx = 3769  ny = 2627
xyz2grd: Working on file NED_58983517\NED_58983517.bil
xyz2grd: More than 9901163 records, only 9901163 was expected (aborting)!


On Wed, May 23, 2012 at 6:56 PM, Vusi Tora <unitmann< at >gmail.com> wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

Paul Wessel — 2012-05-24T01:52:24

So your file is about 208 Mb and the size implied by your -R suggests it should be 1.9 Gb.
Either the file is truncated or compressed or something.  No reasonable -I will give 219146148 exactly.
-p

Paul Wessel, Professor
Dept. of Geology & Geophysics
SOEST, University of Hawaii at Manoa
1680 East-West Rd, Honolulu, HI 96822 USA
1-808-956-4778(phone)/5154(fax)

On May 23, 2012, at 3:35 PM, Vusi Tora wrote:



To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

Vusi Tora — 2012-05-24T01:47:02

I still get an error if I use -I2c instead of half

xyz2grd NED_74828757\\NED_74828757.bil -R-91/-88.5/34/38 -I2c -Gtest.grd -F
-V -ZTLhw -: -N-9999
xyz2grd: nx = 4500  ny = 7200
xyz2grd: Working on file NED_74828757\NED_74828757.bil
xyz2grd: More than 32400000 records, only 32400000 was expected (aborting)!



Is there a way to figure out the exact region covered by the .bil file
before running xyz2grd


On Wed, May 23, 2012 at 8:41 PM, Keith Pickering
<keith.pickering< at >gmail.com>wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

Keith Pickering — 2012-05-24T01:41:30

My understanding is that the NED dataset is available in only 1 arcsec and
1/3 arcsec resolutions; but you're specifying 1/2 arcsec resolution. Based
on the size of your -R region, you should have over 129 million records
even at 1 arcsec resolution. Something's not right there.

Keith Pickering


On Wed, May 23, 2012 at 8:27 PM, J. Luis <jmfluis< at >gmail.com> wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

Vusi Tora — 2012-05-24T01:35:40

Thanks, the size is : 219146148 bytes

On Wed, May 23, 2012 at 7:41 PM, Paul Wessel <pwessel< at >hawaii.edu> wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

J. Luis — 2012-05-24T01:27:19

It probably does not have the ~1 GB that the -R -I imply  but this is 
another example on how simpler life would be if GMT5
GMT5 read those files natively

Joaquim



To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: need help using xyz2grd to convert .bil (ned) to grd

Hongfeng Yang — 2012-05-24T01:13:00

Looks like the number of your data points does not match what you have 
specified.
Make the bound and the increment in xyz2grd exactly the same with these 
of the data and try it again. Hope that helps.

Hongfeng

On 5/23/12 7:56 PM, Vusi Tora wrote:

Re: need help using xyz2grd to convert .bil (ned) to grd

Paul Wessel — 2012-05-24T00:41:17

What is the size of your NED_74828757\NED_74828757.bil in bytes?
-p

Paul Wessel, Professor
Dept. of Geology & Geophysics
SOEST, University of Hawaii at Manoa
1680 East-West Rd, Honolulu, HI 96822 USA
1-808-956-4778(phone)/5154(fax)

On May 23, 2012, at 1:56 PM, Vusi Tora wrote:



To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

need help using xyz2grd to convert .bil (ned) to grd

Vusi Tora — 2012-05-23T23:56:52

Hi all, I need help to convert a bil file downloaded from USGS NED

Here is my command and error message I obtained:

xyz2grd NED_74828757.bil -R-91/-88.5/34/38 -I0.5c -Gtest.grd -F -V -ZTLhw
-N-9999
xyz2grd: nx = 18000  ny = 28800
xyz2grd: Working on file NED_74828757\NED_74828757.bil
xyz2grd: Found 109573074 records, but 518400000 was expected (aborting)!

I don't know what exactly I'm doing wrong.
Thanks in advance

To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: COARDS/CF-1.0 problem

Florian Wobbe — 2012-05-23T22:56:12

Just to clarify: This is not in contradiction with what Remko stated. I too interpret the CF conventions defining valid_range as entirely optional. What I meant here is that the GMT manual says "actual_range (or valid_range) - Minimum and maximum x and y of region". The "(or valid_range)" is wrong and misleading and should be removed from this context.

In fact CF-1.6 suggest not to use valid_range if there is only one missing value:
"NUG defines missing data as all values outside of the valid_range, and specifies how the valid_range should be defined from the _FillValue (which has library specified default values) if it hasn't been explicitly specified. If only one missing value is needed for a variable then we recommend strongly that this value be specified using the _FillValue attribute. Doing this guarantees that the missing value will be recognized by generic applications that follow either the before or after version 2.4 conventions."

So for me the message is:
1. valid_range is optional. there is nothing wrong about using it yourself but you cannot rely that other software writes provides this attribute for you. so you must check _FillValue as well.
2. valid_range is discouraged if you can avoid it in order to increase the compatibility.

I consider this a GMT bug. actual_range should *not* include the _FillValue:

$ ncdump -h g.nc
short z(y, x) ;
z:long_name = "z" ;
z:_FillValue = -999s ;
z:actual_range = -999., 2. ;

$ grdinfo g.nc
g.nc: Command: xyz2grd -I1 -R0/1/0/1 -Z -Gg.nc=ns/1/0/-999 g.z
g.nc: z_min: -999 z_max: 2 name: z

$ grd2xyz g.nc
01NaN
111
000
102

I mixed the links up in my previous mail. It is defined in CDC: http://www.esrl.noaa.gov/psd/data/gridded/conventions/cdc_netcdf_standard.shtml

Like stated already actual_range is just for convenience so that, e.g., grdinfo does not have to recompute the [min,max] range each time.

I didn't say that and in fact I would welcome an extension to grdedit so that the user can edit the nc-header and can add own convenience variables. Maybe even populate them with statistics from grdmath etc.

Florian

To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: calculate the position (longitude, latitude) from the distance of 2 points.

Paul Wessel — 2012-05-23T20:19:49

grdmath computes a grid with distances from your coastline.  To extract the 12 nm contour you would do something like this:

# 1. Compute distances on grid and scale to nautical miles.
 grdmath -R133.50/133.75/35.40/35.65 -I0.01 -fg -V coastlines_s.d LDIST 1.852 DIV = dist_in_nautical_mile_from_s.grd
# 2. Extract the 12 nautical mile contour:
grdcontour dist_in_nautical_mile_from_s.grd -C1 -L11.5/12.5 -D12nm_contour.txt -m > /dev/null

You can now use the 12nm_contour.txt file.  To get a more accurate file, reduce the grid spacing (-I).

Paul


On May 23, 2012, at 3:49 AM, Namba Takaya wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

COARDS/CF-1.0 problem

Frédéric Bouchette — 2012-05-23T17:38:43

Hi all,

Florian + Remko + Joaquim : thank you for these detailed answers. This
list is definitely very active.

Now, I am not sure to agree 100% with all your explanations.

Florian said:
"GMT claims to be COARDS compliant which is probably true although the
documentation says actual_range == valid_range,"

And then Remko wrote:
"CF-1.0 does NOT require valid_range, valid_min, or valid_max. It's
totally optional. In fact, valid_range is generally not very
informative, the data type often tells you already what the
valid_range is,....
I have also not encountered ANY program that actually requires
valid_range, and they shouldn't."

Well. I don't understand the last sentence: valid_range is in the norm
CF-1.0 / 1.6. Why shouldn't we use it ? It can be usefull.

We use valid_range to set the scale of color palettes in the WPS/WFS.
It is convenient.
An alternative method would be to use actual_range and _FillValue set to NANf.
Two examples:
_FillValue set to Nanf -> actual_range IS NOT affected.
_FillValue set to -999 -> actual_rang IS affected ( -999,max_value ) Hmmm...

So the equivalence actual_range / valid_range is speculative and
depends upon another attribute.

Additionally, this means that GMT asks NetCDF users to use a parameter
that IS NOT in the CF-1.0 CF-1.6 convention. Indeed, if you search the
term "actual_range" in :

http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.0/cf-conventions.html
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html

Nothing ----> Actual_range is out of Conventions CF 1.0 / 1.6.

Of course, GMT can use attributes NOT relative to convention (really :-).

But the question is:
is it logical to introduce in GMT a non-normative parametre
(usual_range) and to NOT introduce in GMT a normative parametre that
can do the same job ?

At this point, we can adapt our strategy and introduce actual_range in
the list of attributes to be catched by the WPS/WFS, with the relevant
_FillValue. This is the solution. Thank you again for the comments and
the help.

I well understood that you are convinced that GMT shouldn't be adapted.
But, presently, I am not so confident (see what I mentioned above).

Notice that i well understood the sentence of Remko: "One can also not
simply replace actual_range by valid_range, because the former is in
unpacked units, the later in packed units.". But, to my opinion, this
is a distinct questioning. Perhaps I am wrong. I am definitely not as
easy with all these concepts as you are.

Best regards,
Frederic.










2012/5/23, Joaquim Luis <jluis< at >ualg.pt>:

Re: calculate the position (longitude, latitude) from the distance of 2 points.

Namba Takaya — 2012-05-23T13:49:23

Dear Sir
Thank you very much for your reply.
I am very confused.  What the output result of netcdf file in the GMTMATH with LDIST is?  I thought that the grids of the netcdf file are the grids in which the points  at a distance from the points on the coastal line are. Is it right?    Could you tell me what the "z(lon, lat)" in the output netcdf file means.
Takaya
       

To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: COARDS/CF-1.0 problem ?

Joaquim Luis — 2012-05-23T13:42:32

Hi,

It is now possible to write netCDF CF-1.5 with the current GMT5 SVN via GDAL

grdreformat

usage: grdreformat <ingrid>[=<id>[/<scale>/<offset>[/<nan>]]] 
<outgrid>[=<id>[/<scale>/<offset>[/<nan>]][:<driver>[/<dataType>]]]
...
         When <id>=gd for output it means the grid will be saved using 
the GDAL library.
         Specify <driver> and <dataType>. Driver names are as in GDAL 
(e.g netCDF, GTiFF, etc).
<dataType> is u8|i8|u16|i16|u32|i32|float32; i|u stand for 
signed|unsigned integers.
         Both driver names and data types are case insensitive [Default 
is netCDF/float32].

As an example do:

grdmath -Rd -I10 X Y MUL = lixo_f32.nc=gd:netCDF

grdmath -Rd -I10 X Y MUL = lixo_u8.nc=gd:netCDF/u8

Now if we look into the header with ncdump we see that there is no 
valid_range nor actual_range. I gree with Remko that it's really bad not 
having the actual_range stored in the file because it will oblige to 
compute it every time one need to report the basic file statistics


ncdump -h lixo_f32.nc
netcdf lixo_f32 {
dimensions:
         lon = 37 ;
         lat = 19 ;
variables:
         double lat(lat) ;
...
         double lon(lon) ;
...
         float Band1(lat, lon) ;
                 Band1:long_name = "GDAL Band Number 1" ;
                 Band1:_FillValue = NaNf ;

// global attributes:
                 :Conventions = "CF-1.5" ;
                 :GDAL = "GDAL 1.9.0, released 2011/12/29" ;
                 :history = "grdmath -Rd -I10 X Y MUL = 
lixo_f32.nc=gd:netCDF" ;


However, things are different for the unsigned int file

ncdump -h lixo_u8.nc
netcdf lixo_u8 {
dimensions:
         lon = 37 ;
         lat = 19 ;
variables:
         double lat(lat) ;
...
         double lon(lon) ;
...
         byte Band1(lat, lon) ;
                 Band1:long_name = "GDAL Band Number 1" ;
                 Band1:_Unsigned = "true" ;
                 Band1:valid_range = 0s, 255s ;
                 Band1:_FillValue = 0b ;

Note that now we have a valid_range and a _Usigned attribute. I think 
here the "valid_range" attribute is very important e very likely related 
with this sentence of the CF-1.5 conventions doc 
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html

"It is possible to treat the|byte|type as unsigned by using the NUG 
convention of indicating the unsigned range using 
the|valid_min|,|valid_max|, or|valid_range|attributes."

And indeed GDAL says "lixo_u8.nc" is unsigned but GMT tells us that it 
is signed

gdalinfo lixo_u8.nc -stats
....
     STATISTICS_MAXIMUM=252
     STATISTICS_MEAN=128
     STATISTICS_MINIMUM=4
     STATISTICS_STDDEV=70.437982892487
     valid_range={0,255}

grdinfo lixo_u8.nc -L
...
lixo_u8.nc: z_min: -128 z_max: 124 name: GDAL Band Number 1
lixo_u8.nc: scale_factor: 1 add_offset: 0
lixo_u8.nc: 67 nodes set to NaN
lixo_u8.nc: mean: -4.0251572327 stdev: 75.5499271482 rms: 75.5977438018

So, "valid_range" though optional, matters.

Joaquim



To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: calculate the position (longitude, latitude) from the distance of 2 points.

Walter (HF) Smith — 2012-05-23T13:29:58

You are using grdmath. The -I sets how closely spaced the points are in your output grid. The LDIST operator is computing the distance. It is not clearly documented how to set the distance unit to be in nautical miles. If the distance unit is something else an additional multiplication could be put on the stack in grdmath to get it into nautical miles.



On May 23, 2012, at 9:19 AM, Namba Takaya wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: calculate the position (longitude, latitude) from the distance of 2 points.

Namba Takaya — 2012-05-23T13:19:01

Dear Sir
Thank you very much for your quick response.
I am not clear of -I option of GMTMATH. I thought -I12n means 12 nautical miles from the point on the coastal line, which I wish to calculate.If I choose a smaller value for -i in grdmath, which option I should use for 12 nautical miles in GMTMATH?

takaya
 

       

To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: calculate the position (longitude, latitude) from the distance of 2 points.

Walter (HF) Smith — 2012-05-23T12:57:29

looks like you want a smaller value for -I in grdmath

w

On May 23, 2012, at 6:25 AM, Namba Takaya wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: calculate the position (longitude, latitude) from the distance of 2 points.

Namba Takaya — 2012-05-23T12:56:51

Dear Sir
Thank you very much for your help.
I have an additional question on this topic.
I executed the script below for the 12 nautical miles from the coastal line.What does  the "z(lon,lat)" mean in the output file,"dist_in_nautical_mile_from_s". 

pscoast -Jm -R133.50/133.75/35.40/35.65 -M -Dc -W1 > coastlines_s.dgrdmath -R133.50/133.75/35.40/35.65 -I12n -fg -V coastlines_s.d LDIST = dist_in_nautical_mile_from_s.grdnetcdf dist_in_nautical_mile_from_s {dimensions:        lon = 2 ;        lat = 2 ;variables:        double lon(lon) ;                lon:long_name = "longitude" ;                lon:units = "degrees_east" ;                lon:actual_range = 133.5, 133.75 ;        double lat(lat) ;                lat:long_name = "latitude" ;                lat:units = "degrees_north" ;                lat:actual_range = 35.4, 35.65 ;        float z(lat, lon) ;                z:long_name = "z" ;                z:_FillValue = NaNf ;                z:actual_range = 2.71491599082947, 25.0093059539795 ;
Date: Tue, 22 May 2012 08:15:23 -1000
From: pwessel< at >hawaii.edu
Subject: Re: [GMT-HELP] calculate the position (longitude, latitude) from the distance of 2 points.
To: GMT-HELP< at >lists.hawaii.edu

If you have a suitable coastline then you can use grdmath's LDIST operator to create a grid with distances from this line; the 12 mile contour of this grid is then your desired line.  The finer the grid the more accurate the curve and the slower it is to compute.

Paul Wessel, ProfessorDept. of Geology & GeophysicsSOEST, University of Hawaii at Manoa1680 East-West Rd, Honolulu, HI 96822 USA+1-808-956-4778/pwessel< at >hawaii.edu

On May 21, 2012, at 9:06 PM, Namba Takaya wrote:Dear Keith
Thank you very much for your quick reply.
I wish to calculate the territorial sea of 12 miles from the costal line. So two position, A and B are on the coastal line. So I wish to calculate the location of the point, C in the sea not in the land side.
Thanks in advance.
Takaya
Date: Tue, 22 May 2012 01:55:44 -0500
From: keith.pickering< at >GMAIL.COM
Subject: Re: [GMT-HELP] calculate the position (longitude, latitude) from the distance of 2 points.
To: GMT-HELP< at >lists.hawaii.edu

In nearly all cases, there are two such points at x distance from two given points. You would need a third given point to resolve the ambiguity.
Keith Pickering

On Tue, May 22, 2012 at 1:32 AM, Namba Takaya <takayanamba< at >hotmail.com> wrote:
Dear Sir
I would like to calculate the location of the point (longitude, latitude) from 2 points by a certain distance (for example 12 miles).
For example, there are two points, A(longitude=E125.25, latitude=N25.25) and B(longitude=E125.30, latitude=N25.20).I would like to calculate the location of the point, C, at the distance of 12 miles form A and B.
 Thank you in advance.
Takaya NambaTo unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu
To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.eduTo unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu
To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu       

To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu

Re: COARDS/CF-1.0 problem ?

Remko Scharroo — 2012-05-23T12:28:46

Dear Frédéric,

CF-1.0 does NOT require valid_range, valid_min, or valid_max. It's totally optional. In fact, valid_range is generally not very informative, the data type often tells you already what the valid_range is, e.g.:
type byte:
_FillValue = -128;
valid_range = -127, 127;
On the contrary, actual_range actually DOES add information, indicating the minimum and maximum values in the grid.
One can also not simply replace actual_range by valid_range, because the former is in "unpacked" units, the later in "packed" units.

So there is no need to change GMT for this. I have also not encountered ANY program that actually requires valid_range, and they shouldn't.

Remko


On 23 May 2012, at 12:58, Florian Wobbe wrote:


To unsubscribe, send the message "signoff gmt-help" to listserv< at >lists.hawaii.edu