make your own NSW topo maps [large images]

NSW & ACT specific bushwalking discussion.
Forum rules
NSW & ACT specific bushwalking discussion. Please avoid publishing details of access to sensitive areas with no tracks.

make your own NSW topo maps [large images]

Postby mholling » Sun 19 Feb, 2012 4:46 pm

Hi folks,

I don't frequent these forums but thought some of you might be interested in a script I've come up with for making high-resolution topographic maps using the data from the NSW geospatial portal (http://gsp.maps.nsw.gov.au).

You can create a custom map for your area of interest at a resolution and scale you specify (typically 300 pixels per inch, 1:25000 or 1:50000). The output is in any image format you specify, including georeferencing for use with OziExplorer/Google Earth/GIS software.

A bit of computer savvy is required, i.e. command-line familiarity and the installation of some (free) pre-requisite software packages. If you can manage that, the result is some pretty nice maps as shown below (click to see full size). They come up really nicely on a good colour laser printer or the like.

The code, and detailed instructions for using it, are at https://github.com/mholling/nswtopo

Hope some of you find it useful!

-- Matthew

25k-sample.png
1:25000 map sample


50k-sample.png
1:50000 map sample
mholling
Nothofagus gunnii
Nothofagus gunnii
 
Posts: 21
Joined: Sun 25 Dec, 2011 4:01 pm
Region: Australian Capital Territory
Gender: Male

Re: make your own NSW topo maps [large images]

Postby Webguy » Tue 21 Feb, 2012 6:48 pm

So, how hard to dump a single All of NSW image file suitable for Ozi? The Maps look good.
User avatar
Webguy
Athrotaxis cupressoides
Athrotaxis cupressoides
 
Posts: 144
Joined: Sun 22 Jan, 2012 10:49 pm
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby flatfoot » Tue 21 Feb, 2012 8:18 pm

Wow!!!! Great stuff! I particularly like the support you have added for magnetic declination lines. I'll see if I can get this going on my Mac.
Flat-footed Mainlander
User avatar
flatfoot
Phyllocladus aspleniifolius
Phyllocladus aspleniifolius
 
Posts: 572
Joined: Wed 13 Jan, 2010 9:27 pm
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Wed 22 Feb, 2012 2:16 pm

The maps look great. I'll definitely be checking it out when I have time. I've tried all sorts of things to get half decent topo data layers on my computer, and this might be the best way yet.
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby colinm » Sat 25 Feb, 2012 9:02 am

Tried it out, the map it generated looks *fantastic* (I've only tried urban Sydney maps so far.) Had some fun getting a recent enough version of ImageMagick, but still - kudos!

How does this even work? Is LPI are providing you vector data, or are they just permitting you to distinctly select each layer as georeferenced images, so you can stitch it together?

I've been told, BTW, that LPI are going to start releasing the vector data.
sig pending approval
User avatar
colinm
Athrotaxis cupressoides
Athrotaxis cupressoides
 
Posts: 389
Joined: Wed 27 Jul, 2011 10:39 am
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby michael_p » Sat 25 Feb, 2012 12:50 pm

Matthew,

Great idea and well executed. Have you seen Festy's maps for Oziexplorer: http://maps.festy.org/.

Each map that Festy has created comes with an Oziexplorer map file. I am not a programmer so I am not sure if this would work. Do you think it would be possible to extract the map extents data from the Oziexplorer map files and use it with your script?

Cheers,
Michael.
One foot in front of the other.
User avatar
michael_p
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1376
Joined: Sun 15 Nov, 2009 6:58 pm
Location: Macarthur Region of Sydney.
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby mholling » Sat 25 Feb, 2012 4:59 pm

The data comes from the NSW geospatial portal (http://gsp.maps.nsw.gov.au), which is an ArcIMS server, so no vector data, just rasters. I have a copy of Festy's map set, which is derived from the same server, but just uses all the default settings and looks a bit so-so for printing. (It's also a latitude/longitude projection, which isn't ideal as it yields non-square grids.)

My script just uses custom ArcXML requests instead of the default ones, in order to get the nice hi-resolution rendering in the correct projection. It downloads tiles for each of the various layers, then just mosaics and georeferences them.

It would be much better if LPI served up their topographic data in vector form, as you could then render it any way you wanted, assuming you had the right GIS software. I also believe government mapping data should be freely available to the public, so it would be great if that were to eventuate.

As it happens, I've had my home server pulling down rasters for all the NSW 25k and 50k map sheets; it should be a much better version of the Festy map set when it's done. (Also a lot more gigabytes...)
mholling
Nothofagus gunnii
Nothofagus gunnii
 
Posts: 21
Joined: Sun 25 Dec, 2011 4:01 pm
Region: Australian Capital Territory
Gender: Male

Re: make your own NSW topo maps [large images]

Postby colinm » Sun 26 Feb, 2012 10:30 am

Well spotted, and well done. I didn't realise they were running a full raster feed (I bet some of the LPI/LPMA poohbahs didn't either, or they'd have put a stop to it. A big 'Hi There!' to the LPI/LPMA people who read bushwalk.com :)

I completely agree about public mapping information. It should be public. I believe the policy now is that vector data should be available 'at cost.' The question I want resolved is whether the gatekeepers will provide at marginal cost or whether they'll try to recover costs back to Surveyor Mitchell + interest.

In the USA they've successfully used the FOI legislation to access mapping data. I imagine we could use GIPA (http://www.legislation.nsw.gov.au/maint ... 009+cd+0+N) the same way, if it comes to that.
sig pending approval
User avatar
colinm
Athrotaxis cupressoides
Athrotaxis cupressoides
 
Posts: 389
Joined: Wed 27 Jul, 2011 10:39 am
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Mon 27 Feb, 2012 10:27 pm

A couple of things that might help other people get up and running:
1. "Download by clicking the 'ZIP' button, or simply copying and pasting the script out of your browser."
- I copied and pasted the script, but that results in losing the indentation, which in turn causes errors in the YAML config in the program. I suggest you don't copy/paste, and just download the ZIP file
2. Under Windows 7, I found I had to add an environment variable for GDAL_DATA. This is the path to gsc.csv, in my case "C:\Program Files (x86)\maps\FWTools2.4.7\data\". Otherwise I got errors

Still having some problems with geotifcp crashing on my computer, but that's probably nothing to do with the script! I still got a test map that looks great - in PNG only.
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby colinm » Tue 28 Feb, 2012 9:04 am

I think convert (even latest version) generates a tiff that gdal doesn't like/accept, causing the crash. I can't suggest an alternative to convert/imagemagick for that step yet.
sig pending approval
User avatar
colinm
Athrotaxis cupressoides
Athrotaxis cupressoides
 
Posts: 389
Joined: Wed 27 Jul, 2011 10:39 am
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby Rob A » Tue 06 Mar, 2012 10:27 pm

Worthwhile? I know what petrol costs me to get to the foot of a walk, let alone provisons and the cost of my time, and I know I get at least two uses out of the paper map which has no errors. MGRS.
Every four seconds, somewhere in the world, an Harlequin Mills and Boon is sold ... Wot ...
Rob A
Phyllocladus aspleniifolius
Phyllocladus aspleniifolius
 
Posts: 506
Joined: Mon 29 Nov, 2010 2:01 pm
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby colinm » Fri 09 Mar, 2012 2:50 pm

It's not just about the cost. This program makes it possible to produce custom topos - you can project tracks over the topo instead of drawing them on with pencil, for example.
sig pending approval
User avatar
colinm
Athrotaxis cupressoides
Athrotaxis cupressoides
 
Posts: 389
Joined: Wed 27 Jul, 2011 10:39 am
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Fri 09 Mar, 2012 9:06 pm

Rob A wrote:Worthwhile? I know what petrol costs me to get to the foot of a walk, let alone provisons and the cost of my time, and I know I get at least two uses out of the paper map which has no errors. MGRS.


No errors?! Which map was that?
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby wildwalks » Sat 10 Mar, 2012 7:16 pm

tom_brennan wrote:No errors?! Which map was that?


:)

Matt :)
wildwalks
Magnus administratio
Magnus administratio
 
Posts: 946
Joined: Mon 22 Nov, 2010 4:35 pm
ASSOCIATED ORGANISATIONS: Wildwalks, Bushwalk.com & NPA NSW
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Sun 11 Mar, 2012 9:45 am

tom_brennan wrote:No errors?! Which map was that?


More seriously, I haven't found any maps that aren't littered with errors or missing data. Things like foot tracks seem to be rarely mapped correctly in my experience. And as anyone who has walked in the Northern Bluies can tell you, there's a heck of a lot more sizeable cliffs than you'll find on the topo.

I note that Matthew has released a new version of his tool - https://github.com/mholling/nswtopo
It adds support for GPX & KML based layers, which is handy for overlaying tracks or other data. I haven't quite got it all working, but I can produce the layer and overlay it manually in Photoshop. I wonder if it is possible to add patterns to tracks (sort of like__ __ __ __ or -------- or .......)
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby wildwalks » Sun 11 Mar, 2012 7:08 pm

tom_brennan wrote:I haven't found any maps that aren't littered with errors or missing data.

DItto

But interestingly over the last 6months or so I have noticed a great improvement in LPMA data appearing on their map server. Topography is still not as good as it could be but the walking tracks are getting better in some areas. I think we will see some big improvements in topography as they start to roll out the data there are collecting through their new fancy lidar system which is slowly scanning the state. maybe even cliffs and canyons will become better marked.

Matt :)
wildwalks
Magnus administratio
Magnus administratio
 
Posts: 946
Joined: Mon 22 Nov, 2010 4:35 pm
ASSOCIATED ORGANISATIONS: Wildwalks, Bushwalk.com & NPA NSW
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby colinm » Sun 11 Mar, 2012 9:17 pm

I understand that LPMA (as part of their push to get digital vector data out there) are going to be soliciting improvements of the data on-the-ground. So (as it has been represented to me) you'll be able to inform them when they're missing a cliff, or a track on the map isn't there.

That's gotta be a good thing (oh, and Yay LIDAR! They're talking about amazingly precise resolution.)
sig pending approval
User avatar
colinm
Athrotaxis cupressoides
Athrotaxis cupressoides
 
Posts: 389
Joined: Wed 27 Jul, 2011 10:39 am
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby wildwalks » Mon 12 Mar, 2012 6:31 am

colinm wrote:soliciting improvements of the data on-the-ground. So (as it has been represented to me) you'll be able to inform them when they're missing a cliff, or a track on the map isn't there.

It will require a bit of a cultural change - I have submitted some changes that took 4 years to be updated - as you can imagine I stopped giving them changes about 3.5 years ago when I saw a lack of action. I suspect they can see that OSM is starting to do a better job on roads and major features, and have just thought 'we will use the taking part of the model, but not the sharing part'. Don't get me wrong I hope it does work, it would be great to see more community involvement in Government Departments. The individual people are great and keen to see that engagement, but as a group things tend to break down.

colinm wrote:oh, and Yay LIDAR! They're talking about amazingly precise resolution.

It is amazing hey. I have seen some of the data, you can even see individual wires on linking power poles. (but it also differentiates the solid ground below the wires). The aerial photos that accompany it are also stunning.

Matt :)
wildwalks
Magnus administratio
Magnus administratio
 
Posts: 946
Joined: Mon 22 Nov, 2010 4:35 pm
ASSOCIATED ORGANISATIONS: Wildwalks, Bushwalk.com & NPA NSW
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby WarrenH » Mon 16 Apr, 2012 8:38 am

I've been trying to access the NSW GeoSpatial Portal and all I see is,

Service Temporarily Unavailable
The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.


Does anyone have access?

This is now the third week of seeing this no access info to the NSW GeoSpatial Portal, ... Could the site be down for this long? I've tried gaining access from several directions, all to no avail. I've been to the LPI trouble shooting section and followed the suggestions and procedures.

I had good access in the past to the portal, it is just in the last few weeks that there has been the no access message. Today I sent LPI an email.

Warren.
Last edited by WarrenH on Mon 16 Apr, 2012 11:58 am, edited 1 time in total.
WarrenH
Phyllocladus aspleniifolius
Phyllocladus aspleniifolius
 
Posts: 991
Joined: Thu 07 Jan, 2010 6:54 pm
Region: Australian Capital Territory

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Mon 16 Apr, 2012 11:46 am

Yes, it's been down for me for the past couple of weeks. I probably should have emailed them too, but kept thinking they'd get around to fixing it!
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby WarrenH » Mon 16 Apr, 2012 12:04 pm

Tom, thank you.

This now appears on the homepage;

A searchable repository of available services to the public and to professionals (via subscription ) will soon be available. If you need access now, please contact us on : generalenquiry@lpi.nsw.gov.au

I hope this doesn't mean that LPI are planning to introduce fees for non-commercial access. I hope it only means (via subscription) to professionals still and the status quo is kept.

Cheers Mate.

Warren.
WarrenH
Phyllocladus aspleniifolius
Phyllocladus aspleniifolius
 
Posts: 991
Joined: Thu 07 Jan, 2010 6:54 pm
Region: Australian Capital Territory

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Tue 17 Apr, 2012 10:29 am

Reply from LPI, not really sure what it means:

"Hello Tom,

I have been informed the GeoSpatial Portal is experiencing issues and is currently
unavailable until at least Thursday. There is a meeting in regards to this service
with the possibility of rolling out the new SIXmaps which replaces the old viewers.
Unfortunately until then it appears the service will not be available.

Kind regards,"
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Wed 09 May, 2012 4:21 pm

Thursday has come ... and gone. I might chase them up again, though I'm not hopeful
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby tom_brennan » Sun 13 May, 2012 4:31 pm

I've sent another email to LPI, just awaiting a response.

However, I note on Matthew Hollingworth's page the following note:
[3/5/12: N.B. The NSW topographic map server used by this software has been decommissioned, and maps cannot currently be produced. A replacement server using a different protocol is available, however I will need to write new code to use this server. I will remove this note if/when I finish this work.]

I'm guessing he has also contacted them, and it sounds like the current system is gone...
Bushwalking NSW - http://bushwalkingnsw.com
User avatar
tom_brennan
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1350
Joined: Wed 29 Sep, 2010 9:21 am
Location: Sydney
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby colinm » Thu 12 Jul, 2012 11:31 am

Yay, looks like mholling has released a new version of his nswtopo tool (new as at 30June2012) https://github.com/mholling/nswtopo

It appears that this may emit SVG ... which is kind of terrific.

Colin.
sig pending approval
User avatar
colinm
Athrotaxis cupressoides
Athrotaxis cupressoides
 
Posts: 389
Joined: Wed 27 Jul, 2011 10:39 am
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby ryan » Tue 23 Oct, 2012 7:28 pm

The vegext link on github and in the README file for the vegetation download from CANRI doesn't work (as at 22/10/12)..

This link: http://test.canri.nsw.gov.au/download/veg_extent.html does work and links to all the vegetation extents, you do still need to submit a name and email address form.

Cheers
ryan
Nothofagus cunninghamii
Nothofagus cunninghamii
 
Posts: 1
Joined: Tue 23 Oct, 2012 7:24 pm
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby L2obin » Mon 05 Nov, 2012 3:24 pm

Has anyone managed to get this to work? Would love some highly detailed maps that are compatible with the "PDF Maps" app from Avanza (http://avenza.com/pdf-maps)
L2obin
Nothofagus gunnii
Nothofagus gunnii
 
Posts: 14
Joined: Thu 18 Oct, 2012 12:08 am
Region: New South Wales

Re: make your own NSW topo maps [large images]

Postby christophermoore » Mon 05 Nov, 2012 9:11 pm

I'm having this issue with it, and would also love to get it working. But these sort of OSS projects depend on the spare time of those with the know-how...
christophermoore
Nothofagus gunnii
Nothofagus gunnii
 
Posts: 36
Joined: Thu 19 Apr, 2012 5:49 pm
Region: New South Wales
Gender: Male

Re: make your own NSW topo maps [large images]

Postby WarrenH » Tue 13 Nov, 2012 6:37 am

When the amazing NSW Geospatial Portal shut down, I changed to LPI's SIX Lite site. SIX Lite was a major drop in image quality and speed (paint peels quicker than SIX Lite refreshes).

On the SIX Lite site now there is a notice saying and detailing the changes to SIX Lite with SIX Maps, happening at the end of November 2012. The changes sound good, like maps with higher quality images and faster speed.

Fingers crossed that it is this November 2012 being mentioned though. When Geospatial Portal went off line the note was, Site temporarily closed until Thursday ... well Thursday didn't come.

Warren.
WarrenH
Phyllocladus aspleniifolius
Phyllocladus aspleniifolius
 
Posts: 991
Joined: Thu 07 Jan, 2010 6:54 pm
Region: Australian Capital Territory

Re: make your own NSW topo maps [large images]

Postby michael_p » Thu 15 Nov, 2012 12:54 pm

WarrenH wrote:On the SIX Lite site now there is a notice saying and detailing the changes to SIX Lite with SIX Maps, happening at the end of November 2012. The changes sound good, like maps with higher quality images and faster speed.

Warren,

I've found this site http://maps.six.nsw.gov.au/ some months back and have been using/playing with it ever since. It had a "Beta" message in the corner for ages but this disappeared recently. Maybe this is the replacement for the SIX Viewers. It certainly works a lot better than the Viewers.

Regards,
Michael.
One foot in front of the other.
User avatar
michael_p
Athrotaxis selaginoides
Athrotaxis selaginoides
 
Posts: 1376
Joined: Sun 15 Nov, 2009 6:58 pm
Location: Macarthur Region of Sydney.
Region: New South Wales
Gender: Male

Next

Return to New South Wales & ACT

Who is online

Users browsing this forum: No registered users and 25 guests