Friday, January 6, 2012

Tips for caching with ArcGIS Server 10

I recently received a request for best practices for map caching. Below was my response.

First, read the help. The help has really improved on caching so it is well worth the read even if you have read it all before.

Here are a couple of my additional suggestions:
  • For basemaps you should use Mixed for the format. This will allow your basemap to overlay other basemaps. Try out different compression values before you build your cache. With Imagery I’ve found 55 is good but for vector basemaps 90 is better. As the number decreases the quality decreases but the speed increases.
  • Use MSD based services for caching whenever you can. If you have aerial imagery in a mosaic dataset you might find that the MSD service applies a stretch to the imagery. This is documented in NIM-070858. There is a workaround in that bug description that you can use if you run into this. This is fixed in 10.1.
  • Use WGS 1984 Web Mercator (Auxiliary Sphere) as your coordinate system and use the standard Esri/Google/Bing tiling scheme. This is the standard for the web. There is no reason to use anything else. If you have 6 inch resolution imagery you can build your cache down to 576 but I wouldn’t go any further than that. Also don’t remove any scales from the tiling scheme. If you don’t want a scale just don’t build it but leave the scale in your tiling scheme. I know I am taking an extreme position on using the standard CS and tiling scheme but if we all do this we will contribute to the same web GIS platform instead of our own web GIS silo.
  • Always test your cache settings before building a cache. I like to use on demand caching to get a quick visual of what the cache will look like before I build. If you want to get an idea of how big the cache will be and how long it will take to build, build 5% of your cache using cache by feature class. When evaluating the cache using a tool like Fiddler or Firebug to inspect the tiles. If they are bigger than 50 KB you probably need to change your image format or compression.
  • Always use compact cache. It is faster to build and the same speed for delivering tiles.
  • Build your cache with a staging server. You should never build cache directly on a production system. It can also be handy to have a big physical box somewhere for building cache. If you store your cache on a storage device like a SAN you can have your big cache builder machine build the cache in the same server directory as the production system. The easiest way to do this is to copy the configuration file (.cfg) from your production system to your cache building / staging system.
  • If you are dealing with vector data, project it all to WGS 1984 Web Mercator (Auxiliary Sphere) in a local (absolute path on the SOC machine) file geodatabase and use an MSD based map service.
  • Use on-demand caching very sparingly. It is rarely useful outside of testing.
  • After the cache is built you can change the map document used by the service. This is useful for reducing the amount of RAM used on the production system and removing a dependency on the source data. Once the cache is built, the content of the map document is only used for queries so it makes sense to simplify that map for just those items that will be queried.
  • If you are using cache by feature class make your features REALLY BIG. The help says this too. For a state like California I might only use 15 or 20 features. For a small city or county you probably don’t even need to do cache by feature class.
  • Read all of the help. There is a lot of great stuff in there.
  • The ArcGIS Server Blog also has some great content on map caching.

Caching will change with 10.1 as well so make sure to read the updated help when you start with 10.1. This was humbling for me. I've read the help many times but I found it incredibly useful to start at the beginning of the new 10.1 help and read it all again.

6 comments:

  1. Just a quick question. Could one set cache on demand on certain scales when it is checked.

    Thank you.

    Dan

    ReplyDelete
  2. No, cache on demand works on all cache scales that have not been built. Most of the time those are the largest scales.

    ReplyDelete
  3. Hi Tom, you wrote findBadTiles.py to validate ArcGIS Server Cache. I've had some problems with 10.2. After some time, the connection was closed with the following error message :

    error(10053, 'An established connection was aborted by the software in your host machine')

    I could fix it with by adding a try-except-block when getting the response from arcgis server, and re-opening the connection in this case:

    Original (line 244):
    ---
    response = conn.getresponse()
    ---

    Patch:
    ---
    try:
    response = conn.getresponse()
    except:
    # retry
    conn = httplib.HTTPConnection(parsedUrl.netloc)
    conn.request("GET", partTileURL, headers=txtheaders)
    response = conn.getresponse()
    ---

    I think the cause of the problem was that the number of keep-alive connections to arcgis servers tomcat hits the limit of 100 (the default value).

    After applying my patch, the script runs without error.

    (I post this patch here because I cannot remember where I downloaded findBadTiles.py and I found your name in the comments of the script).

    Best regards,
    Stefan Offermann (twitter: @st3fan_de)

    ReplyDelete
  4. Your blog is amazing! thank you so much for your creative inspirations and thank you for sharing. Please visit http://goo.gl/sAnVrz

    ReplyDelete