Merge branch 'devel'

docs/config.rst

@@ -358,6 +358,8 @@ values. The valid configuration keys are listed below.
     This is a where a specific texture pack can be found to be used during this render.
     It can be either a folder or a directory. Its value should be a string.
 
+.. _crop:
+
 ``crop``
     You can use this to render a small subset of your map, instead of the entire
     thing. The format is (min x, min z, max x, max z).
@@ -387,15 +389,20 @@ values. The valid configuration keys are listed below.
         once it has been rendered once.
 
         For an expansion to the bounds, because chunks in the new bounds have
-        the same mtime as the old, tiles will not automatically be updated. You
+        the same mtime as the old, tiles will not automatically be updated,
-        may need to use :option:`--forcerender` to force those tiles to update.
+        leaving strange artifacts along the old border. You may need to use
-        (You can define the ``forcerender`` option on just one render)
+        :option:`--forcerender` to force those tiles to update.  (You can use
+        the ``forcerender`` option on just one render by adding ``'forcerender':
+        True`` to that render's configuration)
 
-        For reductions to the bounds, because there is currently no mechanism to
+        For reductions to the bounds, you will need to render your map at least
-        detect tiles that shouldn't exist but do, old tiles may remain and will
+        once with the :option:`--check-tiles` mode activated, and then once with
-        not get deleted. The only fix for this currently is to delete that
+        the :option:`--forcerender` option. The first run will go and delete tiles that
-        render directory entirely and render it again with
+        should no longer exist, while the second will render the tiles around
-        :option:`--forcerender`. 
+        the edge properly. Also see :ref:`this faq entry<cropping_faq>`.
+
+        Sorry there's no better way to handle these cases at the moment. It's a
+        tricky problem and nobody has devoted the effort to solve it yet.
 
 ``forcerender``
     This is a boolean. If set to ``True`` (or any non-false value) then this
@@ -443,6 +450,21 @@ values. The valid configuration keys are listed below.
         removed some tiles, you may need to do some manual deletion on the
         remote side.
 
+.. _option_markers:
+
+``markers``
+    This controls the display of markers, signs, and other points of interest
+    in the output HTML.  It should be a list of filter functions.  
+
+    .. note::
+
+       Setting this configuration option alone does nothing.  In order to get
+       markers and signs on our map, you must also run the genPO script.  See
+       the :doc:`Signs and markers<signs>` section for more details and documenation.
+
+    
+    **Default:** ``[]`` (an empty list)
+
 .. _customrendermodes:
 
 Custom Rendermodes and Rendermode Primitives

docs/faq.rst

@@ -90,3 +90,61 @@ If you are seeing exorbitant memory usage, then it is likely either a bug or a
 subtly corrupted world. Please file an issue or come talk to us on IRC so we can
 take a look! See :ref:`help`.
 
+.. _cropping_faq:
+
+I've deleted some sections of my world but they still appear in the map
+-----------------------------------------------------------------------
+Okay, so making edits to your world in e.g. worldedit has some caveats,
+especially regarding deleting sections of your world.
+
+This faq also applies to using the :ref:`crop<crop>` option.
+
+Under normal operation with vanilla Minecraft and no external tools fiddling
+with the world, Overviewer performs correctly, rendering areas that have
+changed, and everything is good.
+
+Often with servers one user will travel reeeeally far out and cause a lot of
+extra work for the server and for The Overviewer, so you may be tempted to
+delete parts of your map. This can cause problems, so read on to learn what you
+can do about it.
+
+First some explanation: Until recently (Mid May 2012) The Overviewer did not
+have any facility for detecting parts of the map that should no longer exist.
+Remember that the map is split into small tiles. When Overviewer starts up, the
+first thing it does is calculate which tiles should exist and which should be
+updated. This means it does not check or even look at tiles that should not
+exist. This means that parts of your world which have been deleted will hang
+around on your map because Overviewer won't even look at those tiles and notice
+they shouldn't be there. You may even see strange artifacts around the border as
+tiles that should exist get updated.
+
+Now, with the :option:`--check-tiles` option, The Overviewer *will* look for and
+remove tiles that should no longer exist. So you can render your map once with
+that option and all those extra tiles will get removed automatically. However,
+this is only half of the solution. The other half is making sure the tiles along
+the border are re-rendered, or else it will look like your map is being cut off.
+
+Explanation: The tiles next to the ones that were removed are tiles that should
+continue to exist, but parts of them have chunks that no longer exist. Those
+tiles then should be re-rendered to show that. However, since tile updates are
+triggered by the chunk last-modified timestamp changing, and the chunks that
+still exist have *not* been updated, those tiles will not get re-rendered.
+
+The consequence of this is that your map will end up looking cut-off around the
+new borders that were created by the parts you deleted. You can fix this one of
+two ways.
+
+1. You can run a render with :option:`--forcerender`. This has the unfortunate
+   side-effect of re-rendering *everything* and doing much more work than is
+   necessary.
+
+2. Manually navigate the tile directory hierarchy and manually delete tiles
+   along the edge. Then run once again with :option:`--check-tiles` to re-render
+   the tiles you just deleted. This may not be as bad as it seems. Remember each
+   zoom level divides the world into 4 quadrants: 0, 1, 2, and 3 are the upper
+   left, upper right, lower left, and lower right. It shouldn't be too hard to
+   navigate it manually to find the parts of the map that need re-generating.
+
+3. The third non-option is to not worry about it. The problem will fix itself if
+   people explore near there, because that will force that part of the map to
+   update.

docs/index.rst

@@ -181,6 +181,7 @@ Documentation Contents
    building
    running
    config
+   signs
    win_tut/windowsguide
    faq
    design/designdoc

docs/running.rst

@@ -110,41 +110,80 @@ The first render can take a while, depending on the size of your world.
 Options
 -------
 
-These options change the way the render works, and are intended to be things you
+The following three options change the way The Overviewer determines which tiles
-only have to use once-in-a-while.
+to update, and are intended to be things you only have to use in special
+situations. You should not normally have to specify these options; the default
+is typically correct.
+
+.. cmdoption:: --no-tile-checks
+
+    With this option, The Overviewer will determine which tiles to render by
+    looking at the saved last-render timestamp and comparing it to the
+    last-modified time of the chunks of the world. It builds a tree of tiles
+    that need updating and renders only those tiles.
+
+    This option does not do *any* checking of tile mtimes on disk, and thus is
+    the cheapest option: only rendering what needs updating while minimising
+    disk IO.
+
+    The caveat is that the *only* thing to trigger a tile update is if Minecraft
+    updates a chunk. Any other reason a tile may have for needing re-rendering
+    is not detected. This means that changes in your render configuration will
+    not be reflected in your world except in updated chunks. It could also cause
+    problems if the system clock of the machine running Minecraft is not stable.
+    
+    **This option is the default** unless :option:`--forcerender` or
+    :option:`--check-tiles` is in effect.  This option conflicts with
+    :option:`--forcerender` and :option:`--check-tiles`.
+
+.. cmdoption:: --check-tiles
+
+    Forces The Overviewer to check each tile on disk and check to make sure it
+    is up to date. This also checks for tiles that shouldn't exist and deletes
+    them.
+
+    This is functionally equivalent to :option:`--no-tile-checks` with the
+    difference that each tile is individually checked. It is therefore useful if
+    the tiles are not consistent with the last-render timestamp that is
+    automatically stored. This option was designed to handle the case where the
+    last render was interrupted -- some tiles have been updated but others
+    haven't, so each one is checked before it is rendered.
+    
+    This is slightly slower than :option:`--no-tile-checks` due to the
+    additonaly disk-io involved in reading tile mtimes from the filesystem
+    
+    Since this option also checks for erroneous tiles, **It is also useful after
+    you delete sections of your map, e.g. with worldedit, to delete tiles that
+    should no longer exist.**
+
+    The caveats with this option are the same as for :option:`--no-tile-checks`
+    with the additional caveat that tile timestamps in the filesystem must be
+    preserved. If you copy tiles or make changes to them with an external tool
+    that modifies mtimes of tiles, it could cause problems with this option.
+
+    This option is automatically activated when The Overviewer detects the last
+    render was interrupted midway through. This option conflicts with
+    :option:`--forcerender` and :option:`--no-tile-checks`
 
 .. cmdoption:: --forcerender
 
     Forces The Overviewer to re-render every tile regardless of whether it
-    thinks it needs updating or not. This is similar to deleting your output
+    thinks it needs updating or not. It does no tile mtime checks, and therefore
-    directory and rendering from scratch.
+    ignores the last render time of the world, the last modification times of
+    each chunk, and the filesystem mtimes of each tile. It unconditionally
+    renders every tile that exists.
 
-    This is the default mode for first-time renders. This option overrides
+    The caveat with this option is that it does *no* checks, period. Meaning it
-    :option:`--check-tiles` and :option:`--no-tile-checks`
+    will not detect tiles that do exist, but shouldn't (this can happen if your
+    world shrinks for some reason. For that specific case,
+    :option:`--check-tiles` is actually the appropriate mode).
 
-.. cmdoption:: --check-tiles
+    This option is useful if you have changed a render setting and wish to
+    re-render every tile with the new settings.
 
-    Forces The Overviewer to check each tile on disk and compare its
+    This option is automatically activated for first-time renders. This option
-    modification time to the modification time of the part of the world that
+    conflicts with :option:`--check-tiles` and :option:`--no-tile-checks`
-    tile renders. This is slightly slower than the default, but can be useful if
-    there are some tiles that somehow got skipped.
 
-    This option is the default when The Overviewer detects the last render was
-    interrupted midway through. This option overrides :option:`--forcerender`
-    and :option:`--no-tile-checks`
-
-.. cmdoption:: --no-tile-checks
-
-    With this option, The Overviewer will not do any checking of tiles on disk
-    to determine what tiles need updating. Instead, it will look at the time
-    that the last render was performed, and render parts of the map that were
-    changed since then. This is the fastest option, but could cause problems if
-    the clocks of the Minecraft server and the machine running The Overviewer
-    are not in sync.
-
-    This option is the default unless the condition for :option:`--forcerender`
-    or :option:`--check-tiles` is in effect.  This option overrides
-    :option:`--forcerender` and :option:`--check-tiles`.
 
 .. cmdoption:: -p <procs>, --processes <procs>
 
@@ -154,6 +193,23 @@ only have to use once-in-a-while.
 
     This option can also be specified in the config file as :ref:`processes <processes>`
 
+.. cmdoption:: -v, --verbose
+
+    Activate a more verbose logging format and turn on debugging output. This
+    can be quite noisy but also gives a lot more info on what The Overviewer is
+    doing.
+
+.. cmdoption:: -q, --quiet
+
+    Turns off one level of logging for quieter output. You can specify this more
+    than once. One ``-q`` will suppress all INFO lines. Two will suppress all
+    INFO and WARNING lines. And so on for ERROR and CRITICAL log messages.
+
+    If :option:`--verbose<-v>` is given, then the first ``-q`` will counteract
+    the DEBUG lines, but not the more verbose logging format. Thus, you can
+    specify ``-v -q`` to get only INFO logs and higher (no DEBUG) but with the
+    more verbose logging format.
+
 .. _installing-textures:
 
 Installing the Textures

docs/signs.rst

@@ -0,0 +1,75 @@
+.. _signsmarkers:
+
+=================
+Signs and Markers
+=================
+
+The Overviewer can display signs, markers, and other points of interest on your
+map.  This works a little differently than it has in the past, so be sure to read
+these docs carefully.
+
+In these docs, we use the term POI (or point of interest) to refer to entities and
+tileentities.
+
+
+Configuration File
+==================
+
+
+Filter Functions
+----------------
+
+A filter function is a python function that is used to figure out if a given POI
+should be part of a markerSet of not.  The function should accept one argument
+(a dictionary, also know as an associative array), and return a boolean::
+
+    def signFilter(poi):
+        "All signs"
+        return poi['id'] == 'Sign'
+
+The single argument will either a TileEntity, or an Entity taken directly from 
+the chunk file.  In this example, this function returns true only if the type
+of entity is a sign.  For more information of TileEntities and Entities, see
+the `Chunk Format <http://www.minecraftwiki.net/wiki/Chunk_format>`_ page on
+the Minecraft Wiki.
+
+.. note::
+   The doc string ("All signs" in this example) is important.  It is the label
+   that appears in your rendered map
+
+A more advanced filter may also look at other entity fields, such as the sign text::
+
+    def goldFilter(poi):
+        "Gold"
+        return poi['id'] == 'Sign' and (\
+            'gold' in poi['Text1] or
+            'gold' in poi['Text2'])
+           
+This looks for the word 'gold' in either the first or second line of the signtext.
+
+Since writing these filters can be a little tedious, a set of predefined filters
+functions are provided.  See the :ref:`predefined_filter_functions` section for
+details.
+
+Render Dictionary Key
+---------------------
+
+Each render can specify a list of zero or more filter functions.  Each of these
+filter functions become a selectable item in the 'Signs' drop-down menu in the
+rendered map.  For example::
+
+    renders['myrender'] = {
+            'world': 'myworld',
+            'title': "Example",
+            'markers': [allFilter, anotherFilter],
+    }
+
+
+
+.. _predefined_filter_functions:
+
+Predefined Filter Functions
+===========================
+
+TODO write some filter functions, then document them here
+

genPOI.py

@@ -0,0 +1,132 @@
+#!/usr/bin/python2
+
+'''
+genPOI.py
+
+Scans regionsets for TileEntities and Entities, filters them, and writes out
+POI/marker info.
+
+A markerSet is list of POIs to display on a tileset.  It has a display name,
+and a group name.
+
+markersDB.js holds a list of POIs in each group
+markers.js holds a list of which markerSets are attached to each tileSet
+
+
+'''
+import os
+import logging
+import json
+from optparse import OptionParser
+
+from overviewer_core import logger
+from overviewer_core import nbt
+from overviewer_core import configParser, world
+
+helptext = """
+%prog --config=<config file>"""
+
+logger.configure()
+
+def handleSigns(rset, outputdir, render, rname):
+    	
+    if hasattr(rset, "_pois"):
+        return
+
+    logging.info("Looking for entities in %r", rset)
+
+    filters = render['markers']
+    rset._pois = dict(TileEntities=[], Entities=[])
+
+    for (x,z,mtime) in rset.iterate_chunks():
+        data = rset.get_chunk(x,z)
+        rset._pois['TileEntities'] += data['TileEntities']
+        rset._pois['Entities']     += data['Entities']
+
+
+def main():
+    parser = OptionParser(usage=helptext)
+    parser.add_option("--config", dest="config", action="store", help="Specify the config file to use.")
+
+    options, args = parser.parse_args()
+    if not options.config:
+        parser.print_help()
+        return
+
+    # Parse the config file
+    mw_parser = configParser.MultiWorldParser()
+    mw_parser.parse(options.config)
+    try:
+        config = mw_parser.get_validated_config()
+    except Exception:
+        logging.exception("An error was encountered with your configuration. See the info below.")
+        return 1
+
+    destdir = config['outputdir']
+    # saves us from creating the same World object over and over again
+    worldcache = {}
+
+    markersets = set()
+    markers = dict()
+
+    for rname, render in config['renders'].iteritems():
+        try:
+            worldpath = config['worlds'][render['world']]
+        except KeyError:
+            logging.error("Render %s's world is '%s', but I could not find a corresponding entry in the worlds dictionary.",
+                    rname, render['world'])
+            return 1
+        render['worldname_orig'] = render['world']
+        render['world'] = worldpath
+        
+        # find or create the world object
+        if (render['world'] not in worldcache):
+            w = world.World(render['world'])
+            worldcache[render['world']] = w
+        else:
+            w = worldcache[render['world']]
+        
+        rset = w.get_regionset(render['dimension'])
+        if rset == None: # indicates no such dimension was found:
+            logging.error("Sorry, you requested dimension '%s' for %s, but I couldn't find it", render['dimension'], render_name)
+            return 1
+      
+        for f in render['markers']:
+            markersets.add((f, rset))
+            name = f.__name__ + hex(hash(f))[-4:] + "_" + hex(hash(rset))[-4:]
+            try:
+                l = markers[rname]
+                l.append(dict(groupName=name, displayName = f.__doc__))
+            except KeyError:
+                markers[rname] = [dict(groupName=name, displayName=f.__doc__),]
+
+        handleSigns(rset, os.path.join(destdir, rname), render, rname)
+
+    logging.info("Done scanning regions")
+    logging.info("Writing out javascript files")
+    markerSetDict = dict()
+    for (flter, rset) in markersets:
+        # generate a unique name for this markerset.  it will not be user visible
+        name = flter.__name__ + hex(hash(flter))[-4:] + "_" + hex(hash(rset))[-4:]
+        markerSetDict[name] = dict(created=False, raw=[])
+        for poi in rset._pois['TileEntities']:
+            if flter(poi):
+                markerSetDict[name]['raw'].append(poi)
+    #print markerSetDict
+
+    with open(os.path.join(destdir, "markersDB.js"), "w") as output:
+        output.write("var markersDB=")
+        json.dump(markerSetDict, output, indent=2)
+        output.write(";\n");
+    with open(os.path.join(destdir, "markers.js"), "w") as output:
+        output.write("var markers=")
+        json.dump(markers, output, indent=2)
+        output.write(";\n");
+    with open(os.path.join(destdir, "baseMarkers.js"), "w") as output:
+        output.write("overviewer.util.injectMarkerScript('markersDB.js');\n")
+        output.write("overviewer.util.injectMarkerScript('markers.js');\n")
+        output.write("overviewer.collections.haveSigns=true;\n")
+    logging.info("Done")
+
+if __name__ == "__main__":
+    main()

overviewer.py

@@ -375,12 +375,11 @@ dir but you forgot to put quotes around the directory, since it contains spaces.
 
         # create our TileSet from this RegionSet
         tileset_dir = os.path.abspath(os.path.join(destdir, render_name))
-        if not os.path.exists(tileset_dir):
-            os.mkdir(tileset_dir)
 
         # only pass to the TileSet the options it really cares about
         render['name'] = render_name # perhaps a hack. This is stored here for the asset manager
         tileSetOpts = util.dict_subset(render, ["name", "imgformat", "renderchecks", "rerenderprob", "bgcolor", "imgquality", "optimizeimg", "rendermode", "worldname_orig", "title", "dimension", "changelist"])
+        tileSetOpts.update({"spawn": w.find_true_spawn()}) # TODO find a better way to do this
         tset = tileset.TileSet(rset, assetMrg, tex, tileSetOpts, tileset_dir)
         tilesets.append(tset)
 

overviewer_core/assetmanager.py

@@ -42,11 +42,6 @@ directory.
         self.outputdir = outputdir
         self.renders = dict()
 
-        #  stores Points Of Interest to be mapped with markers
-        #  This is a dictionary of lists of dictionaries
-        #  Each regionset's name is a key in this dictionary
-        self.POI = dict()
-
         # look for overviewerConfig in self.outputdir
         try:
             with open(os.path.join(self.outputdir, "overviewerConfig.js")) as c:
@@ -65,13 +60,6 @@ directory.
         return dict()
         
 
-
-    def found_poi(self, regionset, poi_type, contents, chunkX, chunkY):
-        if regionset.name not in self.POI.keys():
-            POI[regionset.name] = []
-        # TODO based on the type, so something
-        POI[regionset.name].append
-
     def initialize(self, tilesets):
         """Similar to finalize() but calls the tilesets' get_initial_data()
         instead of get_persistent_data() to compile the generated javascript
@@ -152,6 +140,12 @@ directory.
             global_assets = os.path.join(util.get_program_path(), "web_assets")
         mirror_dir(global_assets, self.outputdir)
 
+		# write a dummy baseMarkers.js if none exists
+        if not os.path.exists(os.path.join(self.outputdir, "baseMarkers.js")):
+            with open(os.path.join(self.outputdir, "baseMarkers.js"), "w") as f:
+                f.write("// if you wants signs, please see genPOI.py\n");
+
+
         # create overviewer.js from the source js files
         js_src = os.path.join(util.get_program_path(), "overviewer_core", "data", "js_src")
         if not os.path.isdir(js_src):

overviewer_core/data/js_src/overviewer.js

@@ -29,7 +29,19 @@ overviewer.collections = {
          */
         'infoWindow':   null,
 
-        'worldViews': []
+        'worldViews': [],
+
+        'haveSigns': false,
+
+        /**
+         * Hold the raw marker data for each tilest
+         */
+        'markerInfo': {},
+
+        /**
+         * holds a reference to the spawn marker. 
+         */
+        'spawnMarker': null,
     };
 
 overviewer.classes = {

overviewer_core/data/js_src/util.js

@@ -58,6 +58,13 @@ overviewer.util = {
         var coordsdiv = new overviewer.views.CoordboxView({tagName: 'DIV'});
         coordsdiv.render();
 
+        if (overviewer.collections.haveSigns) {
+            var signs = new overviewer.views.SignControlView();
+            signs.registerEvents(signs);
+        }
+
+        var spawnmarker = new overviewer.views.SpawnIconView();
+
         // Update coords on mousemove
         google.maps.event.addListener(overviewer.map, 'mousemove', function (event) {
             coordsdiv.updateCoords(event.latLng);    
@@ -72,6 +79,7 @@ overviewer.util = {
             overviewer.mapView.updateCurrentTileset();
 
             compass.render();
+            spawnmarker.render();
 
             // re-center on the last viewport
             var currentWorldView = overviewer.mapModel.get("currentWorldView");
@@ -102,6 +110,7 @@ overviewer.util = {
                 overviewer.map.setZoom(zoom);
             }
 
+
         });
 
         var worldSelector = new overviewer.views.WorldSelectorView({tagName:'DIV'});
@@ -116,15 +125,43 @@ overviewer.util = {
         // Jump to the hash if given
         overviewer.util.initHash();
 
+        overviewer.util.initializeMarkers();
 
         /*
            overviewer.util.initializeMapTypes();
            overviewer.util.initializeMap();
-           overviewer.util.initializeMarkers();
            overviewer.util.initializeRegions();
            overviewer.util.createMapControls();
            */
     },
+
+    'injectMarkerScript': function(url) {
+        var m = document.createElement('script'); m.type = 'text/javascript'; m.async = false;
+        m.src = url;
+        var s = document.getElementsByTagName('script')[0]; s.parentNode.appendChild(m);
+    },
+
+    'initializeMarkers': function() {
+        return;
+
+    },
+
+    'createMarkerInfoWindow': function(marker) {
+            var windowContent = '<div class="infoWindow"><img src="' + marker.icon +
+                '"/><p>' + marker.title.replace(/\n/g,'<br/>') + '</p></div>';
+            var infowindow = new google.maps.InfoWindow({
+                'content': windowContent
+            });
+            google.maps.event.addListener(marker, 'click', function() {
+                if (overviewer.collections.infoWindow) {
+                    overviewer.collections.infoWindow.close();
+                }
+                infowindow.open(overviewer.map, marker);
+                overviewer.collections.infoWindow = infowindow;
+            });
+        },
+
+
     /**
      * This adds some methods to these classes because Javascript is stupid
      * and this seems like the best way to avoid re-creating the same methods
@@ -239,7 +276,6 @@ overviewer.util = {
 
         var zoomLevels = model.get("zoomLevels");
         var north_direction = model.get('north_direction');
-        //console.log("fromWorldToLatLng: north_direction is %r", north_direction);
 
         // the width and height of all the highest-zoom tiles combined,
         // inverted
@@ -406,7 +442,6 @@ overviewer.util = {
         // save this info is a nice easy to parse format
         var currentWorldView = overviewer.mapModel.get("currentWorldView");
         currentWorldView.options.lastViewport = [x,y,z,zoom];
-        //console.log("Updated lastViewport: %r" , [x,y,z,zoom]);
         window.location.replace("#/" + Math.floor(x) + "/" + Math.floor(y) + "/" + Math.floor(z) + "/" + zoom + "/" + w + "/" + maptype);
     },
     'updateHash': function() {

overviewer_core/data/js_src/views.js

@@ -3,13 +3,9 @@ overviewer.views= {}
 
 overviewer.views.WorldView = Backbone.View.extend({
     initialize: function(opts) {
-        //console.log("WorldView::initialize()");
-        //console.log(this.model.get("tileSets"));
         this.options.mapTypes = [];
         this.options.mapTypeIds = [];
         this.model.get("tileSets").each(function(tset, index, list) {
-            //console.log(" eaching");
-            //console.log("  Working on tileset %s" , tset.get("name"));
             var ops = {
                 getTileUrl: overviewer.gmap.getTileUrlGenerator(tset.get("path"), tset.get("base"), tset.get("imgextension")),
                 'tileSize':     new google.maps.Size(
@@ -57,7 +53,6 @@ overviewer.views.WorldSelectorView = Backbone.View.extend({
         "change select":  "changeWorld"
     },
     changeWorld: function() {
-        //console.log("change world!");
         var selectObj = this.$("select")[0];
         var selectedOption = selectObj.options[selectObj.selectedIndex]; 
 
@@ -120,11 +115,8 @@ overviewer.views.CoordboxView = Backbone.View.extend({
 
 overviewer.views.GoogleMapView = Backbone.View.extend({
     initialize: function(opts) {
-        //console.log(this);
         this.options.map = null;
         var curWorld = this.model.get("currentWorldView").model;
-        //console.log("Current world:");
-        //console.log(curWorld);
 
         var curTset = curWorld.get("tileSets").at(0);
 
@@ -145,12 +137,6 @@ overviewer.views.GoogleMapView = Backbone.View.extend({
 
         var mapOptions = {};
     // 
-        curWorld.get("tileSets").each(function(elem, index, list) {
-            //console.log("Setting up map for:");
-            //console.log(elem);
-            //console.log("for %s generating url func with %s and %s", elem.get("name"), elem.get("path"), elem.get("base"));
-
-        });
         // init the map with some default options.  use the first tileset in the first world
         this.options.mapOptions = {
             zoom:                   curTset.get("defaultZoom"),
@@ -174,7 +160,6 @@ overviewer.views.GoogleMapView = Backbone.View.extend({
         // register every ImageMapType with the map
         $.each(overviewer.collections.worldViews, function( index, worldView) {
             $.each(worldView.options.mapTypes, function(i_index, maptype) {
-                //console.log("registered %s with the maptype registery", worldView.model.get("name") + maptype.shortname);
                 overviewer.map.mapTypes.set(overviewerConfig.CONST.mapDivId + 
                     worldView.model.get("name") + maptype.shortname , maptype);
             });
@@ -185,7 +170,6 @@ overviewer.views.GoogleMapView = Backbone.View.extend({
      * Should be called when the current world has changed in GoogleMapModel
      */
     render: function() {
-        //console.log("GoogleMapView::render()"); 
         var view = this.model.get("currentWorldView");
         this.options.mapOptions.mapTypeControlOptions = {
             mapTypeIds: view.options.mapTypeIds};
@@ -200,14 +184,11 @@ overviewer.views.GoogleMapView = Backbone.View.extend({
      * Keeps track of the currently visible tileset
      */
     updateCurrentTileset: function() {
-                              //console.log("GoogleMapView::updateCurrentTileset()");
         var currentWorldView = this.model.get("currentWorldView");
         var gmapCurrent = overviewer.map.getMapTypeId();
         for (id in currentWorldView.options.mapTypeIds) {
             if (currentWorldView.options.mapTypeIds[id] == gmapCurrent) {
-                //console.log("updating currenttileset");
                 this.options.currentTileSet = currentWorldView.model.get("tileSets").at(id);
-                //console.log(this);
             }
         }
 
@@ -219,3 +200,188 @@ overviewer.views.GoogleMapView = Backbone.View.extend({
 });
 
 
+
+
+/**
+ * SignControlView
+ */
+overviewer.views.SignControlView = Backbone.View.extend({
+    /** SignControlView::initialize
+     */
+    initialize: function(opts) {
+        $(this.el).addClass("customControl");
+        overviewer.map.controls[google.maps.ControlPosition.TOP_RIGHT].push(this.el);
+
+    },
+    registerEvents: function(me) {
+        google.maps.event.addListener(overviewer.map, 'maptypeid_changed', function(event) {
+            overviewer.mapView.updateCurrentTileset();
+
+            // workaround IE issue.  bah!
+            if (typeof markers=="undefined") { return; }
+            me.render();
+            // hide markers, if necessary
+            // for each markerSet, check:
+            //    if the markerSet isnot part of this tileset, hide all of the markers
+            var curMarkerSet = overviewer.mapView.options.currentTileSet.attributes.path;
+            var dataRoot = markers[curMarkerSet];
+            if (!dataRoot) { 
+                // this tileset has no signs, so hide all of them
+                for (markerSet in markersDB) {
+                    if (markersDB[markerSet].created) {
+                        jQuery.each(markersDB[markerSet].raw, function(i, elem) {
+                            elem.markerObj.setVisible(false);
+                        });
+                    }
+                }
+
+                return; 
+            }
+            var groupsForThisTileSet = jQuery.map(dataRoot, function(elem, i) { return elem.groupName;})
+            for (markerSet in markersDB) {
+                if (jQuery.inArray(markerSet, groupsForThisTileSet) == -1){
+                    // hide these
+                    if (markersDB[markerSet].created) {
+                        jQuery.each(markersDB[markerSet].raw, function(i, elem) {
+                            elem.markerObj.setVisible(false);
+                        });
+                    }
+                    markersDB[markerSet].checked=false;
+                }
+                // make sure the checkboxes checked if necessary
+                $("[_mc_groupname=" + markerSet + "]").attr("checked", markersDB[markerSet].checked);
+
+            }
+
+        });
+
+    },
+    /**
+     * SignControlView::render
+     */
+    render: function() {
+
+        var curMarkerSet = overviewer.mapView.options.currentTileSet.attributes.path;
+        //var dataRoot = overviewer.collections.markerInfo[curMarkerSet];
+        var dataRoot = markers[curMarkerSet];
+
+        this.el.innerHTML=""
+        
+        // if we have no markerSets for this tileset, do nothing:
+        if (!dataRoot) { return; }
+
+
+        var controlText = document.createElement('DIV');
+        controlText.innerHTML = "Signs";
+
+        var controlBorder = document.createElement('DIV');
+        $(controlBorder).addClass('top');
+        this.el.appendChild(controlBorder);
+        controlBorder.appendChild(controlText);
+
+        var dropdownDiv = document.createElement('DIV');
+        $(dropdownDiv).addClass('dropDown');
+        this.el.appendChild(dropdownDiv);
+        dropdownDiv.innerHTML='';
+
+        // add the functionality to toggle visibility of the items
+        $(controlText).click(function() {
+                $(controlBorder).toggleClass('top-active');
+                $(dropdownDiv).toggle();
+        });
+
+
+        // add some menus
+        for (i in dataRoot) {
+            var group = dataRoot[i];
+            this.addItem({label: group.displayName, groupName:group.groupName, action:function(this_item, checked) {
+                markersDB[this_item.groupName].checked = checked;
+                jQuery.each(markersDB[this_item.groupName].raw, function(i, elem) {
+                    elem.markerObj.setVisible(checked);
+                });
+            }});
+        }
+
+        iconURL = overviewerConfig.CONST.image.signMarker;
+        //dataRoot['markers'] = [];
+        //
+        for (i in dataRoot) {
+            var groupName = dataRoot[i].groupName;
+            if (!markersDB[groupName].created) {
+                for (j in markersDB[groupName].raw) {
+                    var entity = markersDB[groupName].raw[j];
+                    var marker = new google.maps.Marker({
+                            'position': overviewer.util.fromWorldToLatLng(entity.x,
+                                entity.y, entity.z, overviewer.mapView.options.currentTileSet),
+                            'map':      overviewer.map,
+                            'title':    jQuery.trim(entity.Text1 + "\n" + entity.Text2 + "\n" + entity.Text3 + "\n" + entity.Text4), 
+                            'icon':     iconURL,
+                            'visible':  false
+                    }); 
+                    if (entity['id'] == 'Sign') {
+                        overviewer.util.createMarkerInfoWindow(marker);
+                    }
+                    jQuery.extend(entity, {markerObj: marker});
+                }
+                markersDB[groupName].created = true;
+            }
+        }
+
+
+    },
+    addItem: function(item) {
+        var itemDiv = document.createElement('div');
+        var itemInput = document.createElement('input');
+        itemInput.type='checkbox';
+
+        // give it a name
+        $(itemInput).data('label',item.label);
+        $(itemInput).attr("_mc_groupname", item.groupName);
+        jQuery(itemInput).click((function(local_item) {
+            return function(e) {
+                item.action(local_item, e.target.checked);
+            };
+        })(item));
+
+        this.$(".dropDown")[0].appendChild(itemDiv);
+        itemDiv.appendChild(itemInput);
+        var textNode = document.createElement('text');
+        if(item.icon) {
+            textNode.innerHTML = '<img width="15" height="15" src="' + 
+                item.icon + '">' + item.label + '<br/>';
+        } else {
+            textNode.innerHTML = item.label + '<br/>';
+        }
+
+        itemDiv.appendChild(textNode);
+
+
+    },
+});
+
+/**
+ * SpawnIconView
+ */
+overviewer.views.SpawnIconView = Backbone.View.extend({
+    render: function() {
+        // 
+        var curTileSet = overviewer.mapView.options.currentTileSet;
+        if (overviewer.collections.spawnMarker) {
+            overviewer.collections.spawnMarker.setMap(null);
+            overviewer.collections.spawnMarker = null;
+        }
+        var spawn = curTileSet.get("spawn");
+        if (spawn) {
+            overviewer.collections.spawnMarker = new google.maps.Marker({
+                'position': overviewer.util.fromWorldToLatLng(spawn[0],
+                    spawn[1], spawn[2], overviewer.mapView.options.currentTileSet),
+                'map':      overviewer.map,
+                'title':    'spawn',
+                'icon':     overviewerConfig.CONST.image.spawnMarker,
+                'visible':  false
+                }); 
+            overviewer.collections.spawnMarker.setVisible(true);
+        }
+    }
+});
+

overviewer_core/data/web_assets/index.html

@@ -17,6 +17,7 @@
 <script type="text/javascript" src="backbone.js"></script>
 <script type="text/javascript" src="overviewerConfig.js"></script>
 <script type="text/javascript" src="overviewer.js"></script>
+<script type="text/javascript" src="baseMarkers.js"></script>
 
 </head>
 

overviewer_core/settingsDefinition.py

@@ -75,6 +75,7 @@ renders = Setting(required=True, default=util.OrderedDict(),
             "rerenderprob": Setting(required=True, validator=validateFloat, default=0),
             "crop": Setting(required=False, validator=validateCrop, default=None),
             "changelist": Setting(required=False, validator=validateStr, default=None),
+            "markers": Setting(required=False, validator=validateMarkers, default=[]),
             
             # Remove this eventually (once people update their configs)
             "worldname": Setting(required=False, default=None,

overviewer_core/settingsValidators.py

@@ -43,6 +43,13 @@ def checkBadEscape(s):
         fixed = True
     return (fixed, fixed_string)
 
+def validateMarkers(filterlist):
+    if type(filterlist) != list:
+        raise ValidationException("Markers must specify a list of filters")
+    for x in filterlist:
+        if not callable(x):
+            raise ValidationException("%r must be a function"% x)
+    return filterlist
 
 def validateWorldPath(worldpath):
     _, worldpath = checkBadEscape(worldpath)

overviewer_core/tileset.py

@@ -178,10 +178,7 @@ class TileSet(object):
         This class does nothing with it except pass it through.
 
         outputdir is the absolute path to the tile output directory where the
-        tiles are saved. It is assumed to exist already.
+        tiles are saved. It is created if it doesn't exist
-        TODO: This should probably be relative to the asset manager's output
-        directory to avoid redundancy.
-
         
         Current valid options for the options dictionary are shown below. All
         the options must be specified unless they are not relevant. If the
@@ -211,12 +208,15 @@ class TileSet(object):
                 partial interrupted render left off.
 
             1
-                For render-tiles, render all whose chunks have an mtime greater
+                "check-tiles" mode. For render-tiles, render all whose chunks
-                than the mtime of the tile on disk, and their upper-tile
+                have an mtime greater than the mtime of the tile on disk, and
-                ancestors.
+                their upper-tile ancestors.
                 
                 Also check all other upper-tiles and render any that have
                 children with more rencent mtimes than itself.
+
+                Also remove tiles and directory trees that do exist but
+                shouldn't.
                 
                 This is slower due to stat calls to determine tile mtimes, but
                 safe if the last render was interrupted.
@@ -270,6 +270,7 @@ class TileSet(object):
         self.regionset = regionsetobj
         self.am = assetmanagerobj
         self.textures = texturesobj
+        self.outputdir = os.path.abspath(outputdir)
 
         config = self.am.get_tileset_config(self.options.get("name"))
         self.config = config
@@ -277,17 +278,42 @@ class TileSet(object):
         self.last_rendertime = config.get('last_rendertime', 0)
 
         if "renderchecks" not in self.options:
+            # renderchecks was not given, this indicates it was not specified
+            # in either the config file or the command line. The following code
+            # attempts to detect the most appropriate mode
             if not config:
-                # No persistent config? This is a full render then.
+                # No persistent config?
+                if os.path.exists(self.outputdir):
+                    # Somehow there's no config but the output dir DOES exist.
+                    # That's strange!
+                    logging.warning(
+                        "For render '%s' I couldn't find any persistent config, "
+                        "but I did find my tile directory already exists. This "
+                        "shouldn't normally happen, something may be up, but I "
+                        "think I can continue...", self.options['name'])
+                    logging.info("Switching to --check-tiles mode")
+                    self.options['renderchecks'] = 1
+                else:
+                    # This is the typical code path for an initial render, make
+                    # this a "forcerender"
+                    self.options['renderchecks'] = 2
+                    logging.debug("This is the first time rendering %s. Doing" +
+                            " a full-render",
+                            self.options['name'])
+            elif not os.path.exists(self.outputdir):
+                # Somehow the outputdir got erased but the metadata file is
+                # still there. That's strange!
+                logging.warning(
+                        "This is strange. There is metadata for render '%s' but "
+                        "the output directory is missing. This shouldn't "
+                        "normally happen. I guess we have no choice but to do a "
+                        "--forcerender", self.options['name'])
                 self.options['renderchecks'] = 2
-                logging.debug("This is the first time rendering %s. Doing" +
-                        " a full-render",
-                        self.options['name'])
             elif config.get("render_in_progress", False):
                 # The last render must have been interrupted. The default should be
-                # 1 then, not 0
+                # a check-tiles render then
                 logging.warning(
-                        "The last render for %s didn't finish. I'll be " +
+                        "The last render for '%s' didn't finish. I'll be " +
                         "scanning all the tiles to make sure everything's up "+
                         "to date.",
                         self.options['name'],
@@ -304,9 +330,15 @@ class TileSet(object):
                         )
                 self.options['renderchecks'] = 0
 
-        # Throughout the class, self.outputdir is an absolute path to the
+        if not os.path.exists(self.outputdir):
-        # directory where we output tiles. It is assumed to exist.
+            if self.options['renderchecks'] != 2:
-        self.outputdir = os.path.abspath(outputdir)
+                logging.warning(
+                "The tile directory didn't exist, but you have specified "
+                "explicitly not to do a --fullrender (which is the default for "
+                "this situation). I'm overriding your decision and setting "
+                "--fullrender for just this run")
+                self.options['rednerchecks'] = 2
+            os.mkdir(self.outputdir)
 
         # Set the image format according to the options
         if self.options['imgformat'] == 'png':
@@ -375,6 +407,7 @@ class TileSet(object):
         This method returns an iterator over (obj, [dependencies, ...])
         """
 
+        # The following block of code implementes the changelist functionality.
         fd = self.options.get("changelist", None)
         if fd:
             logging.debug("Changelist activated for %s (fileno %s)", self, fd)
@@ -485,6 +518,8 @@ class TileSet(object):
                 last_rendertime = self.max_chunk_mtime,
                 imgextension = self.imgextension,
                 )
+        if (self.regionset.get_type() == "overworld"):
+            d.update({"spawn": self.options.get("spawn")})
         try:
             d['north_direction'] = self.regionset.north_dir
         except AttributeError:
@@ -492,7 +527,6 @@ class TileSet(object):
 
         return d
 
-
     def _find_chunk_range(self):
         """Finds the chunk range in rows/columns and stores them in
         self.minrow, self.maxrow, self.mincol, self.maxcol
@@ -569,6 +603,11 @@ class TileSet(object):
                 logging.warning("Your map seems to have shrunk. Did you delete some chunks? No problem. Re-arranging tiles, just a sec...")
                 for _ in xrange(curdepth - self.treedepth):
                     self._decrease_depth()
+                logging.info(
+                        "There done. I'm switching to --check-tiles mode for "
+                        "this one render. This will make sure any old tiles that "
+                        "should no longer exist are deleted.")
+                self.options['renderchecks'] = 1
 
     def _increase_depth(self):
         """Moves existing tiles into place for a larger tree"""
@@ -928,11 +967,17 @@ class TileSet(object):
             os.utime(tmppath, (max_chunk_mtime, max_chunk_mtime))
 
     def _iterate_and_check_tiles(self, path):
-        """A generator function over all tiles that need rendering in the
+        """A generator function over all tiles that should exist in the subtree
-        subtree identified by path. This yields, in order, all tiles that need
+        identified by path. This yields, in order, all tiles that need
         rendering in a post-traversal order, including this node itself.
 
-        This method actually yields tuples in this form:
+        This method takes one parameter:
+
+        path
+            The path of a tile that should exist
+
+
+        This method yields tuples in this form:
             (path, mtime, needs_rendering)
         path
             is the path tuple of the tile that needs rendering
@@ -945,8 +990,9 @@ class TileSet(object):
             is a boolean indicating this tile does in fact need rendering.
 
         (Since this is a recursive generator, tiles that don't need rendering
-        are not propagated all the way out of the recursive stack, but the
+        are not propagated all the way out of the recursive stack, but are
-        immediate parent call needs to know its mtime)
+        still yielded to the immediate parent because it needs to know its
+        childs' mtimes)
 
         """
         if len(path) == self.treedepth:
@@ -967,7 +1013,16 @@ class TileSet(object):
                 # If a tile has been modified more recently than any of its
                 # chunks, then this could indicate a potential issue with
                 # this or future renders.
-                logging.warning("I found a tile with a more recent modification time than any of its chunks. This can happen when a tile has been modified with an outside program, or by a copy utility that doesn't preserve mtimes. Overviewer uses the filesystem's mtimes to determine which tiles need rendering and which don't, so it's important to preserve the mtimes Overviewer sets. Please see our FAQ page on docs.overviewer.org or ask us in IRC for more information")
+                logging.warning(
+                        "I found a tile with a more recent modification time "
+                        "than any of its chunks. This can happen when a tile has "
+                        "been modified with an outside program, or by a copy "
+                        "utility that doesn't preserve mtimes. Overviewer uses "
+                        "the filesystem's mtimes to determine which tiles need "
+                        "rendering and which don't, so it's important to "
+                        "preserve the mtimes Overviewer sets. Please see our FAQ "
+                        "page on docs.overviewer.org or ask us in IRC for more "
+                        "information")
                 logging.warning("Tile was: %s", imgpath)
 
             if max_chunk_mtime > tile_mtime:
@@ -975,20 +1030,23 @@ class TileSet(object):
                 yield (path, None, True)
             else:
                 # This doesn't need rendering. Return mtime to parent in case
-                # they do need to render
+                # its mtime is less, indicating the parent DOES need a render
                 yield path, max_chunk_mtime, False
         
         else:
             # A composite-tile.
-            # First, recurse to each of our children
             render_me = False
             max_child_mtime = 0
+
+            # First, recurse to each of our children
             for childnum in xrange(4):
                 childpath = path + (childnum,)
 
                 # Check if this sub-tree should actually exist, so that we only
                 # end up checking tiles that actually exist
                 if not self.dirtytree.query_path(childpath):
+                    # Here check if it *does* exist, and if so, nuke it.
+                    self._nuke_path(childpath)
                     continue
 
                 for child_path, child_mtime, child_needs_rendering in \
@@ -1034,6 +1092,31 @@ class TileSet(object):
                     # Nope.
                     yield path, max_child_mtime, False
 
+    def _nuke_path(self, path):
+        """Given a quadtree path, erase it from disk. This is called by
+        _iterate_and_check_tiles() as a helper-method.
+
+        """
+        if len(path) == self.treedepth:
+            # path referrs to a single tile
+            tileobj = RenderTile.from_path(path)
+            imgpath = tileobj.get_filepath(self.outputdir, self.imgextension)
+            if os.path.exists(imgpath):
+                # No need to catch ENOENT since this is only called from the
+                # master process
+                logging.debug("Found an image that shouldn't exist. Deleting it: %s", imgpath)
+                os.remove(imgpath)
+        else:
+            # path referrs to a composite tile, and by extension a directory
+            dirpath = os.path.join(self.outputdir, *(str(x) for x in path))
+            imgpath = dirpath + "." + self.imgextension
+            if os.path.exists(imgpath):
+                logging.debug("Found an image that shouldn't exist. Deleting it: %s", imgpath)
+                os.remove(imgpath)
+            if os.path.exists(dirpath):
+                logging.debug("Found a subtree that shouldn't exist. Deleting it: %s", dirpath)
+                shutil.rmtree(dirpath)
+
 ##
 ## Functions for converting (x, z) to (col, row) and back
 ##

overviewer_core/world.py

@@ -19,6 +19,8 @@ import os.path
 from glob import glob
 import logging
 import hashlib
+import time
+import random
 
 import numpy
 
@@ -91,8 +93,16 @@ class World(object):
         if not os.path.exists(os.path.join(self.worlddir, "level.dat")):
             raise ValueError("level.dat not found in %s" % self.worlddir)
 
-        # Hard-code this to only work with format version 19133, "Anvil"
         data = nbt.load(os.path.join(self.worlddir, "level.dat"))[1]['Data']
+        # it seems that reading a level.dat file is unstable, particularly with respect
+        # to the spawnX,Y,Z variables.  So we'll try a few times to get a good reading
+        # empirically, it seems that 0,50,0 is a "bad" reading
+        # update: 0,50,0 is the default spawn, and may be valid is some cases
+        # more info is needed
+        data = nbt.load(os.path.join(self.worlddir, "level.dat"))[1]['Data']
+            
+
+        # Hard-code this to only work with format version 19133, "Anvil"
         if not ('version' in data and data['version'] == 19133):
             logging.critical("Sorry, This version of Minecraft-Overviewer only works with the 'Anvil' chunk format")
             raise ValueError("World at %s is not compatible with Overviewer" % self.worlddir)
@@ -163,7 +173,7 @@ class World(object):
         # location
 
         ## read spawn info from level.dat
-        data = self.data
+        data = self.leveldat
         disp_spawnX = spawnX = data['SpawnX']
         spawnY = data['SpawnY']
         disp_spawnZ = spawnZ = data['SpawnZ']
@@ -175,24 +185,32 @@ class World(object):
         ## clamp spawnY to a sane value, in-chunk value
         if spawnY < 0:
             spawnY = 0
-        if spawnY > 127:
+        if spawnY > 255:
-            spawnY = 127
+            spawnY = 255
         
         # Open up the chunk that the spawn is in
-        regionset = self.get_regionset(0)
+        regionset = self.get_regionset("overworld")
         try:
             chunk = regionset.get_chunk(chunkX, chunkZ)
         except ChunkDoesntExist:
             return (spawnX, spawnY, spawnZ)
+    
+        def getBlock(y):
+            "This is stupid and slow but I don't care"
+            targetSection = spawnY//16
+            for section in chunk['Sections']:
+                if section['Y'] == targetSection:
+                    blockArray = section['Blocks']
+                    return blockArray[inChunkX, inChunkZ, y % 16]
+
 
-        blockArray = chunk['Blocks']
 
         ## The block for spawn *within* the chunk
         inChunkX = spawnX - (chunkX*16)
         inChunkZ = spawnZ - (chunkZ*16)
 
         ## find the first air block
-        while (blockArray[inChunkX, inChunkZ, spawnY] != 0) and spawnY < 127:
+        while (getBlock(spawnY) != 0) and spawnY < 256:
             spawnY += 1
 
         return spawnX, spawnY, spawnZ