• “Dungeons & Dragons done right on Microsoft Surface”.
• “SurfaceScapes Follow-up: Bringing D&D to Microsoft Surface”.
• “New gameplay video with D&D on Surface”.

Some of it is quite clumsy, but the potential is clearly there. And looks amazing. While this could definitely aid the playing of D&D, there are many games that could benefit from this kind of treatment. Specifically, any games that benefit both from having people physically present and that would also benefit from computation and helpful interfaces (yes, I’m looking at you, Twilight Imperium) could get a huge boost from this technology.

It’s a long way away from being available or affordable, and probably a long way from being easily modifiable, too. But we don’t seem to be that far from having commonplace enhanced tabletop games. In the meantime, for roleplaying games there’s the RPTools suite—a suite which may well eventually be a big part of an open source tabletop gaming toolset.

Tags: entertainment, games, roleplaying, software, tech

Related Posts

• Inkscape Mon 08 Mar 2010
• AI and Games Thu 26 Feb 2009
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• Comments on GMing After a 15-Year Hiatus Sun 03 Jan 2010
• 2010 Goals Fri 01 Jan 2010
• The Perfect is the Enemy of the Good: Duke Nukem Forever Tue 22 Dec 2009
• RPGs I’ve Played Tue 15 Dec 2009
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009

You can put Python directly into Vim script files, but I wanted to do that as minimally as possible. I added a Vim script that would (essentially) include a Python file, and then I had that Python file import from other Python files as necessary. The Python file includes a class, TBase, that contains the functions I want to run; as much code as possible is kept out of those functions and is in other files—files which ideally are individually testable.

To get to those functions from within Vim, I did this:

if filereadable($HOME."/.vim/plugin/tadhg/tadhg.py")
    pyfile $HOME/.vim/plugin/tadhg/tadhg.py
endif

" Create the commands pointing the Python in tadhg.py:
if !exists(":T")
  command! -range -nargs=+ T python tadhgbase = TBase('rs=<line1> rf=<line2>'); tadhgbase('<args>')
endif

nmap <D-CR> :T

That code goes in a Vim script that goes in ~/.vim/plugins/tadhg/; it’s loaded when Vim is. This means that the code in tadhg.py is evaluated at Vim start, so the TBase class is available. Furthermore the user Ex mode command “T” creates a new instance of TBase and then calls it with whatever the user enters after “T”. Finally, Command-Enter is mapped to enter Ex mode and type “T ”, which is quite close indeed to the jEdit functionality I had, but without the disadvantages. I have much more control over the namespace, and I can pass arguments.

When the T command is invoked, a new TBase instance is created and it gets passed the range start and range end line numbers from Vim, which is pretty important for some of the functions. Then the new instance is called, with the argument of whatever was typed after “T ”.

This is the code for handling that latter aspect:

def __call__(self, argstring):
    """
    The first arg is the command; the rest are arguments to that command.
    """
    args = argstring.split(u" ")

    #Need to extract the keyword args out of args
    kwargs = dict([(arg.split("=")[0], arg.split("=")[1]) for arg in s.split(" ") if "=" in arg])
    args = [arg for arg in s.split(" ") if "=" not in arg]

    if args and args[0] in self.__class__.__dict__.keys():
        import types
        f = self.__class__.__dict__[args[0]]
        if type(f) == types.FunctionType:
            if args[1:]:
                f(self, *args[1:])
            else:
                f(self)
    else:
        print "no command by that name"

In other words, if the first (non-keyword) argument matches the name of a function in the class, run it and pass the rest of the (non-keyword) arguments to it. If I ever need keyword arguments as well, I’ll add the ability to pass them along too.

In order to actually manipulate Vim from these scripts, you have to use import vim. When I started writing this, I wasn’t sure how I’d test that, and I tried to keep the meat of the code in other files, e.g.:

def mpc(self):
    """
    Count the words, then insert them into the document's word count line.
    """
    from write_wordcount import WriteWordcount
    wwc = WriteWordcount(tadhgbase)
    wwc.vim_main(vim)

(Note that I use import sys; sys.path.append(myscriptdir) at the top of the file to make these imports work.)

One of the advantages of this construction is that WriteWordCount can theoretically be used as a standalone script, on any file, and also that its vim_main method takes vim as an argument. For testing purposes, I had to created a MockVim class, which currently looks like this:

class MockVimBuffer(object):

    def __init__(self):
        self.lines = []

    def __contains__(self, item):
        return item in self.lines

    def __iter__(self):
        return self.lines.__iter__()

    def __getitem__(self, index):
        return self.lines.__getitem__(index)

    def __setitem__(self, index, value):
        return self.lines.__setitem__(index, value)

class MockVimCurrent(object):

    def __init__(self):
        self.__dict__["buffer"] = MockVimBuffer()

    def __setattr__(self, attr, value):
        if attr == "buffer":
            self.__dict__["buffer"].lines = value
        else:
            self.__dict__[attr] = value

class MockVim(object):

    def __init__(self):
        self.current = MockVimCurrent()
        self.commands = []

    def eval(self, command):
        command_list = {
            "&ft": lambda: self.rawmodes,
            "exists('b:TotalWordCount')": lambda: False,
            "tvar": lambda: self.tvar,

        }
        return command_list.get(command, lambda: "")()

    def command(self, command):
        self.commands.append(command)

The eval method in MockVim simply returns whatever was appropriate for various tests I was running; I should change it to return self.command_list.get(command, lambda: "")() instead, and make command_list an instance variable so that the various tests can manipulate it as needed instead of having to put specific commands in the base code.

Because the methods in TBase pass vim along to the instances or methods of the modules they import, testing those modules is relatively simple: in the test, you create a new MockVim object, call it vim, and then pass it along instead.

So everything is great (and testable). Unless, that is, you (meaning me) lapse from full test-driven development discipline and bits and pieces of functionality creep into TBase. Because TBase has that import vim line at the top of it and running it when you’re not actually in Vim produces an ImportError.

The right way to deal with this is to refactor TBase to take vim as one of its initialization arguments, and then have it set that as an instance variable, and then pass that in its methods, while also altering the line in the parent Vim script to:

command! -range -nargs=+ T python import vim; tadhgbase = TBase(vim, 'rs=<line1> rf=<line2>'); tadhgbase('<args>')

That way TBase is much easier to test, since the fake vim can be passed in at test time. However, for a bunch of reasons I didn’t want to do that just yet, partly because it means some reasonably heavy refactoring of pieces of code that I use every day—without tests, because the whole issue here is that TBase isn’t testable. So I wanted a way to make it testable while altering as little of its existing code (which I know currently works, after all) as possible. This took me a while to figure out, and is rather hacky—but it works.

First, in the test file, do this:

from mockvim import MockVim
#Hack a mock Vim into the global namespace so that we can actually test:
import __builtin__
mv = MockVim()
__builtin__.mv = mv
from tadhg import TBase

Then alter the import vim line above the TBase class to instead read:

try:
    import vim
except ImportError:
    import __builtin__
    vim = __builtin__.mv

What this really does is hack a giant global variable, vim, into all of the Python that’s run after those lines in the test file. Normally doing this seems like a rather bad idea, but it’s also rather necessary to handle the less-than-ideal situation I’m dealing with. Once I have the test harness using this hackery running properly, I’ll feel a lot better about switching over to a better architecture.

Tags: coding, Jython, python, software, tech, text editing, Vim

Related Posts

• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• Better Word Count in jEdit Sun 19 Jul 2009
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009
• Python Script for Subversion Status Sun 02 Aug 2009
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• jEdit Macros in Python Fri 17 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Thunderbird, Muttator, and Filters Thu 18 Mar 2010
• Using Vim with Thunderbird Fri 12 Mar 2010

text = init.textArea.getSelectedText()

This is the Python for getting the current visual selection in Vim:

self.vim.command('normal gv"wygv')
self.vim.command("let tvar = getreg(’t’)")
text = unicode(self.vim.eval("tvar"), "utf-8")

From a programming perspective, jEdit is clearly a lot nicer to deal with.

I have two primary reasons for making the switch:

• Future proofing: I couldn’t switch to the various pre-release versions of jEdit 4.3 because I couldn’t get my Jython scripts to work with them, and that made me worry that there simply weren’t enough people scripting jEdit in Python to make it likely that the plugin allowing Jython macros would be supported. Given how useful my scripts are, that would have trapped me with jEdit 4.2, or some future version, if support for JythonInterpreter flagged. Vim, on the other hand, seems to have a lot of people scripting for it in Python, and I don’t see Python support simply disappearing anytime soon with a new release.
• Power/speed: I’m someone who’s quite inclined towards keyboard shortcuts in all applications, and this means Vim should be a highly efficient environment.
• Mentality: Related to the previous point, I think that being able to manipulate text more powerfully will reinforce the approach of scripting as many steps as possible, rather than doing them (no matter how efficiently) manually. Even over the last couple of weeks, I’ve noticed that I’m more inclined to write scripts than to put up with various manual steps in my daily workflow.

So far I’ve mainly worked to replicate my jEdit setup in Vim, with a few small improvements here and there; the largest one is probably the live word count, which means that I can glance at the status line to see the current word count instead of having to select the text to be counted and then invoke the word counter. I’ve also tweaked it so it only counts the relevant parts of the document—for example, in this blog post, it doesn’t count the title or the datestamp.

I don’t intend to abandon jEdit, and have a feeling that I’ll use it for various things here and there, but I’ve switched over the Vim for the majority of my writing and programming.

Tags: coding, Jython, python, software, tech, text editing, Vim, writing

Related Posts

• Better Word Count in Vim Sun 17 Jan 2010
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• Better Word Count in jEdit Sun 19 Jul 2009
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009
• Python Script for Subversion Status Sun 02 Aug 2009
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• jEdit Macros in Python Fri 17 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Thunderbird, Muttator, and Filters Thu 18 Mar 2010
• Using Vim with Thunderbird Fri 12 Mar 2010

• :set hidden. Absolutely critical, this. Stops Vim from complaining when you try to switch buffers and your current buffer has unsaved changes.
• bufexplorer. Makes switching buffers a lot easier.
• A better Python syntax file. I didn’t like the defaults.
• My own indentation and syntax files for reStructuredText.

Really, though, the key first one was :set hidden. Before that I felt that I had completely misunderstood Vim’s file management model.

Once I got the syntax highlighting to a reasonable state, I ported my word count macro over to Vim. This wasn’t too hard after the inevitable character encoding problems. For working within Vim scripts, I strongly suggest using the following:

ulines = [unicode(line, "utf-8") for line in vim.current.buffer]

(Assuming your Vim encoding is set to UTF-8, of course.)

The Vim version of my script isn’t as versatile yet as the jEdit version, because I don’t know how to make it work with only selected lines (something that’s easy in jEdit). Apart from that, though, it seems to work quite well; the next version of it might do “live” word count in the status bar.

I’m currently writing this in Vim, or kind of: the actual writing is in Vim, but the creation of the file and the template, the automated expansion of various reStructuredText entities, output to reStructuredText, and the publication, are in jEdit as I haven’t ported those over yet.

The script:

function! WordCount()
python << EOF
import re
import vim

class WordCounter(object):
    """
    Vim script for better word count.
    """

    LINE_SEPARATORS = (
        "\r",
        "\n"
    )

    WORD_SEPARATORS = (
        " ",        # space
        "\t",       # tab
        "/",        # slash
        "&",        # ampersand
        ’"’,        # double quotation mark, straight
        u"\u201C",  # double quotation mark, left
        u"\u201D",  # double quotation mark, right
        u"\u2018",  # single quotation mark, left
        u"\u2013",  # en dash
        u"\u2014",  # em dash
        ">",        # greater than symbol
        "<",        # less than symbol
        "+",        # plus
        "=",        # equals
    )

    REPEATER_SEPARATORS = (
    #These are only separators if they’re present consecutively, e.g. -- or ..
        "-",
        "."
    )

    IGNORE = (
    #Not separators per se, but should not be treated as word content
        "’",        # single quotation mark, straight
        u"\u2019",  # single quotation mark, right
        "(",        # left parenthesis
        ")",        # right parenthesis
        "[",        # left bracket
        "]",        # right bracket
        "{",        # left curly bracket
        "}",        # right curly bracket
        "|",        # bar
        "-",        # hyphen
        "#",        # hash mark
        ".",        # period
        "_",        # underscore
        "`",        # backtick
        "\\",        # backslash
    )

    def word_count(self):

        ulines = [unicode(line, "utf-8") for line in vim.current.buffer]
        text = u"\n".join(ulines)

        chars = len(text) #Pretty sure I want the actual char count, not the adjusted char count.

        text = self.remove_directives(text)
        text = self.adjust_for_rest(text)
        words, lines = self.count_words(text)

        print "chars: %s, words: %s, lines: %s" % (chars, words, lines)

    def remove_directives(self, text):
        textlines = text.split("\n")
        newlines = []
        comment = re.compile(r"[ ]*\.\. [a-zA-Z0-9_\|]")
        argument = re.compile(r"    :[^\:]*:")
        for line in textlines:
            if not comment.match(line) and not argument.match(line):
                newlines.append(line)
        return "\n".join(newlines)

    def adjust_for_rest(self, text):
        """
            Go through each of the special cases for reST.
        """
        text = self.rest_adjust_pipe_space(text)

        return text

    def rest_adjust_pipe_space(self, text):
        """
            Special-case "|\ " to make sure e.g. "|Hypnotic Specter|\ s"
            doesn’t get counted as three words.

            |Incinerate|\ s |Hypnotic Specter|\ —|Hypnotic Specter|\ s
            The above line should be counted as five words.
        """
        spacere = re.compile(r"\|\\ ([^ ]{1})")
        finds = spacere.findall(text)
        text = spacere.sub("|\g<1>", text)
        return text

    def count_words(self, text):
        words, lines = 0, 1
        #go through the text character by character:
        word, previous_character = 0, None
        for character in text:
            if character in (self.LINE_SEPARATORS + self.WORD_SEPARATORS) or (character in self.REPEATER_SEPARATORS and previous_character in self.REPEATER_SEPARATORS):
                #it’s a separator
                word = 0
                if character in (self.LINE_SEPARATORS):
                    lines = lines + 1
            elif character in (self.IGNORE):
                pass
            else:
                #it’s part of a word.
                if not word:
                    words = words + 1
                    word = 1
            previous_character = character

        return (words, lines)

WordCounter().word_count()
EOF
endfunction

if !exists(":WW")
  command! WW  :call WordCount()
endif

Tags: coding, Jython, python, software, tech, text editing, Vim, writing

Related Posts

• First Post With Vim Sun 14 Feb 2010
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• Better Word Count in jEdit Sun 19 Jul 2009
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009
• Python Script for Subversion Status Sun 02 Aug 2009
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• jEdit Macros in Python Fri 17 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Thunderbird, Muttator, and Filters Thu 18 Mar 2010
• Using Vim with Thunderbird Fri 12 Mar 2010

Tags: Blog, coding, PHP, tech, web-development, WordPress

Related Posts

• WordPress 2.8 Upgrade Sun 14 Jun 2009
• WP Plugins: Recent Commentary and Show Tags in RSS Mon 23 Mar 2009
• Some RSS Changes Thu 19 Mar 2009
• New Blog Functionality: Recent Commentary Mon 23 Feb 2009
• My WordPress Development Setup Fri 13 Feb 2009
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Related Posts Plugin and SQL Trickery Tue 10 Feb 2009
• HTML Past and Future Tue 10 Nov 2009
• 404 Pages Thu 10 Sep 2009

Wired has a long look at what happened, and it seems fair to conclude that one of the problems was a lack of limits.

“pig cop” by arboltsef

Duke Nukem Forever was a project that had almost unlimited money, and a significantly longer development period than any other game I can think of. Its project lead, George Broussard, was more concerned with quality than money or deadlines, but this appears to have led to an inability to ever call it done. More than other creative projects, computer games are yoked to technical advances. There will always be a better graphics engine than the one you built your game on.

I enjoyed Duke Nukem 3D when it came out, and looked forward to the sequel for a couple of years. My interests moved more towards multiplayer, to competitive Quake 3 Arena and CPMA, but Half-Life made clear that extremely compelling single-player first-person shooter games were still relevant, and I thought that perhaps Duke Nukem Forever would be in the same class.

Extremely long development cycles aren’t necessarily indicators of failure; Half-Life 2 took five years, and Team Fortress 2 was released almost ten years after it was first announced. But in that regard at least, Duke (or his reputation?) turned out to indeed be bigger and badder, an ultimately invincible final boss.

Tags: creativity, games, tech

Related Posts

• The Future of Tabletop Games? D&D on the Microsoft Surface Fri 26 Feb 2010
• Pool-Playing Robot: Deep Green Thu 17 Sep 2009
• AI and Games Thu 26 Feb 2009
• sfmagic.org Database Structure Fri 11 May 2007
• Loop!Station and Cheap Computing Power Sun 06 May 2007
• Distractivities Tue 02 Jan 2007
• Some Plans for sfmagic.org Wed 09 Aug 2006
• Inkscape Mon 08 Mar 2010
• Doom and Modern First-Person Shooters Fri 05 Mar 2010
• Vim: the Killer Instinct of Text Editors Thu 04 Mar 2010

It is still questionable as to whether small, highly maneuverable fighter craft make any sense; I think it depends on how maneuverable the larger craft are. If they’re not particularly agile, then simply sending lots of firepower in a straight line will be much more cost-effective.

The comments for the article are worth reading through as well… how space combat would really be is still unclear, but the discussion there has some great food for thought.

Tags: science-fiction, tech, war

Related Posts

• Future Shock Thu 24 Jan 2008
• “Let’s Enhance” Sun 07 Mar 2010
• The Future of Tabletop Games? D&D on the Microsoft Surface Fri 26 Feb 2010
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• WordPress 2.9 Upgrade Sun 10 Jan 2010
• The Perfect is the Enemy of the Good: Duke Nukem Forever Tue 22 Dec 2009
• Doomsday Book Review Fri 13 Nov 2009
• HTML Past and Future Tue 10 Nov 2009

I came across a contrasting Mark Pilgrim article that’s also worth reading: “Why do we have an IMG element?”, which goes over the history of that element in HTML, the objections raised at the time, and how it won out over alternatives.

Tags: coding, tech, web-development

Related Posts

• WordPress 2.9 Upgrade Sun 10 Jan 2010
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• Some Character Encoding Gotchas Thu 16 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• WordPress 2.8 Upgrade Sun 14 Jun 2009
• Pylons Via Apache Port Issues Fri 12 Jun 2009
• PyWebSF: Meetup for SF-Area Python Web Developers Tue 09 Jun 2009
• Yahoo!'s Website Performance Rules Thu 12 Mar 2009
• My WordPress Development Setup Fri 13 Feb 2009
• Editable Google Maps Tue 18 Mar 2008

Tags: economics, hardware, politics, tech

Related Posts

• danah boyd on Online Class Divisions Sun 24 Jun 2007
• Escape Velocity Sun 24 Dec 2006
• Cards Versus Cash Tue 19 Jan 2010
• Tiger Woods and His Sponsors Mon 14 Dec 2009
• Facebook Astroturfing Thu 10 Dec 2009
• Bullying: Just a Hunch Thu 03 Dec 2009
• State Bandits Tue 03 Nov 2009
• Economics, Fairness, and Football Mon 19 Oct 2009
• Wall Street: Even Worse Than You Thought Sun 18 Oct 2009
• Energy Consumption, Data Centers, and Heat Thu 15 Oct 2009

In the meantime, as processors get faster and demand increases, data centers are going to become more energy-hungry, and hotter—which essentially translates to being even more energy-hungry as they need cooling. Hence Iceland’s bid to become a data center haven makes sense, because they have a conveniently cold climate and a conveniently cheap electricity supply.

(The optimist in me hopes that this trend might somehow drive us towards cheap electricity; certainly here in California we don’t make anything near efficient use of natural resources to generate electricity, and of course Buckminster Fuller’s Global Energy Grid would make a lot of sense—but I suspect too many profit-oriented vested interests stand in the way of that; if you’re interested in that topic, see Sharon Beder’s Power Play.)

Tags: economics, energy, tech

Related Posts

• Plug Fun Fri 30 Oct 2009
• Inundation of Spam Mon 05 Nov 2007
• Consumer Ephemera Sun 22 Jul 2007
• danah boyd on Online Class Divisions Sun 24 Jun 2007
• Cellphone Plans Mon 12 Feb 2007
• Escape Velocity Sun 24 Dec 2006
• Height and Negotiation Tue 16 Mar 2010
• The Future of Tabletop Games? D&D on the Microsoft Surface Fri 26 Feb 2010
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• First Post With Vim Sun 14 Feb 2010

• textutil. If you’re not running OS X, I discuss some alternatives below.
• pandoc. In Haskell, so you need to install the Haskell Platform for your system, and then run sudo cabal install pandoc.

Once you have those, it becomes pretty simple:

textutil -convert html -stdout {source_file} | pandoc -f html -t rst --no-wrap > {output_file}

If you don’t have access to textutil, you could try UnRTF, some Perl scripts, rtf2latex2e (if you don’t mind the lack of Unicode support), or rtf2html. If your source files are Word documents rather than RTF documents, you could use OpenOffice.org’s batch conversion utility to get them into RTF and proceed from there. (AbiWord is also an option.)

This is a rather powerful set of tools. It’s extremely useful for me to be able to automate the conversion of my older files, and once they’re in reStructuredText it’s not that hard to get them to other useful formats, including LaTeX and PDF. I have some work yet to do (mainly on styling), but soon I should, for example, be able to have a single reStructuredText file for my résumé that a script can automatically convert to HTML, PDF, HTML suitable for my blog, RTF, Word doc, OpenDocument Text, and plain text. The same should apply for any of my other documents.

It took me quite some time to find the right pieces for this process, which I’ll describe below in case it’s of use to others.

That did still leave me with the problem of getting from RTF to that unspecified sane format. rtf2latex2e looked extremely promising, but it turned out not to support Unicode in RTF. I found UnRTF, but it also had Unicode issues and furthermore would segfault whenever I ran it. rtf2latex had some other issues. I found a set of Perl scripts aimed at the problem, but they gave me odd errors that I couldn’t fix. Finally I came across rtf2txt, which had a critical reference to textutil.

textutil is built into OS X, at least versions starting with 10.4, and handles formats including text, RTF, HTML… and even Word documents. Furthermore, the HTML it produces is quite sane, and in many respects better than the OpenOffice.org or AbiWord HTML output, particularly for simple documents—e.g. italics are surrounded with <i> tags and not <span> tags with some difficult-to-parse class defined elsewhere in the document.

I do some additional work in Python to infer document structure from my files and insert that into the reStructuredText, but that’s fairly trivial.

Tags: coding, personal, python, reStructuredText, software, tech, writing

Related Posts

• Better reST–WordPress Pipeline Tue 28 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• Better Word Count in jEdit Sun 19 Jul 2009
• Moving From Word Processors to reStructuredText Sun 12 Jul 2009
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• 2009 Goals Review Tue 29 Dec 2009
• Python Script for Subversion Status Sun 02 Aug 2009
• Some Minor Software Projects Sun 27 Jan 2008

In it, they also demonstrate “augmented reality pool“, a system that determines the angles being shot and projects them onto the pool table. This is presumably a projection of the system that Deep Green itself uses to figure out how to aim, like a rather more sophisticated version of “The Spider”, a pool training aid.

I suspect that projects like these, despite seeming frivolous on the surface, generate significant progress in robotics and artificial intelligence. Whatever about the future, the current state of AI means that its successes will come in limited domains, and games are excellent testbeds for this.

As for pool specifically, it’s possible that robotic pool players will be a luxury accessory at some point in the near future. Meanwhile, the assisted-vision aspect of it may well develop from requiring a setup worth tens of thousands of dollars, with perfect lighting and multiple cameras, projectors, and computers, to requiring merely a contact lens overlay and maybe a small separate camera—potentially bringing to pool halls everywhere the joys of aimbot use accusations.

Tags: artificial-intelligence, games, pool, robotics, tech

Related Posts

• The Future of Tabletop Games? D&D on the Microsoft Surface Fri 26 Feb 2010
• The Perfect is the Enemy of the Good: Duke Nukem Forever Tue 22 Dec 2009
• AI and Games Thu 26 Feb 2009
• Pool Thu 19 Jun 2008
• sfmagic.org Database Structure Fri 11 May 2007
• Distractivities Tue 02 Jan 2007
• Some Plans for sfmagic.org Wed 09 Aug 2006
• Inkscape Mon 08 Mar 2010
• Doom and Modern First-Person Shooters Fri 05 Mar 2010
• Vim: the Killer Instinct of Text Editors Thu 04 Mar 2010

#! /usr/bin/env python
# -*- coding: utf-8 -*-
# Displays added and modified files in two columns (terminal space permitting)

import subprocess
import re

class SubversionSortedStatus(object):

    STATUS_NEW = "svn st | grep '^\?'" #check for new files
    STATUS_MODIFIED =  "svn st | grep -v '^\?'" #check for modified files

    def __init__(self):
        newfiles = self.get_new_files()
        newfiles_width = self.get_filename_width(newfiles)
        modfiles = self.get_modified_files()
        modfiles_width = self.get_filename_width(modfiles)
        terminal_width = self.terminal_size()[0]
        terminal_midpoint = self.get_midpoint(terminal_width, newfiles_width, modfiles_width)
        self.print_status(terminal_midpoint, newfiles, modfiles)

    def get_new_files(self):
        #the shell command:
        files = self.exec_shell_command(self.STATUS_NEW, stdout=subprocess.PIPE)

        #get rid of the status character:
        status_column = re.compile(r"^\?( +)(?=[^ ]*)")
        lines = files.split("\n")
        lines.sort()
        replaced_lines = []
        for line in lines:
            line = status_column.sub("", line)
            replaced_lines.append(line)
        files = "\n".join(replaced_lines)

        return files.strip()

    def get_modified_files(self):
        #the shell command:
        files = self.exec_shell_command(self.STATUS_MODIFIED, stdout=subprocess.PIPE)

        lines = files.split("\n")
        #sort using only the non-status columns:
        lines.sort(cmp = lambda x, y: cmp(x[6:], y[6:]))
        files = "\n".join(lines)

        return files.strip()

    def get_filename_width(self, files):
        width = 0
        lines = files.split("\n")
        for line in lines:
            if len(line) > width:
                width = len(line)
        return width

    def get_midpoint(self, width, newfiles_width, modfiles_width):
        if width <= newfiles_width + modfiles_width + 1:
            return 0
        else:
            return newfiles_width + 1

    def print_status(self, midpoint, newfiles, modfiles):
        if midpoint:
            newlines = newfiles.split("\n")
            modlines = modfiles.split("\n")
            if len(newlines) > len(modlines):
                maxlength = len(newlines)
            else:
                maxlength = len(modlines)
            for i in range(0, maxlength):
                if i < len(newlines):
                    newfile = newlines[i]
                else:
                    newfile = ""
                if i < len(modlines):
                    modfile = modlines[i]
                else:
                    modfile = ""
                separator = " " * ((midpoint - len(newfile)) + 1)
                print "%s%s%s" % (newfile, separator, modfile)
        else:
            print newfiles
            print "-----"
            print modfiles

    def exec_shell_command(self, command, stdout=None, wait=None, test_mode=False):
        if test_mode:
            print command
            return command

        output = ""
        if stdout:
            commandprocess = subprocess.Popen([command], shell=True, stdout=stdout)
            output = commandprocess.communicate()[0]
        else:
            commandprocess = subprocess.Popen([command], shell=True)
            commandprocess.wait()

        if wait:
            import time
            time.sleep(wait)

        return output

    """
    The next two functions are from Chuck Blake's categorized ls, found at http://pdos.csail.mit.edu/~cblake/cls/cls.py

    """
    def ioctl_GWINSZ(self, fd):                  #### TABULATION FUNCTIONS
        try:                                ### Discover terminal width
            import fcntl, termios, struct, os
            cr = struct.unpack('hh', fcntl.ioctl(fd, termios.TIOCGWINSZ, '1234'))
        except:
            return None
        return cr

    def terminal_size(self):                    ### decide on *some* terminal size
        import os
        env = os.environ
        cr = self.ioctl_GWINSZ(0) or self.ioctl_GWINSZ(1) or self.ioctl_GWINSZ(2)  # try open fds
        if not cr:                                                  # ...then ctty
            try:
                fd = os.open(os.ctermid(), os.O_RDONLY)
                cr = self.ioctl_GWINSZ(fd)
                os.close(fd)
            except:
                pass
        if not cr:                            # env vars or finally defaults
            try:
                cr = (env['LINES'], env['COLUMNS'])
            except:
                cr = (25, 80)
        return int(cr[1]), int(cr[0])         # reverse rows, cols

if __name__ == '__main__':
    svnstatus = SubversionSortedStatus()

Tags: coding, python, software, tech, version-control

Related Posts

• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• Better Word Count in jEdit Sun 19 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Web Development Tools Wed 21 Mar 2007
• jEdit Macros in Python Fri 17 Jul 2009
• Some Character Encoding Gotchas Thu 16 Jul 2009

Now I have a slightly better process, and one that I will have used to publish this post.

The previous version was launched via Quicksilver and operated on the contents of the clipboard; the new version is run from the command line—or, even better, from within jEdit—and operates on a file. It:

• Runs the contents of the file through my reST–Wordpress script, which uses rst2wp and some customized other Python scripts.
• Submits the transformed text to WordPress via python-blogger, a script for doing blog XMLRPC operations in Python.
• Runs svn add on the file.
• Opens the WordPress Edit Draft page for the submitted post.

So, the previous steps for me were:

• Write post in reStructuredText in jEdit.
• Select all.
• Copy.
• Invoke a shell script via Quicksilver.
• Switch to WordPress new post tab in Firefox.
• Paste.
• Fiddle with the post settings slightly.
• Post.

While the new ones are:

• Write post in reStructuredText in jEdit.
• Save.
• Invoke a jEdit macro to run the Python script from within the editor.
• Switch to WordPress new post tab in Firefox.
• Fiddle with the post settings slightly.
• Post.

(As a bonus, this version also adds the file to Subversion.)

The jEdit macro seems

These are the components necessary to make it work:

• Docutils—of course, to handle the reStructuredText
• rst2wp, which I’ve modified a little for my own purposes.
• python-blogger, to talk to WordPress in Python.
• WordPress with XMLRPC enabled so that software can talk to it.
• A short script of my own that pulls the pieces together. (This uses os.subprocess, so Python 2.4 or better is required)
• A jEdit macro to run the script from the last step within the editor.

I’m pretty happy with it, and I think it should be fairly cross-platform, which is important to me as I’d like to take this whole setup with me if I switch environments in future.

Tags: coding, personal, python, reStructuredText, software, tech, web-development, WordPress

Related Posts

• Blog Workflow with reStructuredText Tue 14 Jul 2009
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009
• Some Minor Software Projects Sun 27 Jan 2008
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• WordPress 2.9 Upgrade Sun 10 Jan 2010
• 2009 Goals Review Tue 29 Dec 2009
• Python Script for Subversion Status Sun 02 Aug 2009
• Better Word Count in jEdit Sun 19 Jul 2009

jEdit has such a feature. It’s more or less the same as the one that I’ve been using in AbiWord, and in various word processors before that. But for quite some time I’ve wanted a better word counter. Since jEdit is now my application for all writing and I can script for it in Python, it was time to make the word counter I wanted.

The problem was fairly simple: most applications only consider a few characters to be word separators, typically the line-end character, the space, and the tab. I think I’ve encountered some word processor that let you add more separators yourself, but I can’t remember what it was (possibly an earlier version of OpenOffice.org.

What about dashes, though? The phrase "double standard—I don’t" has four words in it, but almost all applications would report that as being three words long. Similar instances include "either/or", "in&out" (granted, there really should be a space there, but it’s still not one word), and "thirty–forty minutes" (the elusive en dash makes an appearance). Save for that one word processor I can’t remember, no tool would account for those correctly.

Furthermore, the double hyphen is used often as a stand-in for the em dash, and the ellipsis is used without spaces from time to time (not a strictly correct practice, but sometimes style takes precedence), and I wanted a word count that would correctly deal with all of these eventualities.

Here it is, as a jEdit macro written in Jython (meaning it requires the JythonInterpreter plugin):

def word_count():
    """
    jEdit macro for better word count.
    """
    import re
    from org.gjt.sp.jedit import Macros

    LINE_SEPARATORS = (
        "\r",
        "\n"
    )

    WORD_SEPARATORS = (
        " ",        # space
        "\t",       # tab
        "/",        # slash
        "&",        # ampersand
        u"\u2013",  # en dash
        u"\u2014",  # em dash
    )

    REPEATER_SEPARATORS = (
    #These are only separators if they're present consecutively, e.g. -- or ..
        "-",
        "."
    )

    # get local reference to textArea
    textArea = view.getTextArea()

    chars, words, lines = 0, 0, 0
    selection = textArea.getSelection()

    if len(selection) == 0:
        text = textArea.getText()
    else:
        text = textArea.getSelectedText()

    chars = chars + len(text)
    lines = lines + 1

    #go through the text character by character:
    word, previous_character = 0, None
    for character in text:
        if character in (LINE_SEPARATORS + WORD_SEPARATORS) or (character in REPEATER_SEPARATORS and previous_character in REPEATER_SEPARATORS):
            #it's a separator
            word = 0
            if character in (LINE_SEPARATORS):
                lines = lines + 1
        else:
            #it's part of a word.
            if not word:
                words = words + 1
                word = 1
        previous_character = character

    message = "Characters: %s\nWords: %s\n Lines: %s" % (chars, words, lines)
    Macros.message(view, message)

if __name__ == '__main__':
    word_count()

Tags: coding, Jython, python, software, tech, writing

Related Posts

• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009
• Python Script for Subversion Status Sun 02 Aug 2009
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• jEdit Macros in Python Fri 17 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Some Character Encoding Gotchas Thu 16 Jul 2009
• Moving From Word Processors to reStructuredText Sun 12 Jul 2009

I haven’t done much scripting with it, though. I recently came up with some use cases for scripts—involving reStructuredText, naturally—but I was a little reluctant to do the scripting because it involves Java and I really want to keep my current focus on Python and JavaScript.

I was therefore rather happy to discover the existence of the JythonInterpreter plugin, which makes it possible to write macros for jEdit in Python.

I grabbed it and to try it out wrote a macro that looks for all the substitutions and link references in a file, then places corresponding lines at the bottom. That is, if you had the following reStructuredText:

`jEdit`_ has long been my text editor of choice, and I'm using it more than ever now that |I'm writing more or less everything in it|.

and you ran this script, you’d have this:

`jEdit`_ has long been my text editor of choice, and I'm using it more than ever now that |I'm writing more or less everything in it|.

.. |I'm writing more or less everything in it|
.. _jEdit:

Thus saving me the hassle of copying and pasting all of them manually. It works pretty well so far. Here’s the source if any jEdit and reStructuredText people are reading this (remember that you need the JythonInterpreter plugin to make it work):

def get_references():
    """
    jEdit macro that finds all of the substitutions and links in the file and places corresponding references to them at the bottom of the file.
    """
    import re

    # get local reference to textArea
    textArea = view.getTextArea()

    # current caret:
    caret = textArea.getCaretPosition()

    lines = textArea.getText().split('\n')

    #Substitutions:
    subs = {
        "find": re.compile("\|[^\|]+\|"),
        "replace": lambda hit: ".. %s " % hit
    }
    #Links:
    lrefs = {
        "find": re.compile("`[^`]+`_"),
        "replace": lambda hit: ".. _%s: " % (hit[1:-2])
    }

    #Citations and footnotes:
    frefs = {
        "find": re.compile("\[[^`]+\]_"),
        "replace": lambda hit: ".. %s " % (hit[0:-1])
    }

    #the function to run for each operation:
    def run_ops(lines, opdict, matchlist):
        for line in lines:
            if not line.startswith(".. "):
                search = opdict["find"].findall(line)
                for hit in search:
                    hit = opdict["replace"](hit)
                    matchlist.append(hit)
        return matchlist

    #run the operations in desired order:
    matches = []
    for op in [subs, lrefs, frefs]:
        run_ops(lines, op, matches)

    #Put the matches into a string:
    mstring = "\n".join(matches)
    mstring = "\n\n%s" % mstring

    #Go to the end of the buffer and insert the string:
    textArea.goToBufferEnd(0)
    endcaret = textArea.getCaretPosition()
    textArea.getBuffer().insert(endcaret, mstring)

    #Go back to the original cursor position:
    textArea.setCaretPosition(caret)

if __name__ == '__main__':
    get_references()

Tags: coding, Java, Jython, python, tech

Related Posts

• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• Better Word Count in jEdit Sun 19 Jul 2009
• RTF/Word–reStructuredText Toolchain Thu 24 Sep 2009
• Python Script for Subversion Status Sun 02 Aug 2009
• Better reST–WordPress Pipeline Tue 28 Jul 2009
• Some Character Encoding Gotchas Thu 16 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Pylons Via Apache Port Issues Fri 12 Jun 2009

I knew this, but despite having been caught by this before, I “knew” it in the sense that if you had asked me whether or not there’s some way outside the file for the OS to know what the encoding is, I would have answered “no”. However, that didn’t stop me from acting as if some such magical property existed.

I started out by playing with rst2html and opening its output in Firefox. As far as character encoding went, that was fine. Then I switched to rst2wp, which outputs a truncated version of the HTML containing essentially just the body text that you’re going to paste into WordPress. It worked, but the character encodings were screwed up. I spent a lot of time searching for what the difference between the two was in the source code before finding the obvious answer: rst2wp outputs just the body text.

In other words, it outputs to a file that doesn’t contain this:

<?xml version="1.0" encoding="UTF-8"?>
<body>

Without that, the browser has to guess what the encoding is. Looking at the source code doesn’t help either, because that source code is also rendered in whatever the browser thinks the encoding is (which makes sense).

After I got past that issue, the next one involved writing to standard output from Python. This was consistently screwed up, and I couldn’t figure it out because when I sent the output to Terminal, the encodings were fine. But if I ran the same script either piped to another script, or outputting to stdout from within the script, they were wrong. It turns out that you need the LC_CTYPE environment variable to be set appropriately in order for Python to write your desired encoding to stdout, e.g. (for tcsh):

setenv LC_CTYPE en_US.utf-8

(Substituting en_US.utf-8 for your preferred encoding, of course.)

This last one is specific to OS X. I’m copying to and from the clipboard using pbpaste and pbcopy as well as the usual keyboard shortcuts. When doing that from my scripts, the character encodings would get screwed up. To force OS X to use UTF by default, you need to set defaultStringEncoding, which is done in tcsh with:

setenv __CF_USER_TEXT_ENCODING 0x1F5:0x8000100:0x8000100

Note that 0x1F5 refers to your uid, in this case 501 expressed in hex.

Hopefully those three tips will save some readers a little pain in their scripting.

Tags: coding, python, tech, web-development

Related Posts

• Better reST–WordPress Pipeline Tue 28 Jul 2009
• Blog Workflow with reStructuredText Tue 14 Jul 2009
• Pylons Via Apache Port Issues Fri 12 Jun 2009
• PyWebSF: Meetup for SF-Area Python Web Developers Tue 09 Jun 2009
• Some Vim Script Implementation, Testing, and Hackery Tue 16 Feb 2010
• First Post With Vim Sun 14 Feb 2010
• Better Word Count in Vim Sun 17 Jan 2010
• WordPress 2.9 Upgrade Sun 10 Jan 2010
• GPS, URLs, Math, Python, Featuritis Tue 08 Dec 2009
• addons.mozilla.org Moving from CakePHP to Django Tue 17 Nov 2009