Some Vim Script Implementation, Testing, and Hackery

23:50 Tue 16 Feb 2010. Updated: 00:57 17 Feb 2010
[, , , , , , ]

As a result of my porting over jEdit (Jython) macros to Vim, I now have a fair amount of (Python) Vim scripts, and have learned some things about how to set up those scripts. I’ll go through some of that below, and hopefully other people writing Python scripts for Vim will find it useful.

In jEdit, you invoke macros either by selecting them from a menu (clearly unacceptable) or by opening the Action Bar (Ctrl-Enter, for me) and then typing the name of the macro, where “name” means the non-extension part of the filename. I put my macro files went in a subdirectory of the macros directory, and there were namespace issues—all macro names had to be unique, and also couldn’t match any internal jEdit actions. Furthermore, you couldn’t pass arguments to them. Despite those issues, it was a powerful and quick piece of functionality, and I wanted to make Vim at least match it.

You can put Python directly into Vim script files, but I wanted to do that as minimally as possible. I added a Vim script that would (essentially) include a Python file, and then I had that Python file import from other Python files as necessary. The Python file includes a class, TBase, that contains the functions I want to run; as much code as possible is kept out of those functions and is in other files—files which ideally are individually testable.

To get to those functions from within Vim, I did this:

if filereadable($HOME."/.vim/plugin/tadhg/tadhg.py")
    pyfile $HOME/.vim/plugin/tadhg/tadhg.py

" Create the commands pointing the Python in tadhg.py:
if !exists(":T")
  command! -range -nargs=+ T python tadhgbase = TBase('rs=<line1> rf=<line2>'); tadhgbase('<args>')

nmap <D-CR> :T

That code goes in a Vim script that goes in ~/.vim/plugins/tadhg/; it’s loaded when Vim is. This means that the code in tadhg.py is evaluated at Vim start, so the TBase class is available. Furthermore the user Ex mode command “T” creates a new instance of TBase and then calls it with whatever the user enters after “T”. Finally, Command-Enter is mapped to enter Ex mode and type “T ”, which is quite close indeed to the jEdit functionality I had, but without the disadvantages. I have much more control over the namespace, and I can pass arguments.

When the T command is invoked, a new TBase instance is created and it gets passed the range start and range end line numbers from Vim, which is pretty important for some of the functions. Then the new instance is called, with the argument of whatever was typed after “T ”.

This is the code for handling that latter aspect:

def __call__(self, argstring):
    The first arg is the command; the rest are arguments to that command.
    args = argstring.split(u" ")

    #Need to extract the keyword args out of args
    kwargs = dict([(arg.split("=")[0], arg.split("=")[1]) for arg in s.split(" ") if "=" in arg])
    args = [arg for arg in s.split(" ") if "=" not in arg]

    if args and args[0] in self.__class__.__dict__.keys():
        import types
        f = self.__class__.__dict__[args[0]]
        if type(f) == types.FunctionType:
            if args[1:]:
                f(self, *args[1:])
        print "no command by that name"

In other words, if the first (non-keyword) argument matches the name of a function in the class, run it and pass the rest of the (non-keyword) arguments to it. If I ever need keyword arguments as well, I’ll add the ability to pass them along too.

In order to actually manipulate Vim from these scripts, you have to use import vim. When I started writing this, I wasn’t sure how I’d test that, and I tried to keep the meat of the code in other files, e.g.:

def mpc(self):
    Count the words, then insert them into the document's word count line.
    from write_wordcount import WriteWordcount
    wwc = WriteWordcount(tadhgbase)

(Note that I use import sys; sys.path.append(myscriptdir) at the top of the file to make these imports work.)

One of the advantages of this construction is that WriteWordCount can theoretically be used as a standalone script, on any file, and also that its vim_main method takes vim as an argument. For testing purposes, I had to created a MockVim class, which currently looks like this:

class MockVimBuffer(object):

    def __init__(self):
        self.lines = []

    def __contains__(self, item):
        return item in self.lines

    def __iter__(self):
        return self.lines.__iter__()

    def __getitem__(self, index):
        return self.lines.__getitem__(index)

    def __setitem__(self, index, value):
        return self.lines.__setitem__(index, value)

class MockVimCurrent(object):

    def __init__(self):
        self.__dict__["buffer"] = MockVimBuffer()

    def __setattr__(self, attr, value):
        if attr == "buffer":
            self.__dict__["buffer"].lines = value
            self.__dict__[attr] = value

class MockVim(object):

    def __init__(self):
        self.current = MockVimCurrent()
        self.commands = []

    def eval(self, command):
        command_list = {
            "&ft": lambda: self.rawmodes,
            "exists('b:TotalWordCount')": lambda: False,
            "tvar": lambda: self.tvar,

        return command_list.get(command, lambda: "")()

    def command(self, command):

The eval method in MockVim simply returns whatever was appropriate for various tests I was running; I should change it to return self.command_list.get(command, lambda: "")() instead, and make command_list an instance variable so that the various tests can manipulate it as needed instead of having to put specific commands in the base code.

Because the methods in TBase pass vim along to the instances or methods of the modules they import, testing those modules is relatively simple: in the test, you create a new MockVim object, call it vim, and then pass it along instead.

So everything is great (and testable). Unless, that is, you (meaning me) lapse from full test-driven development discipline and bits and pieces of functionality creep into TBase. Because TBase has that import vim line at the top of it and running it when you’re not actually in Vim produces an ImportError.

The right way to deal with this is to refactor TBase to take vim as one of its initialization arguments, and then have it set that as an instance variable, and then pass that in its methods, while also altering the line in the parent Vim script to:

command! -range -nargs=+ T python import vim; tadhgbase = TBase(vim, 'rs=<line1> rf=<line2>'); tadhgbase('<args>')

That way TBase is much easier to test, since the fake vim can be passed in at test time. However, for a bunch of reasons I didn’t want to do that just yet, partly because it means some reasonably heavy refactoring of pieces of code that I use every day—without tests, because the whole issue here is that TBase isn’t testable. So I wanted a way to make it testable while altering as little of its existing code (which I know currently works, after all) as possible. This took me a while to figure out, and is rather hacky—but it works.

First, in the test file, do this:

from mockvim import MockVim
#Hack a mock Vim into the global namespace so that we can actually test:
import __builtin__
mv = MockVim()
__builtin__.mv = mv
from tadhg import TBase

Then alter the import vim line above the TBase class to instead read:

    import vim
except ImportError:
    import __builtin__
    vim = __builtin__.mv

What this really does is hack a giant global variable, vim, into all of the Python that’s run after those lines in the test file. Normally doing this seems like a rather bad idea, but it’s also rather necessary to handle the less-than-ideal situation I’m dealing with. Once I have the test harness using this hackery running properly, I’ll feel a lot better about switching over to a better architecture.

One Response to “Some Vim Script Implementation, Testing, and Hackery”

  1. Ted Says:

    Hey there, this looks pretty useful. Have you made any more progress on this? If not (or maybe also if so), would you mind if I incorporated your code into a module? I’m thinking it would make sense to build a `mockvim` python module, which would present the same interface as the actual `vim` module. That way it could just be subbed into sys.modules in place of the normal vim one during testing, or be imported via a conditional `import mockvim as vim` statement.

Leave a Reply