Problem tester for coding contests

Rewritten my problem-tester project from scratch in Python 3. Here are the improvements.


The problem with the old solution was that it ran shell-exec to generate the output. That meant debugging wasn’t possible.

Now you can debug your scripts with PyCharm thanks to a feature of Python 3.

With Python 3 it is possible to redirect STDIN and STOUT by doing

fh_in = open(f_in, 'r')
fh_out = open(f_out, 'w')
sys.stdin = fh_in
sys.stdout = fh_out

We now need to include the problem file and run it in a loop, passing “.in” and “out” files. Which wasn’t that simple.

The first challenge was to import a python module dynamically, not knowing its name in advance. Thankfully there is an answer on the SO regarding that, I just needed to adjust it a little.

The next problem was to reload the module and run it freshly for every new input. The best solution I found was to delete it from sys.modules and import again. I’m yet to check if it has any side-effects but so far it looks good.

def run_problem(mod):
    # remove module
    if mod in sys.modules:
        del sys.modules[mod]
    # dynamically import it again. 
    problem = __import__(mod, globals(), locals(), ['main'], 0)
    # if the module contains 'def main' execute it. 
    # If not, we assume the script was in global scope, importing it forced a run
    # Nothing to do
    if 'main' in dir(problem):


Sometimes there may be several solutions to a problem, like here: In this case you can write a script that tests the solution.

Comparing with another solution

If you have a solution that works, you can compare put it with the name It will generate “ok.out” files.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.