Thoughts @ Dotner

The Tracker resides now on a Rackspace server in the U.K. Should you notice any problems, please contact me.

If your apache2 process suddenly crashes and refuses to restart, you can explore the option of semaphore resources running out. When I have such a situation, my /var/log/apache2/error.log gets a line like this at every start attempt:

[Sun Aug 28 06:25:20 2011] [crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock
Configuration Failed

Check how many semaphores are in use:

ipcs -s | wc -l

In case the number seems high, you can free up all the semaphores using the following command in bash:

for x in `ipcs -s | awk ‘{print $2}’ `; do ipcrm -s $x; done

I wanted to use my displayless Linux server for torrent downloading. However, I’m not using X as a remote desktop for the box, so I wanted to have a web GUI where I could add torrents and have a script browse RSS feeds automatically for new episodes of my favourite shows. I came up with Aria2c, which has an XML interface for managing torrents.

So I created a python script which reads a feeds.txt file for show names and fetches the listings from eztv.it and if any new episodes are available, sends the torrents to Aria2c for downloading. Then I made a django app, which provides GUI access for adding torrents and monitoring the download queue. Items can also be removed from the queue but there are no other functionality at this time.

If you’re interested in the code and if you’d even like to start developing it with me, get in contact through Dotner.

Check out the ultimately cool mind mapping web application called Mindmeister. It gives you roughly the same functionality as MindManager (minus visual templates), but enables collaboration and has a quite responsive user interface.

The following changes have been deployed in production today.

New Features

• The user can specify a default phase for a task. The information is used when people are making time entries regarding the task.

Bug Fixes

• Extra backslashes were appearing in task and project titles and descriptions when items were edited. This should not happen anymore.
• Removed an ajax related warning message when creating projects or tasks.

The following fixes have been deployed in production today.

• Phase selection added to the time entry dialog.
• Priority and deadline can be left empty when editing project details.

Also, the code should load faster now. If you notice any difference, I’d like to hear about it.

This release was deployed in production on March 10th, 2009.

New Features

• When making time entries, you can specify the phase you are working on, if applicable. The phases can be for example:
  • Plan
  • Implement
  • Test
  • Fix
  • Refactor
  • Document
• Absence requests are shown on the dashboard for admins.

Improved Usability

• Form validation improved.
• Tentative absence is shown in the journal with a dashed border.

Security

• Eliminated XSS (Cross-Site Scripting) possibilities.

This Tracker version will become available for production use during February 2009.

The new features and functionality

• Navigation buttons for journal.
• Time entry creation via my journal is no longer listing the closed tasks.
• You can now comment on tasks and projects.
• Comments are shown in task and project detail page in reverse chronological order.
• Latest 5 comments are shown on the dashboard in reverse chronological order.
• When removing projects, you can now either drag them to the Archive folder like before, or if the project is empty, just delete it.
• The time entry form (the one you get by pressing the clock icon) accepts now the regular “1h 30m” format strings instead of the obscure “1.5″ format.
• Task and project priority mechanism works in a new way.
• Multiple items can have same priority.
• Items are listed in priority order in the sub project contents.
• Project search enables the user to list sub projects and tasks by author, assignee, priority and status.
• Descriptions and comments support the textile markup.

Bug fixes

• Inputing an absence entry without choosing a task gives now an error message.
• IE rendering is at least slightly improved in various views.
• Duplicate entries on active task list are handled gracefully.

Visual and usability improvements

• New login screen.
• Hover tips for image buttons.
• Task links show the project path in the hover tip.
• Tabs for report views for choosing the month and user.
• Rounded corners for tabs and headings on Firefox browsers.
• New soothing button design for task and project detail views.
• User can open the project details, search and comments from behind a tab. This saves space and decreases clutter.

Some earlier features I may not have mentioned much anywhere

• Support for OpenID authentication.
• The user can specify a trusted OpenID URL, which the user can use for logging into the account.
• The administrator can disable this feature for the whole system.
• Support for Yubikey authentication.
• The user can specify which physical Yubikey token the user can use for logging into Tracker.
• When logging in, the user inputs their user ID and triggers a one-time password from the Yubikey token.

Riddle me this; What is the worlds smallest keyboard, with only one key, which gives you a different result every time you press it? It’s Yubikey from Yubico. Yubikey is an authentication token, which acts as a USB keyboard. You can use the Yubikey to output a one-time password to a password field of a system which supports this.

Check it out, it’s cool.

Django and python make it a breeze to build and run automatic regression tests. In 1-2 days you can easily build 50 test cases which cover the basic functionality of your application, and make sure no view or model has been broken. The test cases can be divided into three groups:

• Tests for models
• Tests for views
• Tests for various utility functions

Here’s how to do it. You start by adding a tests.py file in your application directory. The Django default test runner will execute the runTest() method for every TestCase derived class when you run manage.py test. Here’s an example tests.py. Django has specific test functions to log you in and out of the standard authentication framework, but this example is using it’s own authentication functionality.

from django.test import *
import doctest

class ViewTests(TestCase):
    def runTest(self):

        c = Client()
        # Try to get private content prior to login
        response = c.get('/projects/')
        self.assertRedirects(response, '/login/?next=/projects/')

        # Attempt login using the wrong password
        response = c.post('/login/',
            {'login': 'eki', 'pass': 'wrongpassword'})
        self.assertContains(response,
            'Please check your user ID and password')

        # Use proper credentials
        response = c.post('/login/?next=/projects/',
            {'login': 'eki', 'pass': 'helloworld'})
        self.assertRedirects(response, '/projects/')

        # Now fetch each of the basic pages
        response = c.get('/projects/')
        self.assertContains(response, 'Example project')
        response = c.get('/users/')
        self.assertContains(response, 'Erkki Tapola')

This is the mechanism for testing views. Next, you can add doctests to your models.py. Doctest is a very handy notation for test cases to be executed from the command line python. Essentially you just run a test case on the command line and cut and paste the command and it’s output to your class or function documentation. For example:

def get_base_dir(path):
    """
    >>> get_base_dir('/first/second')
    'first'
    >>> get_base_dir('first/second')
    'second'
    """
    items = string.split(path, '/')
    return items[1]

The Django test runner runs doctests from models.py automatically, but if you want it to execute doctest cases from other files than models.py as well, you can add a TestCase class to tests.py for that purpose:

class ModuleTests(TestCase):
    def runTest(self):
        def mytestmod(m):
            result = doctest.testmod(m)
            print str(result[1]) + ' test cases ran in ' + str(m)
            self.assertEqual(result[0], 0)

        print "\nRunning test cases for individual modules\n"
        print 80*'*'

        for x in [tracker.main.util, tracker.main.projects]:
            mytestmod(x)

        print 80*'*' + "\n"

When you’re finished with your test cases and able to run them all using manage.py test, you can connect your cool new test suite to svn using the pre-commit hook. This way you can make sure that the code changes are verified prior to commit. Here’s what you do on Linux:

• Go to hooks/ which is under your SVN repository directory and rename the start-commit.tmpl to start-commit
• Edit the file, and before the last exit 0, add the line:
for x in `find . -iname manage.py -exec dirname {} \;`; do cd $x; if ! ./manage.py test; then exit 1; fi; done

I should probably write this in python to make it platform independent. This unix shell command will find any manage.py from the subdirectories of your current location and use it for running tests. If any manage.py returns an error, the commit will be cancelled. This addition will not influence projects which doesn’t contain a manage.py, but if there is one, then the tests must pass.

A similar hooking mechanism exists in cvs and probably most other version control systems.

You can check out the article on Testing Django
applications for a comprehensive tutorial.