Archive for October 2009
Profiling isn’t often important for my python programs, but when a performance black hole appears out of nowhere it’s very useful for narrowing down the problem, even on smaller scripts.
(Recently I had cause to profile a script manipulating large-ish (<50Mb) files, whose performance had taken a turn for the worse. I had a suspicion that poor list manipulation was to blame, but the results spoke for themselves: 99% of CPU time was spent in the function
list.pop(0)! Based on this I discovered PEP-290 which describes the
collections.deque structure, significantly more efficient than a basic list — the replacement function,
deque.popleft(), barely registers a percentage of overall execution time(!)
I could have discovered this using basic profiling, but whilst playing with the profiler I discovered a very straightforward, easy-to-read python profile visualiser, and thought it worth sharing. It doesn’t do anything very beautiful or complicated, but it’s very quick, provides a neat “squaremap” view of your code performance, and importantly Just Works. RunSnakeRun is its name.