Tag Archives: profiling

Python Profiling

Profiling isn’t often important for my python programs, but when a performance black hole appears out of nowhere it’s very useful for narrowing down the problem, even on smaller scripts.

(Recently I had cause to profile a script manipulating large-ish (<50Mb) files, whose performance had taken a turn for the worse. I had a suspicion that poor list manipulation was to blame, but the results spoke for themselves: 99% of CPU time was spent in the function list.pop(0)! Based on this I discovered PEP-290 which describes the collections.deque structure, significantly more efficient than a basic list — the replacement function, deque.popleft(), barely registers a percentage of overall execution time(!)

I could have discovered this using basic profiling, but whilst playing with the profiler I discovered a very straightforward, easy-to-read python profile visualiser, and thought it worth sharing. It doesn’t do anything very beautiful or complicated, but it’s very quick, provides a neat “squaremap” view of your code performance, and importantly Just Works. RunSnakeRun is its name.

Continue reading