Python 性能分析

python性能调试过程中最突出的问题就是耗时，性能测试工具有很多，像profiler，cprofiler等等，都是只能返回函数整体的耗时，而line_profiler就能够很好解决这个问题

项目地址

https://github.com/rkern/line_profiler

这里记录 line_profiler 的两种用法

通过装饰器 @profile

import random


def do_other_stuff(numbers):
    s = sum(numbers)

@profile
def do_stuff(numbers):
    do_other_stuff(numbers)
    l = [numbers[i] / 43 for i in range(len(numbers))]
    m = ['hello' + str(numbers[i]) for i in range(len(numbers))]


if __name__ == '__main__':
    numbers = [random.randint(1, 100) for i in range(1000)]
    do_stuff(numbers)

这是官方的推荐，但是这里对第一次用这个库的人来说并不友好！步骤如下：

• 给需要分析的函数加上 @profile装饰器（这时第一反应肯定是引入装饰器的依赖，不然代码都报错了，但是实际上进行性能分析不是通过运行代码的，是通过命令行）
• 通过命令 kernprof -l .\23.6.12\speed_test.py 生成 speed_test.py.lprof文件
• 最后通过python 调用line_profiler去分析speed_test.py.lprof，命令

python -m line_profiler .\speed_test.py.lprof

分析结果如下：

Timer unit: 1e-06 s

Total time: 0.0004925 s
File: .\23.6.12\speed_test.py
Function: do_stuff at line 7

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     7                                           @profile
     8                                           def do_stuff(numbers):
     9         1          9.3      9.3      1.9      do_other_stuff(numbers)
    10         1        159.2    159.2     32.3      l = [numbers[i] / 43 for i in range(len(numbers))]
    11         1        324.0    324.0     65.8      m = ['hello' + str(numbers[i]) for i in range(len(numbers))]

逐行显示出了函数的源码及其耗时信息。共有六列信息。

• Line #: 在文件中的行号
• Hits: 该行被执行的次数
• Time: 执行该行的总时间，以定时器为单位表示。在表格前的标题信息中，有一行“Timer unit（定时器单位）”，给出了与秒的换算关系。在不同的系统上可能有所不同。
• Per Hit: 以定时器为单位，执行一次该行的平均时间
• % Time: 该行所花费的时间，在该函数的总记录时间中所占的百分比
• Line Contents: 实际的源码。

实例化LineProfiler

from line_profiler import LineProfiler
import random


def do_other_stuff(numbers):
    s = sum(numbers)


def do_stuff(numbers):
    do_other_stuff(numbers)
    l = [numbers[i] / 43 for i in range(len(numbers))]
    m = ['hello' + str(numbers[i]) for i in range(len(numbers))]


numbers = [random.randint(1, 100) for i in range(1000)]
lp = LineProfiler()
lp_wrapper = lp(do_stuff)
lp_wrapper(numbers)
lp.print_stats()

通过实例化LineProfiler，lp_wrapper = lp(do_stuff) 传入需要分析的函数名，lp_wrapper(numbers)传入需要分析函数的参数，lp.print_stats()最后打印结果，如下：

Timer unit: 1e-07 s

Total time: 0.0005496 s
File: M:/code/example/daily-code/23.6.12/speed_test.py
Function: do_stuff at line 9

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     9                                           def do_stuff(numbers):
    10         1        100.0    100.0      1.8      do_other_stuff(numbers)
    11         1       2180.0   2180.0     39.7      l = [numbers[i] / 43 for i in range(len(numbers))]
    12         1       3216.0   3216.0     58.5      m = ['hello' + str(numbers[i]) for i in range(len(numbers))]

这个只是最简单的使用，当函数有别的函数调用，就不能分辨出具体耗时在哪块，这时可以加入需要分析的函数名称，最终代码如下：

from line_profiler import LineProfiler
import random


def do_other_stuff(numbers):
    s = sum(numbers)


def do_stuff(numbers):
    do_other_stuff(numbers)
    l = [numbers[i] / 43 for i in range(len(numbers))]
    m = ['hello' + str(numbers[i]) for i in range(len(numbers))]


numbers = [random.randint(1, 100) for i in range(1000)]
lp = LineProfiler()
lp.add_function(do_other_stuff)  # add additional function to profile
lp_wrapper = lp(do_stuff)
lp_wrapper(numbers)
lp.print_stats()

分析结果：

Timer unit: 1e-07 s

Total time: 6.3e-06 s
File: M:/code/example/daily-code/23.6.12/speed_test.py
Function: do_other_stuff at line 5

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     5                                           def do_other_stuff(numbers):
     6         1         63.0     63.0    100.0      s = sum(numbers)

Total time: 0.0005201 s
File: M:/code/example/daily-code/23.6.12/speed_test.py
Function: do_stuff at line 9

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     9                                           def do_stuff(numbers):
    10         1        127.0    127.0      2.4      do_other_stuff(numbers)
    11         1       1770.0   1770.0     34.0      l = [numbers[i] / 43 for i in range(len(numbers))]
    12         1       3304.0   3304.0     63.5      m = ['hello' + str(numbers[i]) for i in range(len(numbers))]

我们把do_other_stuff函数加入分析中，line_profiler可以帮我们把do_other_stuff的执行时间也打印出来