python manage.py runserver 的时候发生了什么?

在我们日常用django开发的时候,runserver这个manage命令相信大家一定不陌生, 那么你是否有考虑过, 在命令行输入这个命令的时候到底发生了什么?
下面就请跟着我一步一步剖析django的源代码, 看看它到底做了什么…

manage.py

经常用django开发的同学对这个文件一定不陌生, 它是我们运行django-admin startproject 的时候自动生成的. 在这边文章中, 我们不会去讨论django是怎么做到自动生成类似模版文件一样的项目代码的,我们关注的重点是运行 python manage.py runserver 后到底发生了什么.

入口程序

让我们打开我们的manage.py 文件, 你可能会看到以下内容(注:本文示例采用django2.0):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "taskmaster.settings")
try:
from django.core.management import execute_from_command_line
except ImportError as exc:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
) from exc
execute_from_command_line(sys.argv)

程序运行的第一件事就是设置了一个环境变量, 指定了django项目的settings文件, 除去异常处理外, 就到了我们最关键的入口 execute_from_command_line(sys.argv)
如果你常规的运行runserver命令不附带任何其他参数, 此时的sys.argv 应该是 [manage.py文件绝对路径, 'runserver'], 它被作为参数传入了execute_from_command_line这个函数。

execute_from_command_line

下面就让我们看看这个函数到底干了啥。
它一共就两行代码- -

1
2
3
4
def execute_from_command_line(argv=None):
"""Run a ManagementUtility."""
utility = ManagementUtility(argv)
utility.execute()

这个函数只负责创建一个公共设施之类的东西, 它实例化了一个ManagementUtility对象, 它初始化了方法如下:

1
2
3
4
5
6
def __init__(self, argv=None):
self.argv = argv or sys.argv[:]
self.prog_name = os.path.basename(self.argv[0])
if self.prog_name == '__main__.py':
self.prog_name = 'python -m django' # 目前来说这么做的意义为何还不明朗, 希望有知道同学可以解答我的疑惑
self.settings_exception = None

实例化完成了, 它马上调用了它的execute()方法, 顾名思义, 执行方法。
看来我们今天要探索的主角就是它了

execute()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
def execute(self):
"""
Given the command-line arguments, figure out which subcommand is being
run, create a parser appropriate to that command, and run it.
"""
try:
subcommand = self.argv[1]
except IndexError:
subcommand = 'help' # Display help if no arguments were given.

# Preprocess options to extract --settings and --pythonpath.
# These options could affect the commands that are available, so they
# must be processed early.
parser = CommandParser(None, usage="%(prog)s subcommand [options] [args]", add_help=False)
parser.add_argument('--settings')
parser.add_argument('--pythonpath')
parser.add_argument('args', nargs='*') # catch-all
try:
options, args = parser.parse_known_args(self.argv[2:])
handle_default_options(options)
except CommandError:
pass # Ignore any option errors at this point.

try:
settings.INSTALLED_APPS
except ImproperlyConfigured as exc:
self.settings_exception = exc

if settings.configured:
# Start the auto-reloading dev server even if the code is broken.
# The hardcoded condition is a code smell but we can't rely on a
# flag on the command class because we haven't located it yet.
if subcommand == 'runserver' and '--noreload' not in self.argv:
try:
autoreload.check_errors(django.setup)()
except Exception:
# The exception will be raised later in the child process
# started by the autoreloader. Pretend it didn't happen by
# loading an empty list of applications.
apps.all_models = defaultdict(OrderedDict)
apps.app_configs = OrderedDict()
apps.apps_ready = apps.models_ready = apps.ready = True

# Remove options not compatible with the built-in runserver
# (e.g. options for the contrib.staticfiles' runserver).
# Changes here require manually testing as described in
# #27522.
_parser = self.fetch_command('runserver').create_parser('django', 'runserver')
_options, _args = _parser.parse_known_args(self.argv[2:])
for _arg in _args:
self.argv.remove(_arg)

# In all other cases, django.setup() is required to succeed.
else:
django.setup()

self.autocomplete()

if subcommand == 'help':
if '--commands' in args:
sys.stdout.write(self.main_help_text(commands_only=True) + '\n')
elif len(options.args) < 1:
sys.stdout.write(self.main_help_text() + '\n')
else:
self.fetch_command(options.args[0]).print_help(self.prog_name, options.args[0])
# Special-cases: We want 'django-admin --version' and
# 'django-admin --help' to work, for backwards compatibility.
elif subcommand == 'version' or self.argv[1:] == ['--version']:
sys.stdout.write(django.get_version() + '\n')
elif self.argv[1:] in (['--help'], ['-h']):
sys.stdout.write(self.main_help_text() + '\n')
else:
self.fetch_command(subcommand).run_from_argv(self.argv)

有的同学一看到这, 心里马上冒出三个字: 打扰了..
这很正常, 我看到这么长的代码也有点慌, 生怕自己看不懂, 但是我们一定要知道一件事, 任何复杂的系统都是由一个个简单的组建组成的, 如果你还没有全局的概念就试图记忆细节,那么学习就会陷入僵局
不信你仔细看, 去掉英文注释, 代码量就那么一点点, 也是相对好理解的。让我们来尝试自己加注释, 一步步把它解释下来。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
def execute(self):
"""
给定命令行参数,找出正在使用的子命令,创建一个适合该命令的解析器,然后运行它。
"""
# 第一步, 找到子命令名称 本例子中是runserver
try:
subcommand = self.argv[1] # subcommand: 'runserver'
except IndexError:
subcommand = 'help' # 展示help选项, 如果没有指令传进来

# 用于提取的预处理选项--settings和--pythonpath。这些选项可能会影响可用的命令,因此它们必须尽早处理。
parser = CommandParser(None, usage="%(prog)s subcommand [options] [args]", add_help=False) # CommandParser只是对argparse里ArgumentParser做了简单封装
parser.add_argument('--settings')
parser.add_argument('--pythonpath')
parser.add_argument('args', nargs='*') # 捕获剩余的其他参数
try:
options, args = parser.parse_known_args(self.argv[2:]) # 返回一个命名空间和其他参数, 你可能注意到了值得就是--settings和--pythonpath
handle_default_options(options) # 如果有这两项参数的
except CommandError:
pass # 忽略任何错误的设置

try:
settings.INSTALLED_APPS
except ImproperlyConfigured as exc: # 如果INSTALLED_APPS为空会抛出这个异常, 具体查看LazyObject的_setup()方法
self.settings_exception = exc

if settings.configured:
#即使代码被破坏,也要启动自动重新加载开发服务器。硬编码的条件是代码味道不对,但我们不能依赖于在命令类上标记,因为我们还没有找到它
if subcommand == 'runserver' and '--noreload' not in self.argv:
try:
autoreload.check_errors(django.setup)() # 检查错误
except Exception:
# 稍后将在autoreloader启动的子进程中引发异常。通过加载空的应用程序列表假装它没有发生。
apps.all_models = defaultdict(OrderedDict)
apps.app_configs = OrderedDict()
apps.apps_ready = apps.models_ready = apps.ready = True

#删除与内置runserver不兼容的选项, 例如contrib.staticfiles的runserver的选项。此处的更改需要手动测试
_parser = self.fetch_command('runserver').create_parser('django', 'runserver')
_options, _args = _parser.parse_known_args(self.argv[2:])
for _arg in _args:
self.argv.remove(_arg)

# 如果是其它选项,则单纯的setup()Django。
else:
django.setup()

self.autocomplete()

if subcommand == 'help':
if '--commands' in args:
sys.stdout.write(self.main_help_text(commands_only=True) + '\n')
elif len(options.args) < 1:
sys.stdout.write(self.main_help_text() + '\n')
else:
self.fetch_command(options.args[0]).print_help(self.prog_name, options.args[0])
# 特殊例子:我们想要'django-admin --version'和'django-admin --help'能够被使用,以实现向后兼容。
elif subcommand == 'version' or self.argv[1:] == ['--version']:
sys.stdout.write(django.get_version() + '\n')
elif self.argv[1:] in (['--help'], ['-h']):
sys.stdout.write(self.main_help_text() + '\n')
else:
self.fetch_command(subcommand).run_from_argv(self.argv)
# 最后在此调用了runserver的run方法

run_from_argv

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
def run_from_argv(self, argv):
"""
Set up any environment changes requested (e.g., Python path
and Django settings), then run this command. If the
command raises a ``CommandError``, intercept it and print it sensibly
to stderr. If the ``--traceback`` option is present or the raised
``Exception`` is not ``CommandError``, raise it.
"""
self._called_from_command_line = True
parser = self.create_parser(argv[0], argv[1])

options = parser.parse_args(argv[2:])
cmd_options = vars(options)
# Move positional args out of options to mimic legacy optparse
args = cmd_options.pop('args', ())
handle_default_options(options)
try:
self.execute(*args, **cmd_options) # 又回到子命令的执行函数
except Exception as e:
if options.traceback or not isinstance(e, CommandError):
raise

# SystemCheckError takes care of its own formatting.
if isinstance(e, SystemCheckError):
self.stderr.write(str(e), lambda x: x)
else:
self.stderr.write('%s: %s' % (e.__class__.__name__, e))
sys.exit(1)
finally:
try:
connections.close_all()
except ImproperlyConfigured:
# Ignore if connections aren't setup at this point (e.g. no
# configured settings).
pass

runserver command 的 execute

1
2
3
4
5
6
7
def execute(self, *args, **options):
if options['no_color']:
# We rely on the environment because it's currently the only
# way to reach WSGIRequestHandler. This seems an acceptable
# compromise considering `runserver` runs indefinitely.
os.environ["DJANGO_COLORS"] = "nocolor"
super().execute(*args, **options) # 做了一层简单的判断 就调用了父类的execute

BaseCommand execute

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
def execute(self, *args, **options):
"""
Try to execute this command, performing system checks if needed (as
controlled by the ``requires_system_checks`` attribute, except if
force-skipped).
"""
if options['no_color']:
self.style = no_style()
self.stderr.style_func = None
if options.get('stdout'):
self.stdout = OutputWrapper(options['stdout'])
if options.get('stderr'):
self.stderr = OutputWrapper(options['stderr'], self.stderr.style_func)

saved_locale = None
if not self.leave_locale_alone:
# Deactivate translations, because django-admin creates database
# content like permissions, and those shouldn't contain any
# translations.
from django.utils import translation
saved_locale = translation.get_language()
translation.deactivate_all()

try:
if self.requires_system_checks and not options.get('skip_checks'):
self.check()
if self.requires_migrations_checks:
self.check_migrations()
output = self.handle(*args, **options) # 这句是关键
if output:
if self.output_transaction:
connection = connections[options.get('database', DEFAULT_DB_ALIAS)]
output = '%s\n%s\n%s' % (
self.style.SQL_KEYWORD(connection.ops.start_transaction_sql()),
output,
self.style.SQL_KEYWORD(connection.ops.end_transaction_sql()),
)
self.stdout.write(output)
finally:
if saved_locale is not None:
translation.activate(saved_locale)
return output

runserver command 的 handle

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
def get_handler(self, *args, **options):
"""Return the default WSGI handler for the runner."""
return get_internal_wsgi_application()

def handle(self, *args, **options):
from django.conf import settings

if not settings.DEBUG and not settings.ALLOWED_HOSTS:
raise CommandError('You must set settings.ALLOWED_HOSTS if DEBUG is False.')

self.use_ipv6 = options['use_ipv6']
if self.use_ipv6 and not socket.has_ipv6:
raise CommandError('Your Python does not support IPv6.')
self._raw_ipv6 = False
if not options['addrport']:
self.addr = ''
self.port = self.default_port
else:
m = re.match(naiveip_re, options['addrport'])
if m is None:
raise CommandError('"%s" is not a valid port number '
'or address:port pair.' % options['addrport'])
self.addr, _ipv4, _ipv6, _fqdn, self.port = m.groups()
if not self.port.isdigit():
raise CommandError("%r is not a valid port number." % self.port)
if self.addr:
if _ipv6:
self.addr = self.addr[1:-1]
self.use_ipv6 = True
self._raw_ipv6 = True
elif self.use_ipv6 and not _fqdn:
raise CommandError('"%s" is not a valid IPv6 address.' % self.addr)
if not self.addr:
self.addr = self.default_addr_ipv6 if self.use_ipv6 else self.default_addr
self._raw_ipv6 = self.use_ipv6
self.run(**options)

def run(self, **options):
"""Run the server, using the autoreloader if needed."""
use_reloader = options['use_reloader']

if use_reloader:
autoreload.main(self.inner_run, None, options)
else:
self.inner_run(None, **options)

def inner_run(self, *args, **options):
# If an exception was silenced in ManagementUtility.execute in order
# to be raised in the child process, raise it now.
autoreload.raise_last_exception()

threading = options['use_threading']
# 'shutdown_message' is a stealth option.
shutdown_message = options.get('shutdown_message', '')
quit_command = 'CTRL-BREAK' if sys.platform == 'win32' else 'CONTROL-C'

self.stdout.write("Performing system checks...\n\n")
self.check(display_num_errors=True)
# Need to check migrations here, so can't use the
# requires_migrations_check attribute.
self.check_migrations()
now = datetime.now().strftime('%B %d, %Y - %X')
self.stdout.write(now)
self.stdout.write((
"Django version %(version)s, using settings %(settings)r\n"
"Starting development server at %(protocol)s://%(addr)s:%(port)s/\n"
"Quit the server with %(quit_command)s.\n"
) % {
"version": self.get_version(),
"settings": settings.SETTINGS_MODULE,
"protocol": self.protocol,
"addr": '[%s]' % self.addr if self._raw_ipv6 else self.addr,
"port": self.port,
"quit_command": quit_command,
})

try:
handler = self.get_handler(*args, **options)
run(self.addr, int(self.port), handler,
ipv6=self.use_ipv6, threading=threading, server_cls=self.server_cls)
except socket.error as e:
# Use helpful error messages instead of ugly tracebacks.
ERRORS = {
errno.EACCES: "You don't have permission to access that port.",
errno.EADDRINUSE: "That port is already in use.",
errno.EADDRNOTAVAIL: "That IP address can't be assigned to.",
}
try:
error_text = ERRORS[e.errno]
except KeyError:
error_text = e
self.stderr.write("Error: %s" % error_text)
# Need to use an OS exit because sys.exit doesn't work in a thread
os._exit(1)
except KeyboardInterrupt:
if shutdown_message:
self.stdout.write(shutdown_message)
sys.exit(0)

一层套一层,最终调用了下面这个方法

run()

层层深入发现, runserver方法最后调用了以下函数

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def run(addr, port, wsgi_handler, ipv6=False, threading=False, server_cls=WSGIServer):
server_address = (addr, port)
if threading:
httpd_cls = type('WSGIServer', (socketserver.ThreadingMixIn, server_cls), {})
else:
httpd_cls = server_cls
httpd = httpd_cls(server_address, WSGIRequestHandler, ipv6=ipv6)
if threading:
# ThreadingMixIn.daemon_threads indicates how threads will behave on an
# abrupt shutdown; like quitting the server by the user or restarting
# by the auto-reloader. True means the server will not wait for thread
# termination before it quits. This will make auto-reloader faster
# and will prevent the need to kill the server manually if a thread
# isn't terminating correctly.
httpd.daemon_threads = True
httpd.set_app(wsgi_handler)
httpd.serve_forever()

也就是我们平时开发最常使用的单线程服务器(注意这个单线程, 这导致了内置的内存缓存本地生效生产环境不生效的问题, 因为生产环境是多线程的- -)

结束

好啦今天的分享就到这了, 再分享一句昨天看到的很不错的话。

“完美主义是压迫者的声音,是人们的敌人。它会束缚你的想法,毁掉你的生命,同时它也会妨碍你创建较差的草稿初案。我认为完美主义基于一种强迫性的想法:如果你足够细致,每件事情都做得很好,那你就不会失败。但事实是,无论怎么做你都有可能会失败,可是很多人即使不太仔细也会做得比你好,而且其间也会拥有更多欢乐。”

不要追求完美主义, 那是跟自己故意不去, 一点点的进去, 不断的大胆尝试, 错误, 改正, 重复这一切, 这才是最适合我们程序员的学习方式。 共勉!

ojbk