我有这个简单的模型:
模型.py
class Ping(models.Model):
online = models.BooleanField()
created = models.DateTimeField(db_index=True, default=timezone.now)
def __str__(self):
return f'{self.online}, {self.created}'
它给了我以下结果:
mysql [lab]> SELECT * FROM myapp_ping;
+----+--------+----------------------------+
| id | online | created |
+----+--------+----------------------------+
| 1 | 1 | 2018-08-02 13:34:09.435292 |
| 2 | 1 | 2018-08-02 13:35:09.520200 |
| 3 | 0 | 2018-08-02 13:36:09.540638 |
| 4 | 0 | 2018-08-02 13:37:10.529783 |
| 5 | 1 | 2018-08-02 13:38:09.779012 |
| 6 | 1 | 2018-08-02 13:39:09.650365 |
| 7 | 1 | 2018-08-02 13:40:09.625543 |
| 8 | 1 | 2018-08-02 13:41:09.892196 |
| 9 | 1 | 2018-08-02 13:42:09.802186 |
| 10 | 1 | 2018-08-02 13:43:09.864551 |
| 11 | 1 | 2018-08-02 13:44:09.960962 |
| 12 | 1 | 2018-08-02 13:45:09.891947 |
| 13 | 0 | 2018-08-02 13:46:09.141727 |
| 14 | 0 | 2018-08-02 13:47:09.142030 |
| 15 | 0 | 2018-08-02 13:48:09.160942 |
| 16 | 0 | 2018-08-02 13:49:09.152879 |
| 17 | 0 | 2018-08-02 13:50:09.280246 |
| 18 | 1 | 2018-08-02 13:51:09.363184 |
| 19 | 1 | 2018-08-02 13:52:09.405863 |
| 20 | 1 | 2018-08-02 13:53:09.403251 |
+----+--------+----------------------------+
20 rows in set (0.00 sec)
有没有办法得到类似这样的输出(online
为 false 的范围):
停机时间:
from | to | duration
2018-08-02 13:36:09 | 2018-08-02 13:37:10 | 1 minute and 1 second
2018-08-02 13:46:09 | 2018-08-02 13:50:09 | 4 minutes and 0 seconds
我不确定这是否可以通过 Django ORM 完成,或者它是否需要原始 MySQL 查询来使用类似 CASE
或 IF
语句?
更新:UTC 2018 年 8 月 8 日星期三 15:13:15
所以我从 @AKX answer 获得了两种解决方案的概念证明:
模型.py
class PingManager(models.Manager):
def downtime_python(self):
queryset = super().get_queryset().filter(created__gt=timezone.now() - timezone.timedelta(days=30))
offline = False
ret = []
for entry in queryset:
if not entry.online and not offline:
offline = True
_ret = {'start': str(entry.created)}
if entry.online and offline:
_ret.update({'end': str(entry.created)})
ret.append(_ret)
offline = False
return ret
def downtime_sql(self):
queryset = super().get_queryset().filter(created__gt=timezone.now() - timezone.timedelta(days=30))
offline = queryset.filter(online=False).order_by('created').first()
last = queryset.order_by('created').last()
ret = []
if offline:
online = queryset.filter(created__gt=offline.created, online=True).order_by('created').first()
ret.append({'start': str(offline.created), 'end': str(online.created)})
while True:
offline = queryset.filter(created__gt=online.created, online=False).order_by('created').first()
if offline:
online = queryset.filter(created__gt=offline.created, online=True).order_by('created').first()
if (online and offline) and online.created < last.created:
ret.append({'start': str(offline.created), 'end': str(online.created)})
continue
else:
break
return ret
class Ping(models.Model):
online = models.BooleanField()
created = models.DateTimeField(db_index=True, default=timezone.now)
objects = PingManager()
def __str__(self):
return f'{self.online}, {self.created}'
问题:
我应该为此创建静态方法还是自定义
manger
是正确的解决方案?如果两个计算都在内存中运行,为什么执行时间会有如此巨大的差异?有没有办法改进并使其更像 Python 等效方法?
测试:
# python manage.py shell
Python 3.6.5 (default, Apr 10 2018, 17:08:37)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.5.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from myapp.models import Ping
In [2]: Ping.objects.downtime_sql()[0]
Out[2]:
{'start': '2018-07-13 16:32:16.009356+00:00',
'end': '2018-07-13 16:33:15.942784+00:00'}
In [3]: Ping.objects.downtime_python()[0]
Out[3]:
{'start': '2018-07-13 16:32:16.009356+00:00',
'end': '2018-07-13 16:33:15.942784+00:00'}
In [4]: Ping.objects.downtime_sql() == Ping.objects.downtime_python()
Out[4]: True
In [5]: import timeit
In [6]: timeit.timeit(stmt=Ping.objects.downtime_python, number=1)
Out[6]: 5.720254830084741
In [7]: timeit.timeit(stmt=Ping.objects.downtime_sql, number=1)
Out[7]: 0.25946347787976265
最佳答案
扩展我的评论:
I'm not sure even SQL case/if statements can get you that result, since the result rows depend on previous rows. This is easy to do procedurally in Python though.
- 最明显的方法是循环遍历
Ping.objects.all()
(或Ping.objects.iterator()
)并跟踪online
变量来形成你想要的“条纹”。这样做的缺点是您确实需要遍历每个对象,这最终会很慢(和/或耗尽您的内存)。 - 一种更复杂的方法,它使用更多的查询但更少的内存,是找到第一个离线的
Ping
对象,然后找到下一个(按时间)Ping
对象再次联机——这将形成一个连胜。然后冲洗并重复此操作,直到用完要检查的Ping
对象。
编辑
是的,这是方法 2 的具体实现(相当优雅,如果你不介意我这么说的话)(在 https://github.com/akx/so51656477 找到完整的测试库):
class PingQuerySet(models.QuerySet):
def streaks(self):
queryset = self.values_list('created', 'online').order_by('created')
entry = queryset.first()
while entry:
next_entry = queryset.filter(created__gt=entry[0], online=(not entry[1])).first()
yield (entry, next_entry)
entry = next_entry
它是元组的二元组生成器:((start_timestamp, start_online), (end_timestamp, end_online) | None)
。
例如,要获取最近 10 天的上涨/下跌或下跌/上涨对,
for start, end in Ping.objects.filter(created__gt=now() - timedelta(days=10)).streaks():
print(start, end)
会打印类似的东西
[...snip...]
(datetime.datetime(2018, 8, 8, 8, 10, 12, 943500), False) (datetime.datetime(2018, 8, 8, 10, 10, 12, 943500), True)
(datetime.datetime(2018, 8, 8, 10, 10, 12, 943500), True) (datetime.datetime(2018, 8, 8, 11, 10, 12, 943500), False)
(datetime.datetime(2018, 8, 8, 11, 10, 12, 943500), False) (datetime.datetime(2018, 8, 8, 11, 40, 12, 943500), True)
(datetime.datetime(2018, 8, 8, 11, 40, 12, 943500), True) (datetime.datetime(2018, 8, 8, 12, 40, 12, 943500), False)
(datetime.datetime(2018, 8, 8, 12, 40, 12, 943500), False) (datetime.datetime(2018, 8, 8, 16, 40, 12, 943500), True)
(datetime.datetime(2018, 8, 8, 16, 40, 12, 943500), True) (datetime.datetime(2018, 8, 8, 17, 40, 12, 943500), False)
(datetime.datetime(2018, 8, 8, 17, 40, 12, 943500), False) (datetime.datetime(2018, 8, 8, 18, 10, 12, 943500), True)
(datetime.datetime(2018, 8, 8, 18, 10, 12, 943500), True) (datetime.datetime(2018, 8, 8, 19, 40, 12, 943500), False)
(datetime.datetime(2018, 8, 8, 19, 40, 12, 943500), False) (datetime.datetime(2018, 8, 8, 23, 10, 12, 943500), True)
(datetime.datetime(2018, 8, 8, 23, 10, 12, 943500), True) (datetime.datetime(2018, 8, 9, 0, 10, 12, 943500), False)
(datetime.datetime(2018, 8, 9, 0, 10, 12, 943500), False) (datetime.datetime(2018, 8, 9, 3, 10, 12, 943500), True)
(datetime.datetime(2018, 8, 9, 3, 10, 12, 943500), True) (datetime.datetime(2018, 8, 9, 3, 40, 12, 943500), False)
(datetime.datetime(2018, 8, 9, 3, 40, 12, 943500), False) (datetime.datetime(2018, 8, 9, 5, 10, 12, 943500), True)
(datetime.datetime(2018, 8, 9, 5, 10, 12, 943500), True) (datetime.datetime(2018, 8, 9, 5, 40, 12, 943500), False)
(datetime.datetime(2018, 8, 9, 5, 40, 12, 943500), False) (datetime.datetime(2018, 8, 9, 7, 10, 12, 943500), True)
(datetime.datetime(2018, 8, 9, 7, 10, 12, 943500), True) None
一些注意事项:
- 最后的
end
值可能是None
,这意味着机器仍然是up或down(取决于start
的status值> 元组)。 - 如果您只关心机器停机的时间,只需忽略
start
元组的状态值为True
的对。 - 由于这是一个生成器,当您有足够的数据时,您可以停止迭代它,它不会再进行任何查询。
- 由于这是一个
QuerySet
扩展方法,您可以根据需要添加其他过滤器(只要它们不在online
上过滤)。例如,如果您有一个host
字段,Ping.objects.filter(host='example.com').streaks()
。
关于python - 带有 MySQL 后端的 Django - 按时间范围分组,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/51656477/