python - 在 Django 中,我如何从我网站上的已知 URL 列表开始,对每个 View 递归地测试所有链接(检查 200 OK)

标签 python django unit-testing testing pytest

在 Django 应用程序中,如果不明确为每个 View 编写单元测试,我如何测试是否没有损坏的链接?

测试应该“浏览”它在站点上找到的所有链接并测试“200 OK”响应。

最佳答案

下面的一段代码被证明对我非常有用。 它立即显示了一些损坏的链接,所以我想在这里分享。

它递归遍历所有链接,从 URL 列表开始,并检查 200 OK 响应。

还可以给出要避免的 URL 列表。

注意事项:

  • 使用 django 1.8.16 和 python 2.7 测试
  • 需要 beatifulsoup4(pip 安装 beautifulsoup4)

开始吧:

from __future__ import print_function
from django.test import TestCase
from bs4 import BeautifulSoup
import re
from django.contrib.auth.models import User

VERBOSE = True


class TraverseLinksTest(TestCase):
    def setUp(self):
        # By default, login as superuser
        self.superuser = User.objects.create_superuser('superuser1', 'superuser1@example.com', 'pwd')
        if self.client.login(username="superuser1@example.com", password="pwd"):
            if VERBOSE: print('\nLogin as superuser OK')
        else:
            raise BaseException('Login failed')

    @classmethod
    def setUpTestData(cls):
        # Initialise your database here as needed
        pass

    def test_traverse_urls(self):
        # Fill these lists as needed with your site specific URLs to check and to avoid
        to_traverse_list = ['/mysite', '/mysite/sub-page']
        to_avoid_list = ['^/$', '^$', 'javascript:history\.back()', 'javascript:history\.go\(-1\)', '^mailto:.*', '.*github\.io.*']

        done_list = []
        error_list = []
        source_of_link = dict()
        for link in to_traverse_list:
            source_of_link[link] = 'initial'

        (to_traverse_list, to_avoid_list, done_list, error_list, source_of_link) = \
            self.recurse_into_path(to_traverse_list, to_avoid_list, done_list, error_list, source_of_link)

        print('END REACHED\nStats:')
        if VERBOSE: print('\nto_traverse_list = ' + str(to_traverse_list))
        if VERBOSE: print('\nto_avoid_list = ' + str(to_avoid_list))
        if VERBOSE: print('\nsource_of_link = ' + str(source_of_link))
        if VERBOSE: print('\ndone_list = ' + str(done_list))
        print('Followed ' + str(len(done_list)) + ' links successfully')
        print('Avoided ' + str(len(to_avoid_list)) + ' links')

        if error_list:
            print('!! ' + str(len(error_list)) + ' error(s) : ')
            for error in error_list:
                print(str(error) + ' found in page ' + source_of_link[error[0]])

            print('Errors found traversing links')
            assert False
        else:
            print('No errors')

    def recurse_into_path(self, to_traverse_list, to_avoid_list, done_list, error_list, source_of_link):
        """ Dives into first item of to_traverse_list
            Returns: (to_traverse_list, to_avoid_list, done_list, source_of_link)
        """

        if to_traverse_list:
            url = to_traverse_list.pop()

            if not match_any(url, to_avoid_list):
                print('Surfing to ' + str(url) + ', discovered in ' + str(source_of_link[url]))
                response = self.client.get(url, follow=True)

                if response.status_code == 200:
                    soup = BeautifulSoup(response.content, 'html.parser')

                    text = soup.get_text()

                    for link in soup.find_all('a'):
                        new_link = link.get('href')
                        if VERBOSE: print('  Found link: ' + str(new_link))
                        if match_any(new_link, to_avoid_list):
                            if VERBOSE: print('    Avoiding it')
                        elif new_link in done_list:
                            if VERBOSE: print('    Already done, ignoring')
                        elif new_link in to_traverse_list:
                            if VERBOSE: print('    Already in to traverse list, ignoring')
                        else:
                            if VERBOSE: print('    New, unknown link: Storing it to traverse later')
                            source_of_link[new_link] = url
                            to_traverse_list.append(new_link)

                    done_list.append(url)
                    if VERBOSE: print('Done')
                else:
                    error_list.append((url, response.status_code))
                    to_avoid_list.append(url)

            if VERBOSE: print('Diving into next level')
            return self.recurse_into_path(to_traverse_list, to_avoid_list, done_list, error_list, source_of_link)

        else:
            # Nothing to traverse
            if VERBOSE: print('Returning to upper level')
            return to_traverse_list, to_avoid_list, done_list, error_list, source_of_link


def match_any(my_string, regexp_list):
    if my_string:
        combined = "(" + ")|(".join(regexp_list) + ")"
        return re.match(combined, my_string)
    else:
        # 'None' as string always matches
        return True

关于python - 在 Django 中,我如何从我网站上的已知 URL 列表开始,对每个 View 递归地测试所有链接(检查 200 OK),我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/43264291/

相关文章:

python - 如何在急切执行中暂停和恢复渐变录音?

django - 如何从 drf-spectacular API 文档中删除 RetrieveUpdateAPIView 中的 put 方法?

unit-testing - 何时以及为什么应该编写参数化的 JUnit 测试用例?

c# - 使用带有单元测试的 ViewModel 进行 CRUD 编辑操作

python - 某些 python 库在生产中的 Django 应用程序中未加载,但在本地服务器中加载正常

python - 创建不会与其他系统冲突的 UUID

python - InvalidArgumentError : logits and labels must be same size logits_size=[80, 2] labels_size=[1,80]

python - 如何更改 PyCharm 中的 django 版本?

Django - 是否应该始终通过任务处理程序(例如 Celery)发出外部 API 请求?

ios - 单元测试中的属性为零