Wagtail Response Checker

Why?

As with any software It's a good idea to keep the version you are using as up to date as possible. Wagtail has a frequent release policy in place: https://docs.wagtail.org/en/stable/releases/upgrading.html# Some releases change how features you may already be using should be implemented. Always follow the upgrading documentation for the release you are upgrading to.

However, confirming all is well can be a challenge and finding pieces of a site that aren't working is hard. For example checking every content type in the admin and frontend still works as expected can be very time consuming.

I created this management command that can be run in development.

It will find all content types (Pages, Snippets, Settings, Images, Documents) and output a report in the console for each content type. You will need to update registered_modeladmin to suit your own app.

When running the command you'll need to specify 2 params (a username and a password), create this with python manage.py createsuperuser

The report will assume http://localhost:8000 is to be used in the report. If you see an error the url for the error should be available to copy and paste into a browser for further investigation. You can override the url value to use in the console by passing in --report-url option.

# app-name/management/commands/report_responses.py

import requests
from django.apps import apps
from django.conf import settings
from django.core.management.base import BaseCommand
from wagtail.admin.admin_url_finder import AdminURLFinder
from wagtail.admin.utils import get_admin_base_url
from wagtail.contrib.settings.registry import registry as settings_registry
from wagtail.documents import get_document_model
from wagtail.images import get_image_model
from wagtail.models import get_page_models
from wagtail.snippets.models import get_snippet_models


class Command(BaseCommand):
    """
    Checks the admin and frontend responses for models incl pages, snippets, settings and modeladmin.

    The command is only available in DEBUG mode. Set DEBUG=True in your settings to enable it.

    Basic usage:
        python manage.py report_responses <username> <password>

    Options:

        --host
            The URL to check. Defaults to the value of ADMIN_BASE_URL in settings.

        --report-url
            The URL to use for the report. e.g. http://staging.example.com

    Example:
        python manage.py report_responses <username> <password> \
            --report-url http://staging.example.com

        This will alter the displayed URLs in the report but the tested URL will still
        use the --host option.
    """

    help = "Checks the admin and frontend responses for models including pages, snippets, settings and modeladmin."

    checked_url = None
    report_url = None
    report_lines = []

    registered_modeladmin = [
        # add model admin models as they cannot be auto detected. For example ...
        "events.EventType",
    ]

    def add_arguments(self, parser):
        parser.add_argument(
            "username",
            help="The username to use for login",
        )
        parser.add_argument(
            "password",
            help="The password to use for login",
        )
        parser.add_argument(
            "--host",
            default=get_admin_base_url(),
            help="The URL to check",
        )
        parser.add_argument(
            "--report-url",
            help="The URL to use for the report. e.g. http://staging.example.com",
        )

    def handle(self, *args, **options):
        # Check if the command is enabled in settings
        if not settings.DEBUG:
            self.out_message_error(
                "Command is only available in DEBUG mode. Set DEBUG=True in your settings to enable it."
            )
            return

        self.checked_url = options["host"]
        self.report_url = (
            options["report_url"].strip("/") if options["report_url"] else None
        )

        with requests.Session() as session:
            url = f"{options['host']}/admin/login/"

            try:
                session.get(url)
            except requests.exceptions.ConnectionError:
                self.out_message_error(
                    f"Could not connect to {options['host']}. Is the server running?"
                )
                return
            except requests.exceptions.InvalidSchema:
                self.out_message_error(
                    f"Could not connect to {options['host']}. Invalid schema"
                )
                return
            except requests.exceptions.MissingSchema:
                self.out_message_error(
                    f"Could not connect to {options['host']}. Missing schema"
                )
                return

            # Attempt to log in
            logged_in = session.post(
                url,
                data=dict(
                    username=options["username"],
                    password=options["password"],
                    csrfmiddlewaretoken=session.cookies["csrftoken"],
                    next="/admin/",
                ),
            ).content

            if "Forgotten password?" in logged_in.decode("utf-8"):
                # Login failed because the response isn't the Dashboard page
                self.out_message_error(
                    f"Could not log in to {options['host']}. Is the username and password correct?"
                )
                return

            # Reports
            self.report_admin_home(session, options)
            self.report_page(session, options)
            self.report_snippets(session, options)
            self.report_modeladmin(session, options)
            self.report_settings_models(session, options)
            self.report_documents(session, options)
            self.report_images(session, options)

    def report_admin_home(self, session, options):
        self.out_message_info("\nChecking the admin home page (Dashboard) ...")

        admin_home_resp = session.get(f"{options['host']}/admin/")

        if admin_home_resp.status_code == 200:
            message = "\nAdmin home page ↓"
            self.out_message(message)
            self.out_message_success(f"{options['host']}/admin/ ← 200")
        else:
            message = "\nAdmin home page ↓"
            self.out_message(message)
            self.out_message_error(
                f"{options['host']}/admin/ ← {admin_home_resp.status_code}"
            )

    def report_page(self, session, options):
        page_models = self.filter_page_models(get_page_models())

        model_index = []
        results = []

        for page_model in page_models:
            if item := page_model.objects.first():
                model_index.append(item.__class__.__name__)
                results.append(
                    {
                        "title": item.title,
                        "url": f"{options['host']}{item.url}",
                        "id": item.id,
                        "editor_url": f"{self.get_admin_edit_url(options, item)}",
                        "class_name": item.__class__.__name__,
                    }
                )

        # Print the index
        message = f"\nChecking the admin and frontend responses of {len(results)} page types ..."
        self.out_message_info(message)

        for count, content_type in enumerate(sorted(model_index)):
            message = (
                f" {count + 1}. {content_type}"
                if count <= 8  # Fixup the index number alignment
                else f"{count + 1}. {content_type}"
            )
            self.out_message(message)

        # Print the results
        for page in results:
            message = f"\n{page['title']} ( {page['class_name']} ) ↓"
            self.out_message(message)

            # Check the admin response
            response = session.get(page["editor_url"])
            if response.status_code != 200:
                self.out_message_error(f"{page['editor_url']} ← {response.status_code}")
            else:
                self.out_message_success(f"{page['editor_url']} ← 200")

            # Check the frontend response
            response = session.get(page["url"])
            if response.status_code == 200:
                self.out_message_success(f"{page['url']} ← 200")
            else:
                if response.status_code == 404:
                    message = (
                        f"{page['url']} ← {response.status_code} probably a draft page"
                    )
                    self.out_message_warning(message)
                else:
                    self.out_message_error(f"{page['url']} ← {response.status_code}")

    def report_snippets(self, session, options):
        self.out_message_info("\nChecking all SNIPPETS models edit pages ...")

        snippet_models = get_snippet_models()
        self.out_models(session, options, snippet_models)

    def report_modeladmin(self, session, options):
        self.out_message_info("\nChecking all MODELADMIN edit pages ...")

        modeladmin_models = []
        for model in apps.get_models():
            app = model._meta.app_label
            name = model.__name__
            if f"{app}.{name}" in self.registered_modeladmin:
                modeladmin_models.append(apps.get_model(app, name))

        self.out_models(session, options, modeladmin_models)

    def report_settings_models(self, session, options):
        self.out_message_info("\nChecking all SETTINGS edit pages ...")
        self.out_models(session, options, settings_registry)

    def report_documents(self, session, options):
        self.out_message_info("\nChecking the DOCUMENTS edit page ...")

        document_model = get_document_model()
        self.out_models(session, options, [document_model])

    def report_images(self, session, options):
        self.out_message_info("\nChecking the IMAGES edit page ...")

        image_model = get_image_model()
        self.out_models(session, options, [image_model])

    def out_models(self, session, options, models):
        for model in models:
            obj = model.objects.first()
            if not obj:
                # settings model has no objects
                continue

            url = self.get_admin_edit_url(options, obj)

            message = f"\n{model._meta.verbose_name.capitalize()} ↓"
            self.out_message(message)

            response = session.get(url)

            if response.status_code == 200:
                self.out_message_success(f"{url} ← 200")
            else:
                self.out_message_error(f"{url} ← {response.status_code}")

    def out_message(self, message):
        if self.report_url:
            message = message.replace(self.checked_url, self.report_url)
        if message not in self.report_lines:
            self.report_lines.append(message)
        self.stdout.write(message)

    def out_message_info(self, message):
        if self.report_url:
            message = message.replace(self.checked_url, self.report_url)
        if message not in self.report_lines:
            self.report_lines.append(message)
        self.stdout.write(self.style.HTTP_INFO(message))
        self.stdout.write("=" * len(message))

    def out_message_error(self, message):
        if self.report_url:
            message = message.replace(self.checked_url, self.report_url)
        if message not in self.report_lines:
            self.report_lines.append(message)
        self.stderr.write(self.style.ERROR(message))

    def out_message_success(self, message):
        if self.report_url:
            message = message.replace(self.checked_url, self.report_url)
        if message not in self.report_lines:
            self.report_lines.append(message)
        self.stdout.write(self.style.SUCCESS(message))

    def out_message_warning(self, message):
        if self.report_url:
            message = message.replace(self.checked_url, self.report_url)
        if message not in self.report_lines:
            self.report_lines.append(message)
        self.stdout.write(self.style.WARNING(message))

    @staticmethod
    def filter_page_models(page_models):
        """Filter out page models that are not creatable or are in the core apps."""

        filtered_page_models = []

        for page_model in page_models:
            if page_model._meta.app_label == "wagtailcore":
                # Skip the core apps
                continue
            if not page_model.is_creatable:
                # Skip pages that can't be created
                continue
            filtered_page_models.append(page_model)

        return filtered_page_models

    @staticmethod
    def get_admin_edit_url(options, obj):
        admin_url_finder = AdminURLFinder()
        return f"{options['host']}{admin_url_finder.get_edit_url(obj)}"
python
note code Copy code

Example usage scenario

Wagtail 3 has had some extensive changes over previous verions: Documentation

Mostly, if you miss renaming an import it can be seen quite quickly when running the site. But some other changes can be hard to spot.

For example:

class CustomHelpPanel(EditHandler):
  template = 'toolkits/custom_help_panel.html'

  def render(self):
    return mark_safe(render_to_string(self.template, {
        'self': self,
        'title': self.form.parent_page.title
      }))
python
note code Copy code

If you have one page model that all pages use/inherit from where the above is used like so:

content_panels = [
  CustomHelpPanel()
] + Page.content_panels
python
note code Copy code

Won't show up as an error until you try to edit the page it's used on.

But you might just have 1000's of pages to potentially check across many different page models.

report_responses Command

The command can be dropped into any app folder as per the django management command folder structure and run with:

./manage.py report_responses
bash
note code Copy code

Your site will need some content, preferably close to the same content as your live or staging site and will need to be running on your development machine (DEBUG = True)