CIS IT Systems Guide

Python on devweb #

devweb Python diagram

These instructions assume that you are using Linux from a CIS lab, Cafe or a CPU node.

For general information on devweb application containers, see devweb/#app-containers.

Contents #

Getting started #

Devweb supports Python applications via the WSGI interface. Applications are run using the Gunicorn app server, which expects a module named wsgi containing a callable named application to exist in your ~/DEVWEB/2024/python folder.

When your Python container is auto-provisioned, an example app will be copied to your ~/DEVWEB/2024/python folder. The example app will be called myapp.

The example wsgi.py file looks like:

import myapp

application = myapp.app

You can customise this to point to your own application by changing the import line to import your own module which you've copied to your Python folder, and pointing the application variable to your own WSGI application object (e.g. the object created by Flask(__name__) in Flask apps).

To test that your application runs properly under Gunicorn outside devweb (e.g. on a Linux lab computer or your own Linux/Mac/WSL2 development environment), make sure you've installed Gunicorn either in your virtual environment (with pip3 install gunicorn) or your user directory (with pip3 install --user gunicorn). You should then be able to start your app locally by running gunicorn wsgi. Note: you don't need to do this to run your app on devweb - this step is only needed for testing locally in your development environment.

Note: Gunicorn only runs in Unix-like environments. You'll be unable to test Gunicorn in Windows. Please see the docs for more information.

Logs #

Output from your program is logged to ~/DEVWEB/2024/.logs/python/python.out.log

Access logs are created at ~/DEVWEB/2024/.logs/python/python.access.log

Python package management #

Pre-installed packages #

Some packages are pre-installed in your container and available to use within your application.

The following packages are pre-installed from apt:

The following packages are pre-installed from PyPI (via pip3 install):

Installing other packages (using a vendor folder) #

If you want to use packages which aren't listed above, you need to vendor them. This installing the packages alongside your application in a folder which can be seen from the devweb container, and adding the vendor folder to your application's site path so that it checks this folder when trying to import packages.

First, you should be familiar with venvs (virtual environments) in Python.

Create a venv in your local development environment (e.g. on your own PC or a lab PC, perhaps at ~/dev/myproject - not in your devweb folder) and install your dependencies with pip3 install somepackage.

Once you've installed your dependencies and have confirmed that your application runs within your venv, freeze your package list with pip3 freeze > requirements.txt. Copy your requirements.txt file to your devweb folder (~/DEVWEB/2024/python), then from within that folder run pip3 install -r requirements.txt -t vendor to install the packages in your requirements file into the vendor folder.

Note: You should avoid copying the vendor folder between computers as it might not be portable - particularly across architectures (for example, going from Apple Silicon to the x84-64 arch used by devweb servers). Always create the devweb folder from a departmental computer like cafe or a lab PC, and not from your own PC over a network share. This is particularly important for any packages with a C/C++ build step, which may be affected by differences libc or Python versions, in addition to CPU architecture.

Your dependencies will now be installed in the vendor folder.

In order to be able to import these dependencies in your application, you'll need to add vendor to your site path to tell Python to look in this folder when importing modules. You can do this by adding the following to the very start of your own Python module (before you try to import any other modules):

import os
import site
vendor_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), "vendor")
site.addsitedir(vendor_path)

React, Angular and other frontend JavaScript frameworks #

If your app has a frontend JavaScript component (written in, for example, React or Angular), this component should be deployed to your static web container and configured to talk to your API running in your Python container.

Django caveats #

Django is designed with some specific assumptions that are not true of the devweb environment.

Base path #

Django, by default, expects your application to be running at / on the web server. However, your devweb Python app runs at /yourusername-python. You need to reconfigure Django to run properly within a subfolder on the webserver. You should consider setting the FORCE_SCRIPT_NAME setting to configure the base path for your Django app.

Static file path #

Outside of the debug environment (i.e. running Django using python3 manage.py runserver), Django will not serve static files from the Python app server. Django expects you to deploy your static files to a static web server. While this is important for performance reasons in a real production application, it adds some extra work for devweb users. You'll need to configure the STATIC_URL setting to point to a subfolder of your static web container and use collectstatic to gather the static files and copy them to your static container subfolder. The exact details of how to do this will vary from application to application and are outside the scope of these docs, but the linked Django documentation should give you enough of a starting point to apply that method to your own app.

Running manage.py #

You'll probably need to run manage.py against your codebase at some point. If you're using a vendor directory with third party dependencies, you can temporarily add these to your Python site path by setting the PYTHONPATH environment variable: e.g PYTHONPATH=vendor ./manage.py createsuperuser.

FastAPI / ASGI support #

FastAPI (and other async frameworks) use the ASGI protocol to run asynchronous applications, requiring an ASGI app server (like Gunicorn configured to use uvicorn workers). The devweb Python container doesn't support this at the moment.

However, users install the a2wsgi package in the vendor directory to provide a compatibility layer. Gunicorn still launches the app in traditional WSGI mode, but when launched, the application runs the rest of the user code in async mode. See above for instructions on how to use the vendor directory.

Important notes #

For help and support with CIS IT systems, please email cis-support@strath.ac.uk.

Department of Computer and Information Sciences
University of Strathclyde