We should no longer need src/externals, and thus not need `pegasus-config --python-externals`. For RPM and DEB packages setting PYTHONPATH would not be required at all. For binary tarballs we could ship setup.sh, like HTCondor that can set PYTHONPATH with simply `pegasus-config --python`.
# https://stackoverflow.com/a/48285243 cd pegasus-src/lib/pegasus/python/ # Debian 9 docker run -i --rm -w /srv -v `pwd`:/srv debian:9 bash -s <<EOT apt-get update && apt-get -y install python-pip python-setuptools debhelper pip install stdeb rm -rf ./x pip download --no-binary=:all: . -d x cd x tar zxvf pika-1.1.0.tar.gz cd pika-1.1.0 python setup.py --command-packages=stdeb.command bdist_deb EOT # CentOS 7 docker run -i --rm -w /srv -v `pwd`:/srv centos:7 bash -s <<EOT yum -y install epel-release yum -y install python36-pip python36-setuptools rpm-build rm -rf ./x pip download --no-binary=:all: . -d x cd x tar zxvf pika-1.1.0.tar.gz cd pika-1.1.0 python3 setup.py bdist --formats=rpm
Lowest common version across supported platforms is
Lowest common version across supported platforms is /
Mats Rynge Can we expect Python 2.7 to be available on CentOS 6?
Mats Rynge List Python components included in worker package.
Will need to test with the lowest common available version across supported platforms.
Will require more platform specific integration tests.
Mats Rynge Do .rpm, .deb packages support this? Mats: this is only allowed if you do into your own space (which is similar to what we do with externals. You are not allowed to pip install to the system location)
Packages may conflict with system installed packages. yum install python-flask (v0.9) vs pip install Flask (v1.0.2)
Latter two will require more platform specific integration tests to detect incompatible, missing dependencies.
Only for packages not expected to be available on supported platforms.
Copy dependency code into Pegasus.vendor directory.
# Instead of this import six # Use this import Pegasus.vendor.six
Will run unit tests on the minimal supported Python version and above.
Uses assert instead of self.assert*.
import pytest # Simple method, no need for classes def test_method(): # Simple assert, no need for assertEqual, etc. assert a % 2 == 0, "value was odd, should be even" with pytest.raises(ZeroDivisionError): 1 / 0 # One method, multiple tests # https://docs.pytest.org/en/latest/parametrize.html @pytest.mark.parametrize( "a, b", [ (1, 2, 3), (2, 3, 5), (5, -100, -95), ], ) def test_eval(a, b, expected): assert a + b == expected # Dependency Injection @pytest.fixture(scope="function OR class OR module OR package OR session") def client(): import requests s = requests.Session() s.get(".../login") yield return s # Yield can be replaced with return if no cleanup is required. s.get(".../logout") s.close() def test_external(client): assert client.get("/endpoint-1")