Patched auditwheel directly. Should fix manylinux python build.

This commit is contained in:
Thomas Ferreira de Lima 2018-12-19 18:30:46 -05:00
parent 36af902e3f
commit a288b3ce39
No known key found for this signature in database
GPG Key ID: 43E98870EAA0A86E
2 changed files with 15 additions and 8 deletions

View File

@ -140,6 +140,10 @@ I tested step 1-5 with both quay.io/pypa/manylinux1_x86_64 and quay.io/pypa/many
I noticed that the ccache folder ended up with 800MB. I was hoping that the gcc compilation could reuse a lot of previously built objects but that didn't happen. I think that's because each python comes with its own header. So going forward it doesn't make sense to create a docker image for every python version. I will just cache a ccache folder via travis.
The ccache folder after a single build has 657MB. Go figure.
I discovered that fix_wheel script was actually not properly working. So instead I looked into fixing `auditwheel` directly. Here's the commit that fixes it: https://github.com/thomaslima/auditwheel/tree/87f5306ec02cc68020afaa9933543c898b1d47c1
So now the plan is to change the `docker_build.sh` script so it uses the proper auditwheel, instead of their default.
# Chapter 2. Testing CI flow with docker
# Step 1. Testing commands in own computer

View File

@ -16,7 +16,7 @@ echo DOCKER_IMAGE=$DOCKER_IMAGE
# sometimes the epel server is down. retry 5 times
for i in $(seq 1 5); do
yum install -y zlib-devel ccache zip && s=0 && break || s=$? && sleep 15;
yum install -y zlib-devel ccache zip git && s=0 && break || s=$? && sleep 15;
done
[ $s -eq 0 ] || exit $s
@ -37,13 +37,21 @@ fi
echo $PATH
export CCACHE_DIR="/io/ccache"
# Download proper auditwheel program
git clone https://github.com/thomaslima/auditwheel.git /tmp/auditwheel
cd /tmp/auditwheel
git checkout 87f5306ec02cc68020afaa9933543c898b1d47c1 # patched version
AUDITWHEEL_PYTHON=$(cat `which auditwheel` | head -1 | sed -e 's/#!\(.*\)/\1/')
# Install auditwheel, replacing the system's auditwheel binary
$AUDITWHEEL_PYTHON -m pip install .
# Show ccache stats
echo "Cache stats:"
ccache -s
cd /io
# Compile wheel
cd /io
"/opt/python/$PY_VERSION/bin/python" setup.py bdist_wheel -d /io/wheelhouse/ || exit 1
# Show ccache stats
@ -55,11 +63,6 @@ for whl in /io/wheelhouse/*linux_*.whl; do
auditwheel -v repair "$whl" -w /io/wheelhouse/ || exit 1
done
# Fix each wheel generated by auditwheel
for whl in /io/wheelhouse/*manylinux1_*.whl; do
/io/ci-scripts/docker/fix_wheel.sh "$whl" || exit 1
done
# Install packages and test
TEST_HOME=/io/testdata
"/opt/python/$PY_VERSION/bin/pip" install klayout --no-index -f /io/wheelhouse || exit 1