These tools together solve a fraction of the problen. The other parts of the problem are interfacing with classic c, c++ libraries and handling different hardware and different OSes. It is not even funny how tricky it is to use the same GPU/CUDA versions but with different CPU architectures and hopefully most people dont need to be exposed to it. Sometimes parts of the stack depends on a different version of a c++ library than other parts of the stack. Or some require different kernel modules or CUDA driver settings. But I would be happy if there was a standardized way to at least link to the same C++ libraries, hopefully with the same ABI, across different clusters or different OS versions. Python is so far from solved…
We do something similar with a mix of C++, Fortran, and CUDA dependencies. Using https://cibuildwheel.pypa.io/en/stable/ and the repair functionality to restrict the manylinux to something sensible seems to have handled the complexity - while we mostly use Debian and Ubuntu the Arch folks seemed OK too.
We want to share the wheels more smoothly though and having been looking at pyx for that. Seems promising.
Pixi inherited some ot the bad designs from conda, and conda clearly hadnt been close to solving python packaging either and was digging itself into a black hole over time.
