I tried to change the packages mentioned in pyproject.toml into Containerfile to use eg: dnf -y install "python3dist(ansible)" instead of pip install.

The dependent packages in the toml file like ansible-runner causing an error while starting the container from the image created.

Error: Failed to import the callback module (No module named 'ansible_runner') provided in the --callback argument 

So, to avoid this issue I installed the ansible-runner via pip and selected very few packages but seeing below error as the pip is building whole dependent packages mentioned in the toml file

Error: Failed to import the callback module (No module named 'fedora_image_uploader') provided in the --callback argument

Is there a way to avoid this issue by changing toml file.
Any pointers would be helpful. If needed I can update the changes I tested in here

Thanks

Hmm, I think this is because when fedora-messaging is installed in the virtual environment, the script entrypoint in venv/bin/ is configured with a PYTHONPATH environment variable that includes the virtual environment, but when it's installed via the system package it doesn't. So the fix for that particular issue would be to, in the Containerfile, do something like (untested):

ENV PYTHONPATH=/srv/image-uploader/venv/lib/lib/python3.12/site-packages/

However, while I was playing around with this, it seems that there's a few packages that aren't available in Fedora and which the Ansible collection requires (even though we don't use them, or most of the other packages, at all):

• azure-containerregistry
• azure-mgmt-datafactory
• azure-mgmt-nspkg
• azure-nspkg
• azure-mgmt-automation
• azure-iot-hub
• azure-mgmt-notificationhubs

There's a few paths forward with this task I can think of, which aren't necessarily mutually exclusive.

Package Missing Dependencies

We could package the above libraries. One could argue providing the full Azure Python SDK via RPMs is something we'd want anyway, although I'm rather dubious about whether anyone at all uses them instead of using pip to manage their Python dependencies. However, that's a different debate.

It's worth noting if we go down this path we're taking on the following work:

• Maintaining the added packages
• Accepting that we'll need to add more packages as the ansible collection introduces additional dependencies. This is a non-trivial increase in the maintenance burden of this service.
• Having to deal with incompatible versions between the SDK, azure-cli, and the ansible collection.

The last point is particularly painful. The SDK is made up of dozens of packages which each release major versions at different times. The azure-cli depends on a particular set of these packages. The ansible collection depends on a different set of these packages. The two won't always line up. It's not easy to maintain many versions of a package in Fedora, so breakage from this will definitely occur.

Accept Using PyPI

Rather than trying to use packages from Fedora, just accept PyPI. This isn't terribly satisfying, but it's an option.

Stop Using Ansible

I don't think this option will be popular with some folks, but we could just use the Python SDK directly. It's not difficult to use, and I already implemented image cleanup via the SDK. Doing it this way would allow us to trim our dependencies down to just the services we use (compute and storage) and makes depending on the system packages much more viable.

If we went this direction, it would be important to ensure it's usable outside the Fedora ecosystem. Right now the general idea is that other projects can just use the Ansible playbook we've got, but if we stopped doing that we'd need to provide a Python API that wraps the Azure SDK implementation of what the playbook currently handles.

Refactor the Azure Ansible Collection

We could attempt to refactor the ansible collection to defer importing modules until they're actually required so that you don't need the whole stack when you only want to use one small part of it.

We would need to inquire if they're open to this kind of refactor since it doesn't really solve a problem for them (they're already recommending that you just use pip).

We no longer use the Azure Ansible collection so this should be much more achievable

Ultimately, the goal @davdunc and I originally set for this tool was to make it useful and usable way for people to manage uploading images en masse to various clouds. Whether we use Ansible or Python tools directly doesn't really matter as long as we have a good interface for people to use it.