The use of sudo pip3 appears to be causing issues

[borpin]

emonhub/install.sh

Line 29 in f286e18

sudo pip3 install paho-mqtt requests pybluez py-sds011 sdm_modbus minimalmodbus

https://community.openenergymonitor.org/t/emonhub-paho-mqtt-fails-to-install/20522/

pypa/pip#9348 (comment)

[borpin]

And this for reference

pypa/pip#5599

[TrystanLea]

Thanks @borpin

[TrystanLea]

Hello @borpin I think the issue here is that we create an emonhub user to run emonhub and Im pretty sure that user cant access pip modules installed in say the 'pi' user context.

I think it's important to be able to run emonhub in the main user context, it's useful for testing among other things. So I wonder if the best solution is to drop the emonhub specific user and install emonhub and service file using the main user context?

[borpin]

OK, but there are other ways to solve that. Install it (and update it) as a global package I think will work.

I think using pip3 on it's own (i.e. the wrapper) is also now frowned upon :)

The tricky bit will be migrating installations from sudo pip. Probably can't so maybe a completely new branch of the install/update scripts.

[TrystanLea]

Is there an obvious way to install pip packages globally without using sudo?

I think using pip3 on it's own (i.e. the wrapper) is also now frowned upon :)

do you know what the solution is?

[TrystanLea]

From what I can tell searching online and looking at the pip3 install help it's not clear..

[alexandrecuer]

For me when you use sudo pip3, you install the package globally, just like when you use sudo apt install

Many projects dont have the apt package up to date because it is a lot of work for maintainers, eg pymodbus, and so if yu want a fresh version, yu have to install them with sudo pip3 if yu want a global package

So using the user pi instead of emonhub could be a good and rational choice because other services from openenergymonitor depend on the pi user
I dont know why the emonhub user was introduced

[borpin]

For me when you use sudo pip3, you install the package globally

But it is not the right way to do it, and can cause issues and raises errors.

I think the right way is to use venv, but I have no idea how :(

However, moving to a venv could have the advantage of making the update process easier.

[alexandrecuer]

Ok it is not the good practise to do that

in the case of emonhub, why is a specific user needed ? a global package is only needed because emonhub is running with a specific user.

For maintenance, as other services run under the pi user like the service runner, the feedwriter, the emoncms_mqtt, the simpliest choice is to run emonhub with the pi user. Why making a different choice for emonhub ?

And therefore yu dont need anymore neither a global package neither yu need to do sudo pip3

[TrystanLea]

I agree that the simplest solution is to move emonhub to the main users username, keeps things simple and consistent with how we handle the other services.

[TrystanLea]

I think the right way is to use venv, but I have no idea how :(

Nor do I, I guess Im drawn to the solution that I understand already.

[TrystanLea]

Ok, I've got this working and pushed to master 988b4c2

[borpin]

Ok, I've got this working and pushed to master 988b4c2

Does it upgrade gracefully?

I was going to do a build on the new PiOS and see if there were the memory issues. Have you made any other changes for the new OS image?

[borpin]

As part of this, should we install/update pip from Python then call it from Python?

[TrystanLea]

Yes upgrade appears graceful as far as I can tell so far.

I was going to do a build on the new PiOS and see if there were the memory issues. Have you made any other changes for the new OS image?

The main change is php8.1 + associated redis and mosquitto-php builds.

[TrystanLea]

As part of this, should we install/update pip from Python then call it from Python?

Could do, is that to get a newer version?

[borpin]

Could do, is that to get a newer version?

Maybe. As you say the docs/advice looks confused. The main Pi OS pages still say to use sudo pip3 😆

no I think leave as is. I saw on one of the updates I did it was saying about a newer version of pip.

[TrystanLea]

ok great thanks!