Recently I’ve had some issues with 64 bit Arch Linux on my Raspberries.
Upgrading the system added 32 bit libraries to the otherwise 64 bit operating system. That lead to various issues. Foremost applications not working. Quite a bunch of them, like both my file managers.
It was quite tedious to take over the 64 bit libraries from other repos manually and one by one. After taking over like 8 library files, I got it back working, but still had issues. Like for example, a number of menues and combo boxes showing in the top left corner of the display, instead of where they belong. Another issue was, that Atril didn’t work.
After the work, I also wrote an email to the providers of Arch Linux ARM and told them about the issue.
Forgetting about the issues, yesterday I performed another system upgrade and tried to reinstall Atril. It still didn’t work. But I just kept going. So, today my most important application crashed. When I tried to open it again, it completely failed. Once again issues with 32 bit libraries.
This time it was way more severe. The more upgrades in the 32 bit repo, the more packages are upgraded and the more goes wrong on the 64 bit devices. Again a number of applications didn’t run anymore. But this time, even the upgrade didn’t work, since pacman too complained about 32 bit libraries, that should have been 64 bit.
So, first I did take a look at the /etc/pacman.conf. Then, once again, I took over the libraries manually from other repos. Then I tried to upgrade the system. It installed only 10 packages. So, I got a package list, specifically of the installed libraries:
pacman -Qq | grep -i “lib*” > libpkg.txt
Then I tried to reinstall the libraries.
First, after a list of warnings, it just stopped. Among the warnings was an error, that libng-glib couldn’t be found. So, I just removed it from the list. Next I got a bunch of errors. Those are likely related to my particular setup. Luckily, just reinstalling the ffmpeg package reinstalled all packages that caused the other errors.
Then I just reinstalled all (270) installed libaries:
pacman -S – < libpkg.txt –noconfirm –force
That seem to be working. But since among the 10 upgraded packages was a kernel upgrade, I thought I should perform a quit reboot, before I go on doing my stuff.
Well, bad idea. Because that one ended in a kernel panic. So, what now?
Well, here is a little neat trick:
I extracted the original download image and copied over /boot, /usr/lib and /usr/bin, to prevent messing with settings and stuff.
Now this thing booted again, but didn’t recognize the root user and also the VNC server didn’t work anymore. I was able to SSH in though by using the default user and was able to sudo the commands.
This time I got a list of ALL packages installed:
sudo pacman -Qq > pkglist.txt
Then went for the real deal. The full reinstall.
Once again I got 3 errors. I think it was again the before mentioned library files and two packages I installed from AUR. I removed those from the list.
Then the upgrade did run. But it seem kind of strange, that only like 150MB would be downloaded. So, I cancelled and clear the cache:
pacman -Sc
Well, that didn’t do much. In fact, now it said, it’s going to install only less than 115MB. After the downloads it also complained about signature conflicts.
So, I went to /var/cache/pacman/pkg/ and deleted just everything manually, then ran the upgrade again. Well, 944MB. Now we are talking!
The upgrade worked.
After the upgrade I started the VNC server manually and tried a few applications. And they were back working and obviously faster than they were before.
After restarting the thing, now everything seems to be back to normal. VNC server, root user, applications,… everything back working normal again. Including Atril.
Here is the problem:
Someone forgot to change the architecture in the pacman.conf file to aarch64 and instead left the armv7h architecture in there from the 32 bit setup. So, the basic setup was 64 bit, but the upgrade used the 32 bit repos.
So, do yourself a favor. If you are using the 64 bit aarch64 Arch Linux for your ARM device, check the pacman.conf and make sure the architecture is set to aarch64 instead of armv7h. And in case you already have the issue present and issues with 32 bit libraries, make sure you delete everything in your pacman cache directory and then upgrade all packages to aarch64, after making sure it’s in pacman.conf.
I’ve set up this particular device not too long ago. Some days, maybe a week or so? Not sure. So, it might be the current image only. But better safe than sorry, right?