How to upgrade Python using Yocto pinned to Thud?

Question

I'm running an embedded Linux build on the TI Sitara AM335x module, and using their Processor SDK to build a custom u-boot, kernel, and root filesystem.

Their latest version of the SDK is 06.03 and is pinned against Yocto Thud distributions (which are a bit old now). I've got everything I need setup, have my new BSP, can build everything I need to, but there is exactly one thing I need that I can't get working. The SDK comes with Python 3.5, and I need Python 3.6+ to run a certain tool.

Strictly as an experiment, I've built Docker into the system, and I can run newer containers in there - but that comes with a non-trivial size and performance impact, as well as some additional deployment and upgrade considerations.

I'm trying to figure out if there is a way for me to upgrade only the Python distribution, but I can't seem to find much information about that. There are some notes around about sequentially applying all the patches from later distributions, and trying to compile, debug, etc, but there are a lot of patches and it would be easy to miss something.

I do know one option is to throw away the SDK and go straight Yocto Dunfell, but the Processor SDK comes with a lot of kernel and u-boot tweaks baked in - so I'd like to stick with that in the short term (long, long term plan is to move to straight Yocto and get the latest/greatest).

TLDR; Is there any simple/streamlined mechanism to upgrade Yocto Thud to Python3.6+ without too much risk in finding and applying kernel and rfs patches, or using Docker inside of the rootfs, or moving away from the Processor SDK?

Answer 1

I was in the same position and was able to upgrade python 3.5.6 (thud) to python 3.7.7 (warrior).

To do this I did the following steps:

1. Fork poky repository
2. Replace the complete meta/recipes-devtools/python path with the one in the warrior branch
3. Replace the following bbclasses in meta/classes/ with warrior version:
   • distutils-tools.bbclass (delete)
   • distutils.bbclass
   • distutils3.bbclass
   • python3-dir.bbclass
   • python3native.bbclass
4. Bump python version from 3.5 to 3.7 here: meta/recipes-graphics/mesa/files/0004-Use-Python-3-to-execute-the-scripts.patch b/meta/recipes-graphics/mesa/files/0004-Use-Python-3-to-execute-the-scripts.patch
5. Patch the dnf_2.7.5.bb with the following patch:

From acd3f9b300fcd7319a83d09c24bcf252d0bb82b1 Mon Sep 17 00:00:00 2001
From: Yocto thud <[email protected]>
Date: Thu, 2 Nov 2023 08:33:27 +0000
Subject: [PATCH] fix dnf/util.py for python3.7

---
 dnf/util.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/dnf/util.py b/dnf/util.py
index 9726339e7..8d32bd09c 100644
--- a/dnf/util.py
+++ b/dnf/util.py
@@ -39,6 +39,7 @@ import subprocess
 import sys
 import tempfile
 import time
+import errno
 
 logger = logging.getLogger('dnf')
 
@@ -120,7 +121,7 @@ def ensure_dir(dname):
     try:
         os.makedirs(dname, mode=0o755)
     except OSError as e:
-        if e.errno != os.errno.EEXIST or not os.path.isdir(dname):
+        if e.errno != errno.EEXIST or not os.path.isdir(dname):
             raise e
 
 def empty(iterable):
-- 
2.17.1

This worked on my image with the packages that I was using. Other packages still might need some patches.