13
submitted 6 months ago* (last edited 6 months ago) by thingsiplay@beehaw.org to c/programming@programming.dev

Solution was quite easy. Thanks to the user reply here: beehaw.org/comment/3535588 or programming.dev/comment/10034690 (Not sure if the links actually work as expected...)


Hi all. I have a little problem and don't know how to solve. A CLI program in Python is broken since Python 3.12. It was working in Python 3.11. The reason is, that Python 3.12 changed how subclassing of a pathlib.Path works (basically fixed an issue), which now breaks a workaround.

The class in question is:

class File(PosixPath):
    def __new__(cls, *args: Any, **kwargs: Any) -> Any:
        return cls._from_parts(args).expanduser().resolve()  # type: ignore

    def __init__(self, source: str | Path, *args: Any) -> None:
        super().__init__()
        self.__source = Path(source)

    @property
    def source(self) -> Path:
        return self.__source

    @property
    def modified(self) -> Time:
        return Time.fromtimestamp(os.path.getmtime(self))

    @property
    def changed(self) -> Time:
        return Time.fromtimestamp(os.path.getctime(self))

    @property
    def accessed(self) -> Time:
        return Time.fromtimestamp(os.path.getatime(self))

    # Calculate sha512 hash of self file and compare result to the
    # checksum found in given file. Return True if identical.
    def verify_sha512(self, file: File, buffer_size: int = 4096) -> bool:
        compare_hash: str = file.read_text().split(" ")[0]
        self_hash: str = ""
        self_checksum = hashlib.sha512()
        with open(self.as_posix(), "rb") as f:
            for chunk in iter(lambda: f.read(buffer_size), b""):
                self_checksum.update(chunk)
            self_hash = self_checksum.hexdigest()
        return self_hash == compare_hash

and I get this error when running the script:

Traceback (most recent call last):
File "/home/tuncay/.local/bin/geprotondl", line 1415, in 
sys.exit(main())
^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 1334, in main
arguments, status = parse_arguments(argv)
^^^^^^^^^^^^^^^^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 1131, in parse_arguments
default, status = default_install_dir()
^^^^^^^^^^^^^^^^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 1101, in default_install_dir
steam_root: File = File(path)
^^^^^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 97, in __new__
return cls._from_parts(args).expanduser().resolve()  # type: ignore
^^^^^^^^^^^^^^^
AttributeError: type object 'File' has no attribute '_from_parts'. Did you mean: '_load_parts'?

Now replacing _from_parts with _load_parts does not work either and I get this message in that case:

Traceback (most recent call last):
File "/home/tuncay/.local/bin/geprotondl", line 1415, in 
sys.exit(main())
^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 1334, in main
arguments, status = parse_arguments(argv)
^^^^^^^^^^^^^^^^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 1131, in parse_arguments
default, status = default_install_dir()
^^^^^^^^^^^^^^^^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 1101, in default_install_dir
steam_root: File = File(path)
^^^^^^^^^^
File "/home/tuncay/.local/bin/geprotondl", line 97, in __new__
return cls._load_parts(args).expanduser().resolve()  # type: ignore
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/pathlib.py", line 408, in _load_parts
paths = self._raw_paths
^^^^^^^^^^^^^^^
AttributeError: 'tuple' object has no attribute '_raw_paths'

I have searched the web and don't understand how to fix this. Has anyone an idea what to do?

top 8 comments
sorted by: hot top controversial new old
[-] slurp@programming.dev 3 points 6 months ago* (last edited 6 months ago)

Don't really understand why you're overriding __new__. Surely it'd work better to use:

def __init__(self, source: str | Path, *args: Any) -> None:
        super().__init__(Path(source, *args).expanduser().resolve())

But this removes self.__source and the property. I'm not sure what the advantage of using that is but you could always set that before the line super().__init__(Path(source, *args).expanduser().resolve()).

EDIT: if I've completely misunderstood, please could you explain? I don't really understand what subclassing is trying to achieve here, other than simplifying access to certain os.path functions.

[-] thingsiplay@beehaw.org 2 points 6 months ago* (last edited 6 months ago)

I think in Python 3.11 and prior overriding new was needed in order to subclass it correctly, because it was buggy until 3.12. This was a workaround, if I remember correctly. I already tried without new and setting self__source, but couldn't make it work. I was close!

With your suggestion it works now! I will need to test the app a bit now and if everything looks fine, will update the repository. Did exactly what you said and replaced this part:

# class File(PosixPath):
#     def __new__(cls, *args: Any, **kwargs: Any) -> Any:
#         return cls._load_parts(args).expanduser().resolve()  # type: ignore
#
#     def __init__(self, source: str | Path, *args: Any) -> None:
#         super().__init__()
#         self.__source = Path(source)
#
class File(PosixPath):
    def __init__(self, source: str | Path, *args: Any) -> None:
        self.__source = Path(source)
        super().__init__(Path(source, *args).expanduser().resolve())

Thank you for the help and solution!

Edit: Forgot to answer your question. This could work without subclassing too off course, but doing it this way makes it more clean in my opinion. It's the natural way to me to extend functionality and use the extended version for the purpose its designed to. It's not strictly required and I would have done it otherwise too, if this was problematic and not working.

[-] slurp@programming.dev 2 points 6 months ago

Glad I could help! I wasn't sure if I was missing something in what you were trying to do - I get that in some cases folding in those features can make things a lot easier.

[-] thesmokingman@programming.dev 2 points 6 months ago

What’s the constructor in PosixPath look like?

[-] thingsiplay@beehaw.org 2 points 6 months ago* (last edited 6 months ago)

From https://github.com/python/cpython/blob/3.12/Lib/pathlib.py

class PosixPath(Path, PurePosixPath):
    """Path subclass for non-Windows systems.

    On a POSIX system, instantiating a Path should return this object.
    """
    __slots__ = ()

    if os.name == 'nt':
        def __new__(cls, *args, **kwargs):
            raise NotImplementedError(
                f"cannot instantiate {cls.__name__!r} on your system")

Which sub classes PurePosixPath, which itself is basically just PurePath with a Posix flavour to it. And Path itself is PurePath as well. Not sure if I should copy paste them here.

Edit: So I added these to my reply:

PurePath

    def __new__(cls, *args, **kwargs):
        """Construct a PurePath from one or several strings and or existing
        PurePath objects.  The strings and path objects are combined so as
        to yield a canonicalized path, which is incorporated into the
        new PurePath object.
        """
        if cls is PurePath:
            cls = PureWindowsPath if os.name == 'nt' else PurePosixPath
        return object.__new__(cls)


    def __init__(self, *args):
        paths = []
        for arg in args:
            if isinstance(arg, PurePath):
                if arg._flavour is ntpath and self._flavour is posixpath:
                    # GH-103631: Convert separators for backwards compatibility.
                    paths.extend(path.replace('\\', '/') for path in arg._raw_paths)
                else:
                    paths.extend(arg._raw_paths)
            else:
                try:
                    path = os.fspath(arg)
                except TypeError:
                    path = arg
                if not isinstance(path, str):
                    raise TypeError(
                        "argument should be a str or an os.PathLike "
                        "object where __fspath__ returns a str, "
                        f"not {type(path).__name__!r}")
                paths.append(path)
        self._raw_paths = paths

Path


    def __init__(self, *args, **kwargs):
        if kwargs:
            msg = ("support for supplying keyword arguments to pathlib.PurePath "
                   "is deprecated and scheduled for removal in Python {remove}")
            warnings._deprecated("pathlib.PurePath(**kwargs)", msg, remove=(3, 14))
        super().__init__(*args)

    def __new__(cls, *args, **kwargs):
        if cls is Path:
            cls = WindowsPath if os.name == 'nt' else PosixPath
        return object.__new__(cls)
[-] thesmokingman@programming.dev 2 points 6 months ago

What I’m guessing is that these things have a different loader you want to use. You’re headed in the right direction here. The “no method” error just means you have to trace what you need.

You could also look at something like pipx or a virtualenv to guarantee an older version of Python.

[-] thingsiplay@beehaw.org 2 points 6 months ago

The other person got me the solution. __new__() is no longer needed to be overwritten anymore.

[-] pro_grammer@programming.dev 2 points 6 months ago

Hey, I don't know the answer, but try cross posting it to the python community as well, might help you

this post was submitted on 19 May 2024
13 points (100.0% liked)

Programming

17314 readers
239 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 1 year ago
MODERATORS