Add Implicit Resolution Support to IndirectObject?

[shartzog]

There seems to be a fair bit of code sprinkled about the codebase expressly for the purposes of ensuring that an element of interest (e.g. the "/W" widths array for a CID font dictionary) isn't simply an IndirectObject reference pointing to the "real" element. With some clever adjustments to the IndirectObject class, we may be able to avoid those headaches altogether and provide coverage for lots of additional use cases, no pre-checks and get_object() calls required.

Case in point, I recently ran into another unhandled edge case with layout mode text extraction, namely a CID font whose "/W" widths array was defined as an IndirectObject. Rather than adding another if isinstance(_w, IndirectObject): _w = _w.get_object(), I extended IndirectObject with the following implicit resolution routines and retested and it worked like a charm.

    def __getattribute__(self, __name: str) -> Any:
        try:
            return super().__getattribute__(__name)
        except AttributeError as ex:
            obj = self.get_object()
            if not hasattr(obj, __name):
                raise ex
            return getattr(obj, __name)

    def __iter__(self) -> Any:
        return iter(self.get_object())

    def __contains__(self, item: Any) -> bool:
        return item in self.get_object()

    def __getitem__(self, idx_or_key: Any) -> Any:
        return self.get_object()[idx_or_key]

When that worked to solve the TypeError in the _fonts.py module, I made the following simplification for PageObject._layout_mode_fonts() and retested all of my established use cases. Everything worked without exception.

Does anyone see a downside for pursuing this paradigm in a future PR? There are certainly some pitfalls, chief among them the potential for confusing exception tracebacks, but those pitfalls are solvable IMO.

[stefan6419846]

Are you using the latest release (4.1.0)? In #2464, something similar has been implemented if I am not mistaken.

[pubpub-zz]

i agree with @stefan6419846 . #2464 has started to do the job however, __iter__(self) and __contains__(self, item) have not been coded. Can you create a PR to complete this ?

[shartzog]

will do, @pubpub-zz. I should have it ready for review by Monday.

[shartzog]

(and thanks @stefan6419846 for pointing out that I was behind by a release. I was still working on v4.0.2 and have now corrected my watch settings so I don't miss this kinda thing ;))

[shartzog]

Upon further review, @stefan6419846 is correct. I think your original implementation already covers these use cases, @pubpub-zz. I'm not 100% sure since the IndirectObject instances formerly present in the output of PageObject._layout_mode_fonts() are now all pre-resolved as well (due to some other robustness improvement? #2495 perhaps??).

In any case, I thought the TypeErrors generated from python's internals bypassed the standard __getattr__/__getattribute__ mechanisms, e.g. when determining an object's 'iterability', but apparently not. I created the following toy example to check for 'iterabiliy' and a bunch of other conditions and include its output at the bottom.

To summarize:

• I found one potentially undesired but supported feature:
  • there's nothing to prevent the invocation of subobject methods that modify the subobject in place (e.g. list.sort()). That's probably fine IMO, but just pointing it out since __setitem__ wasn't implemented when you implemented __getitem__.
• I found four use cases that are NOT supported and may be worth pursuing:
  • __len__ throws a TypeError even if the subobject supports it
  • __bool__ doesn't consider the subobject's 'truthiness' and always returns True
  • math operations (+/-/+=/*/etc) throw TypeErrors even if they're supported by the subobject
  • (in)equality comparisons (</==/etc) throw TypeErrors even if they're supported by the subobject

The only two unsupported items that seem like obvious value adds are the len() and 'truthiness' items since those two would be both easy to implement and produce the most broadly expected behavior. The math and equality comparisons are a little less obvious since they'd require a fair bit of bloat and would likely be more problematic to implement. Let me know your thoughts.

from typing import Any
class A:
    def __init__(self) -> None:
        self.children = ["a", "b", "c"]

    def __getattr__(self, name: str) -> Any:
        # Attribute not found in object: look in pointed object
        try:
            return getattr(self.children, name)
        except AttributeError:
            raise AttributeError(
                f"No attribute {name} found in IndirectObject or pointed object"
            )

    def __getitem__(self, key: Any) -> Any:
        # items should be extracted from pointed Object
        return self.children[key]  # type: ignore

a = A()

# test implicit call to subobject method
a.append("d")

# test direct item access
print(a[0])

# test slicing
print(a[:])

# test resolution to proper subobject method from another object instance
print(sorted(a, reverse=True))

# test implicit looping
for x in a:
    print(f"for loop-{x}")

# test implicit contains
if "a" in a:
    print("yup, a's in a")

# test subobject manipulation method (reverse output of iter(a) below)
# NOTE: succeeds; is this desirable????
a.sort(reverse=True)

# test implicit iterator
for x in iter(a):
    print(f"iterator-{x}")

# test item deletion (NOTE: fails; likely undesired anyway?)
try:
    del a[2]
except TypeError as ex:
    print(f"\n --> {ex}")

# test item assignment (NOTE: fails; likely undesired anyway?)
try:
    a[1] = 2
except TypeError as ex:
    print(f"\n --> {ex}")

# test implicit len() (NOTE: fails; implement __len__ wrapper?)
try:
    print(len(a))
except TypeError as ex:
    print(f"\n --> {ex}")

# test implicit addition (NOTE: fails; implement __add__/__iadd__/__radd__/etc wrappers?)
try:
    print(a + ["e"])
except TypeError as ex:
    print(f"\n --> {ex}")

# test implicit (in)equality (NOTE: fails; implment __eq__/__gt__/etc wrappers?)
try:
    print(a > [])
except TypeError as ex:
    print(f"\n --> {ex}\n")

# test 'truthiness'
if a:  # with a 'truthy' subobject
    print(f"{bool(a.children)=}; truthiness confirmed when subobject is truthy")
a.children.clear()
if a:  # with a 'falsy' subobject
    print(f"{bool(a.children)=}; truthiness reported *incorrectly* for 'falsy' subobject?")
else:
    print(f"{bool(a.children)=}; falsiness reported correctly for 'falsy' subobject")

and here's the output:

a
['a', 'b', 'c', 'd']
['d', 'c', 'b', 'a']
for loop-a
for loop-b
for loop-c
for loop-d
yup, a's in a
iterator-d
iterator-c
iterator-b
iterator-a

 --> 'A' object doesn't support item deletion

 --> 'A' object does not support item assignment

 --> object of type 'A' has no len()

 --> unsupported operand type(s) for +: 'A' and 'list'

 --> '>' not supported between instances of 'A' and 'list'

bool(a.children)=True; truthiness confirmed when subobject is truthy
bool(a.children)=False; truthiness reported *incorrectly* for 'falsy' subobject?