this post was submitted on 23 Sep 2023
22 points (95.8% liked)

Python

6413 readers
3 users here now

Welcome to the Python community on the programming.dev Lemmy instance!

📅 Events

PastNovember 2023

October 2023

July 2023

August 2023

September 2023

🐍 Python project:
💓 Python Community:
✨ Python Ecosystem:
🌌 Fediverse
Communities
Projects
Feeds

founded 1 year ago
MODERATORS
jnovinger
erlingur
norambna
22
[Answered] Typehints for functions that have variable signatures (lemmy.cafe)
submitted 1 year ago* (last edited 1 year ago) by [email protected] to c/python
18 comments fedilink hide all child comments
 

I know what I am asking is rather niche, but it has been bugging me for quite a while. Suppose I have the following function:

def foo(return_more: bool):
   ....
    if return_more:
        return data, more_data
   return data

You can imagine it is a function that may return more data if given a flag.

How should I typehint this function? When I use the function in both ways

data = foo(False)

data, more_data = foo(True)

either the first or the 2nd statement would say that the function cannot be assigned due to wrong size of return tuple.

Is having variable signature an anti-pattern? Is Python's typehinting mechanism not powerful enough and thus I am forced to ignore this error?

Edit: Thanks for all the suggestions. I was enlightened by this suggestion about the existence of overload and this solution fit my requirements perfectly

from typing import overload, Literal

@overload
def foo(return_more: Literal[False]) -> Data: ...

@overload
def foo(return_more: Literal[True]) -> tuple[Data, OtherData]: ...

def foo(return_more: bool) -> Data | tuple[Data, OtherData]:
   ....
    if return_more:
        return data, more_data
   return data

a = foo(False)
a,b = foo(True)
a,b = foo(False) # correctly identified as illegal
you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 4 points 1 year ago (1 children)

Agreed. I avoid having the shape of the return type be determined by arguments. Having the return type be generic is one thing. But this is different as here you are taking about returning 1 object or 2 objects.

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (2 children)

but from a practical perspective, let's say you retrieve an object and can either return a subset of its fields as your API. Doesn't it make sense to re-use the same function, but change what fields are returned?

I'm specifically talking about the narrow use-case where your API returns either A or B of the fields, and won't extend it in the future

The alternative is to either duplicate the function, or extract it out which seems a bit overkill if it is only 2 different type of functions.

[–] [email protected] 5 points 1 year ago

In my opinion, it doesn't. I'd rather have foo() and detailed_foo() over foo(detailed: bool = False).

Designing APIs can be hard at times. You have to shift your view to the person that will being using the code instead of the person implementing the code. There is also potential down side of returning a tuple or just a single thing if the single thing shares some of the same API as a tuple. Say the return type is Union[str, tuple[str, str]. Now result[0] can either be the first string or the first character of the returned string depending on how the function was called. This could lead to the failure happening farther away from where the bug is, which makes debugging harder. That being said, if you do want to proceed this way, overload with Literal[True] is the correct way to type this as mentioned in other comments.

I also don't think it's overkill to extract functionality just for 2 functions. I often do that even when it is only used in one function. Maybe the number of lines to implement the block starts to make the primary function too long. Or the logic is a bit complicated, so it easier to give it a clearer name.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago) (1 children)

In general, I'd say what you're trying to do is poor form; primarily because it's "just weird."

When you're writing code that will be interacted with later as a sort of API ... the #1 thing is how that API feels to use. Is it consistent? Does it follow normal rules? Are you likely to be surprised by how it behaves? Does it compose well (i.e. how well can it be used in other code)?

You're shoving two functions together and using a boolean flag to determine where to go. That's really weird. Data shouldn't drive the program in this way.

You've basically spelled:

def do_x():
def do_y():

do_x()

As:

def do_(char):

do_('x')

The program:

def bar(k):
  x = do_(k)

Is never going to be valid. I'd never accept a code review with this code in it without an extremely strong justification of why it has to be this way.

Remember, extra lines in your program are cheap. Bugs from being clever to reduce the number of lines aren't.

[–] [email protected] 2 points 1 year ago (1 children)

I think there's a spectrum here, and I'll clarify the stances.

The spectrum ranges from "Data shouldn't cause the function to do (something wildly) different" to "It should be allowed, even to the point of variable returns"

I think you stand on the former while I stand on the latter. Correct me if I'm wrong though, but that's the vibe I'm getting from the tone in your example.

Data shouldn’t drive the program in this way.

Suppose we have a function that calculates a price of an object. I feel it is agreeable for us to have compute_price(with_discount: bool), over compute_price_with_discount() + compute_price_without_discount()

You’ve basically spelled:

I feel your point your making in the example is a bit exaggerated. Again, coming back to my above example, I don't think we would construe it as compute_price('with_discount').

Maybe this is bandwagoning, but one of the reason for my stance is that there are quite a few examples of variable returns.

eg:

  • getattr may return a different type base on the key given
  • quite a few functions in numpy returns different things based on flags. SVD will return S if compute_uv=False and S,U,V otherwise
[–] [email protected] 1 points 1 year ago

I think you stand on the former

Absolutely.

Suppose we have a function that calculates a price of an object. I feel it is agreeable for us to have compute_price(with_discount: bool), over compute_price_with_discount() + compute_price_without_discount()

Well, presumably you'd also actually have some other inputs to a price compute function. In which case, I'd suggest bundling all that information into an Invoice type or something that includes whether or not discounts are applied...

Maybe this is bandwagoning, but one of the reason for my stance is that there are quite a few examples of variable returns.

getattr is really special, it's basically a reflection operator, it shouldn't be a model for how a normal function should behave.

I'm not familiar with numpy. The linked function though looks like a true case of generic behavior where an input changes an output in a specified way for any number of values that meet its requirements. A boolean flag is never generic.