👍 well written and I'd be for it (for what that's worth), plus it'd be a big incentive to move to Python 3 and reduce the 2/3 fragmentation. (even if the libraries themselves couldn't use those operators because of compatibility issues)

Hey @njsmith, we had some discussions ages ago at SciPy'07 or '08, I did my best to summarize them here. That stuff is in an old bazaar repo, but you can get the reST sources here.

I never really followed this up beyond summarizing the conversations in that document. Just mentioning it here in case any of that ancient history is useful, feel free to ignore it in light of the last 5-6 years' worth of experience :)

BTW, there's a trivial makefile too that I was using to make rendered versions of the rst sources; it would be handy to have such a view somewhere, as it does make it much nicer to read.

+1 from me. Nice work. Thanks for putting this together.

This leaves out things like cross product and matrix division. What about using something like @ as a prefix for an operator to define it as a matrix operation? So @, @/, @__. You could then use @@ for the "reverse" version of that operator, so @@ would be cross product, @@/ would be back division.

@toddrjen: the suggestion of using a prefix for multiple operators is similar to PEP-225, and it has not made progress.

Also, I think cross product and matrix division are below in priority to matrix multiplication in necessity.

Probably, but I would say matrix exponent is below all of those in importance.

I think the argument for including @@ is applicable to other mathematical operators as well, and I would say they are far more useful.

So I would either cut out @@, or better justify including it but not any of other operators.

It will be helpful to include a survey of what other languages use. For example, there is a precedent for using $ for dot/matrix multiply. R uses %*%. APL used an equivalent of +.* (multiply compose add).

@fperez You can view a Rendered version on github through the Files Changed tab.

Here's the link to the rendered file: https://github.com/njsmith/numpy/blob/matmul-pep/doc/neps/return-of-revenge-of-matmul-pep.rst

Thanks, @pv, @mforbes; it would be good to add that link to the PR description at the top for readability; I don't have the permissions to edit that box. @njsmith?

In addition to the APL, Kx, and R syntax given by @abalkin above, I skimmed some from Rosetta Code:

I'd say APL's use of non-ASCII characters was historically unpopular. IDL's hash clashes with Python's comment. J uses an ASCII reworking of APL, I think, so it's got a much more elaborate idea of combining multiple infix operators together; powerful but perhaps not really Pythonic. The dot of Mathematica and Maxima is nice but clashes with Python's object attribute/method/module accessor. Julia, Octave, Pari/GP, and TI-83 BASIC's star is nice but would be backward-incompatible with NumPy's elementwise multiplication; Ruby's star relies on the subclass idea eloquently rebutted by the original poster. R's and K's multicharacter things are pretty ugly.
I don't think any of these alternatives are superior to the proposed at-sign, given that star and dot are unavailable.

I would like to see a more complete examination of np.Matrix before
considering new operators. The PEP dismisses the Matrix class without
sufficient justification.

There are perhaps warts with np.Matrix, can these be identified and
removed, with less pain than introducing two new operators?

Colin W.

On 15-Mar-2014 3:33 PM, Robert Bradshaw wrote:

As has been pointed out, the issue with np.matrix vs np.ndarary
largely an illustration of
http://en.wikipedia.org/wiki/Composition_over_inheritance . It is rare
for the two different kinds of multiplication to be needed in the same
formula. Also, this doesn't solve A + r where r is a scalar--if A is a
matrix then this is A + Ir where I is the identity matrix. Subtraction
too. Do we need @+ and @-?

Similarly, when one writes A == 1 in a matrix context, we want to know
if it's equal to the identity matrix (not the singular all-one matrix)
and wants a truth value (not an array of booleans that doesn't have a
boolean operator, which, by the way, is another operator that could
behave differently: the all-zero matrix is False). We're up to quite a
few operators that have both matrix and array versions. Hmm... how to
assign different meanings to the same set of operators on an object
oriented language?

Do you have evidence that the pain of np.matrix is not due to the fact
that it (sometimes, inconsistently) pretends to be a np.ndarray? If
not, this PEP is not needed, and adding new (seldom used) operators is
not to be done lightly.

—
Reply to this email directly or view it on GitHub
#4351 (comment).

@cjw43 There have been several attempts over the years. The PEP acknowledges that history without attempting to recapitulate it, proposal for proposal. The PEP is not the beginning of the examination of the problems with a two-type solution, but the culmination of many years of experience with them (yes, plural). The past threads on the subject can be found in the mailing list archive.

By all means, please make concrete proposals if you have them. In the absence of concrete proposals, I think we have a reasonable case not to hold out hope that a good one will show up after all of these years.

@alan-isaac

In what sense? How many Python users have any real awareness of the bitwise operators (which have additionally been completely and usefully hijacked for other uses).

Every user of a library that hijacks these operators needs to be aware of both the hijacked and original meanings. Anybody who is used to a language where the power operator is written '^' gets painfully acquainted with operator.xor at some point. I bet that thousands of man-hours have been wasted answering the question "Why does 2^2 return 0 instead of 4?"

Any new operator increases the total complexity of the language, takes valuable developer time to implement, uses up some memory in the interpreter, etc. There are costs. They are small but they affect everyone in the Python ecosystem. Given that the benefits will only be felt by a small subset of the Python community, they had better be significant.

On Mar 19, 2014 5:13 AM, "Ronan Lamy" notifications@github.com wrote:

@alan-isaac

In what sense? How many Python users have any real awareness of the
bitwise operators (which have additionally been completely and usefully
hijacked for other uses).

Every user of a library that hijacks these operators needs to be aware of
both the hijacked and original meanings. Anybody who is used to a language
where the power operator is written '^' gets painfully acquainted with
operator.xor at some point. I bet that thousands of man-hours have been
wasted answering the question "Why does 2^2 return 0 instead of 4?"

What specific confusions ate likely with this proposal?

Yes, in a general sense that could be an issue with singer operators, but
we are talking about a specific proposal here, so if you have such
criticisms you need to explain how they are relevant to the current
proposal.

Any new operator increases the total complexity of the language, takes
valuable developer time to implement, uses up some memory in the
interpreter, etc. There are costs. They are small but they affect everyone
in the Python ecosystem. Given that the benefits will only be felt by a
small subset of the Python community, they had better be significant.

First, I don't think the scientific community is a small subset, I think it
is a pretty large subset. Even when they don't directly use linear algebra
in their code, a huge improvement in an operation that underlies a lot of
other common data analysis operations will have a positive impact. The
discussions covered this on a lot of detail. This operator will greatly
simplify and improve key parts of the scientific python software stack,
which benefits all scientific python users.

Second, you need to demonstrate that these changes will be anything other
than negligible. For the memory example, are we talking megabytes,
kilobytes, bytes? The question isn't just whether there are effects, the
question is whether the effects are actually large enough to be noticeable
in the real world. If someone really cared about every last byte they
wouldn't be using python to begin with.

Third, in terms of developer time, you can't just look at the time needed
to implement the operator, you need to compare that to the developer time
need to maintain the status quo. Again, the mailing list discussions
looked at this and there is a lot if time wasted dealing with the current
situation. And it is ongoing, as long as the status quo is maintained,
further development in the scientific software stack will have to work
around the current limitations.

@rlamy I think that it's worth noting that no one on python-ideas, except perhaps yourself, has objected to the PEP on these grounds or asked us to do more soul searching about whether we really need the operator. These are people most of whom will never use matrix multiplication. It's nice to be concerned for them, but they can speak for themselves and have spoken in favor of the PEP.

@toddrjen Implementing @ and @= will require two new pointers added to the PyNumber_Methods table, thus increasing the size of many type objects by 8 or 16 bytes (e.g. all sequences that implement + for concatenation). This does not affect instances, only types, but with things like namedtuple, dynamically creating instances is getting more common.

@ should definitely have right precedence.

@pchanial they are suggesting left precedence and asking for compelling arguments against it on the mailing list

I've just added text for "same-left" associativity/precedence:
http://mail.scipy.org/pipermail/numpy-discussion/2014-April/069834.html
and sent the updated version of the PEP in to the official repository.

This is probably a day late and a dollar short (as my Mom used to say) but has any serious consideration ever been given to defining dot (".") as an inner product operator? That's the way that APL does it, but every time it's brought up, it is dismissed out of hand. Yes, it would give an additional, different, meaning to ".", but "%" is used for very different purposes without too much confusion. Uisng the string "+._" in Python code is a syntax error, so we don't have to worry about breaking pre-existing code. I envision A+._B as syntactic sugar for

inner_product(operator.add, operator.mul)(A, B)

where inner_product is defined (for simple sequence objects) as equivalent to this:

class inner_product(object):
    def __init__(self, f, g):
        self.f, self.g = f, g
    def __call__(self, a, b):
        return reduce(self.f, (self.g(*pair) for pair in zip(a, b)))

The compiler would be allowed to instantiate +.* only once per sequence type, no matter how many times it appeared in the code, however non-standard sequence types (such as numpy's ndarrays) would be able to provide alternative implementations which would only be used by those types. (I envision a dict attached to the PySequenceMethods struct that would cache, for example, inner_product(operator.add, operator.mul) under the key '"+.*")

@samwyse, I am not sure how this could be an option. . already is the __getattr__ operator. There would be no way I can see to distinguish between A @ B and A . B (or as it would normally be written: A.B), neither for the user nor for the language.

Hi Sam,

I think the challenge you'd face if you tried proposing this to
python-ideas/-dev is to justify what all the extra complexity gets you.
This proposal would require a lot of extra machinery that Python doesn't
otherwise use (there are no other "meta-operators" like this), and it's not
clear that the extra complexity provides extra value -- like, it allows a
lot of other weird operators like *.+ and /./ and so forth, but would
anyone actually use these? If you look at the discussion of the PEP on the
Python lists, the core devs spoke explicitly about how they felt it hit the
"sweet spot" of being just the minimal bit of added functionality needed to
provide 99% of the value.

-n

On Mon, Apr 21, 2014 at 5:39 PM, Sam Denton notifications@github.comwrote:

This is probably a day late and a dollar short (as my Mom used to say) but
has any serious consideration ever been given to defining dot (".") as an
inner product operator? That's the way that APL does it, but every time
it's brought up, it is dismissed out of hand. Yes, it would give an
additional, different, meaning to ".", but "%" is used for very different
purposes without too much confusion. Uisng the string "+._" in Python code
is a syntax error, so we don't have to worry about breaking pre-existing
code. I envision A+._B as syntactic sugar for

inner_product(operator.add, operator.mul)(A, B)

where inner_product is defined (for simple sequence objects) as equivalent
to this:

class inner_product(object):
def init(self, f, g):
self.f, self.g = f, g
def call(self, a, b):
return reduce(self.f, (self.g(*pair) for pair in zip(a, b)))

The compiler would be allowed to instantiate +.* only once per sequence
type, no matter how many times it appeared in the code, however
non-standard sequence types (such as numpy's ndarrays) would be able to
provide alternative implementations which would only be used by those
types. (I envision a dict attached to the PySequenceMethods struct that
would cache, for example, inner_product(operator.add, operator.mul) under
the key '"+.*")

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4351#issuecomment-40950255
.

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

@seberg: I'm proposing "." as a meta-operator, which would not be used by itself. That particular character is currently used to access attributes, and also to separate the integral and fractional parts of floating point numbers. This would be a third use.

@njsmith: Way back when I learned APL, I recall lots of ways that f,g could be used besides matrix multiplication. For example, '<.&' would tell you if every number in a list was smaller that the corresponding number in another list. That said, my use of '&' in my example (as a logical and) has different semantics from Python's definition ('&' is bitwise and). Really, you'd want to use '<.and' which I suspect would really only be parsable because of 'and' being a keyword; and yes, Python already allows 'all(x < y for x, y in zip(a,b))' which means the same thing (albeit not as concisely). Iverson provides several examples, but they tend to use APL's 'max' and 'min' operators, which in Python aren't even keywords, and so would look to the compiler a lot like attribute access; I suspect that a proposal to implement GCC's minimum and maximum operators ('<?' and '>?') would be met with cat-calls.

I think a relevant data point here is that back in the day, Guido was
reluctant to even add the power operator ** -- it didn't show up until
Python 1.4. Since then, the only new punctuation operators have been //
(which was necessary to resolve the int/float mess), and now @. So yeah, I
don't think a proposal for a full APL-like system, min/max operators, etc.,
is likely to go very far :-).

On Mon, Apr 21, 2014 at 8:17 PM, Sam Denton notifications@github.comwrote:

@njsmith https://github.com/njsmith: Way back when I learned APL, I
recall lots of ways that f,g could be used besides matrix multiplication.
For example, '<.&' would tell you if every number in a list was smaller
that the corresponding number in another list. That said, my use of '&' in
my example (as a logical and) has different semantics from Python's
definition ('&' is bitwise and). Really, you'd want to use '<.and' which I
suspect would really only be parsable because of 'and' being a keyword; and
yes, Python already allows 'all(x < y for x, y in zip(a,b))' which means
the same thing (albeit not as concisely). Iverson provides several
examples, but they tend to use APL's 'max' and 'min' operators, which in
Python aren't even keywords, and so would look to the com;iler a lot like
attribute access; I suspect that a proposal to implement GCC's minimum and
maximum operators ('<?' and '>?') would be met with cat-cal ls.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4351#issuecomment-40966469
.

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

I see the relative merits of a
    Matrix Class and the proposed operator as having no been
    adequately explored.
    The destruction of the Class was not made clear in the PEP.
    Colin W.
On 4/21/2014 12:39 PM, Sam Denton
  wrote:

  This is probably a day late and a dollar short (as my Mom used
    to say) but has any serious consideration ever been given to
    defining dot (".") as an inner product operator? That's the way
    that APL does it, but every time it's brought up, it is
    dismissed out of hand. Yes, it would give an additional,
    different, meaning to ".", but "%" is used for very different
    purposes without too much confusion. Uisng the string "+.*" in
    Python code is a syntax error, so we don't have to worry about
    breaking pre-existing code. I envision A+.*B as syntactic sugar
    for
  inner_product(operator.add, operator.mul)(A, B)

  where inner_product is defined (for simple sequence objects) as
    equivalent to this:
  class inner_product(object):
def __init__(self, f, g):
    self.f, self.g = f, g
def __call__(self, a, b):
    return reduce(self.f, (self.g(*pair) for pair in zip(a, b)))

  The compiler would be allowed to instantiate +.* only once per
    sequence type, no matter how many times it appeared in the code,
    however non-standard sequence types (such as numpy's ndarrays)
    would be able to provide alternative implementations which would
    only be used by those types. (I envision a dict attached to the
    PySequenceMethods struct that would cache, for example,
    inner_product(operator.add, operator.mul) under the key '"+.*")
  —
    Reply to this email directly or view
      it on GitHub.

@cjw43 There is no "destruction of the Class" to be discussed in the PEP. If you are talking about the possible deprecation of the matrix class, it will only come quite a few years from now, if at all, and is irrelevant to the PEP as it is entirely internal to numpy.

Please, please, please use the Github interface for commenting or else trim your email replies to just your content. But as the PEP has already been accepted, continuing to press this particular point does not appear to be a good use of anyone's time. The decisionmakers in the core Python dev team consider the amount of discussion in the PEP as it stands perfectly adequate.

@njsmith Ready to merge?

Oh right, this PR.

Sure, I guess that makes sense to save the history, though I suppose the
version in the PEPs repository is the canonical one now. They should be
identical though.
On 6 Jul 2014 17:42, "Charles Harris" notifications@github.com wrote:

@njsmith https://github.com/njsmith Ready to merge?

—
Reply to this email directly or view it on GitHub
#4351 (comment).

History saved! Thanks for that.

Super late to the party, but I'd like to cross-reference JuliaLang/julia#4774 here, where it looks like very similar discussions have taken place in the context of how to define vector transposition.

Re the use of a potential @@ operator. In a lot of mathematical contexts, we actually consider exp(A) for some matrix A. exp(A)=I + A + A@@2/2! + A@@3/3! + A@@4/4! + .... continuing like the Taylor Series. Other useful contexts for the @@ operator exist, but it is definitely specifically for math people and may be noise to most non-scientific programmers.

On 30 January 2016 at 17:12, owendavey notifications@github.com wrote:

Re the use of a potential @@ operator. In a lot of mathematical contexts,
we actually consider exp(A) for some matrix A. exp(A)=I + A A@@2/2! + A@@3/3!

• A@@4/4! + .... continuing like the Taylor Series. Other useful contexts
  for the @@ operator exist, but it is definitely specifically for math
  people and may be noise to most non-scientific programmers.

In computer graphics, one multiplies matrices a lot. Considering that this
is certainly going beyond "science",
I don't quite understand what you meant to say.

@dimpase - I was not referring to matrix multiplication. I was referring to matrix exponentiation - hence the double @ operator which above seems to have been considered not important enough to be added in addition to the single @ operator of matrix multiplication for this PEP.

The discussion here has ended. The PEP has been accepted and implemented. If you would like to propose or discuss the merits of further changes, please find an appropriate mailing list. Thanks.