By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
459,186 Members | 1,530 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 459,186 IT Pros & Developers. It's quick & easy.

Linq to Python

P: n/a
Hi

Will LINQ be ported to Python ?

regards
Hrishy

Sep 23 '08 #1
Share this Question
Share on Google+
21 Replies


P: n/a
hrishy wrote:
Hi

Will LINQ be ported to Python ?
Take a look at SQLAlchemy or SQLObject for python-based
ORM/SQL-abstractions.

Apart from that, python is already heavily based on concepts like iterators,
filtering. Take a look at itertools.

Diez
Sep 23 '08 #2

P: n/a
On Sep 23, 7:48*am, hrishy <hris...@yahoo.co.ukwrote:
Hi

Will LINQ be ported to Python ?

regards
Hrishy
I think this question is more appropriate to ask on an IronPython
development list -- LINQ is pretty solidly intertwined with .Net, and
so you'll likely want to look at the .Net implementation of Python.
Sep 23 '08 #3

P: n/a
On Sep 23, 2:07*pm, Jason Scheirer <jason.schei...@gmail.comwrote:
On Sep 23, 7:48*am, hrishy <hris...@yahoo.co.ukwrote:
Hi
Will LINQ be ported to Python ?
regards
Hrishy

I think this question is more appropriate to ask on an IronPython
development list -- LINQ is pretty solidly intertwined with .Net, and
so you'll likely want to look at the .Net implementation of Python.
But surely the idea behind it will eventually spread. It's really
just comprehensions generalized over XML and relational datasets, a
noble goal. Besides, it's main purpose for .NET was to bring
functional programming to it. Python already has that, somewhat...
Sep 23 '08 #4

P: n/a
But surely the idea behind it will eventually spread. *It's really
just comprehensions generalized over XML and relational datasets, a
noble goal. *Besides, it's main purpose for .NET was to bring
functional programming to it. *Python already has that, somewhat...
it's really any object out of the box, i think the sql linq stuff is
more of a query compiler, IMO sqlalchemy does that.
query = select(user_cols,

and_(table_relationship.c.accept_user_id==user.id,

table_relationship.c.start_date==None

),

from_obj=join(

table_relationship,table_user,

onclause=table_user.c.id==table_relationship.c.ini t_user_id

).outerjoin(table_profile)

)

session.execute(query).fetchall()


XML? meh hopefully I would never need it. :)
C# is my day job, and when I got my hands on LINQ back in January my
initial thought was "Finally I have list comprehensions!!!! day job is
fun again"

For the most part, I think C# is catching up.
Sep 23 '08 #5

P: n/a
On Sep 23, 4:48*pm, hrishy <hris...@yahoo.co.ukwrote:
Will LINQ be ported to Python ?
No, because Python already has list comprehensions and we don't need
the XML buzzword.
Sep 23 '08 #6

P: n/a
sturlamolden:
>No, because Python already has list comprehensions and we don't need the XML buzzword.<
LINQ is more than buzzwords. Python misses several of those features.
So maybe for once the Python crowd may recognize such C# feature as
much better than things present in Python.
Said that, I presume Python will go on as usual, and LINQ-like
capabilities will not be integrated in Python. In the meantime where I
live lot of people will keep using C# instead of Python and CLisp,
natural selection at work indeed.

Bye,
bearophile
Sep 24 '08 #7

P: n/a
be************@lycos.com wrote:
sturlamolden:
>No, because Python already has list comprehensions and we don't need the XML buzzword.<

LINQ is more than buzzwords. Python misses several of those features.
So maybe for once the Python crowd may recognize such C# feature as
much better than things present in Python.
Said that, I presume Python will go on as usual, and LINQ-like
capabilities will not be integrated in Python. In the meantime where I
live lot of people will keep using C# instead of Python and CLisp,
natural selection at work indeed.
Perhaps a quick summary of what LINQ offers which might
"be integrated into Python" would help those of us who
are ignorant? (This is a serious comment; I'd like to know).

TJG
Sep 24 '08 #8

P: n/a
On Wed, Sep 24, 2008 at 2:11 PM, <be************@lycos.comwrote:
sturlamolden:
>>No, because Python already has list comprehensions and we don't need the XML buzzword.<

LINQ is more than buzzwords. Python misses several of those features.
So maybe for once the Python crowd may recognize such C# feature as
much better than things present in Python.
Said that, I presume Python will go on as usual, and LINQ-like
capabilities will not be integrated in Python. In the meantime where I
live lot of people will keep using C# instead of Python and CLisp,
natural selection at work indeed.
Why do you swing so widely between being an interesting poster with
interesting, useful things to say and mindless trolling? There's a lot
of reasons to use C# instead of Python or Lisp (or any other language
for that matter), but I can't imagine that someone would make that
decision based solely on LINQ. I'd go so far as to say that if they
do, I question their professional competence.

LINQ is an interesting implementation of an old idea. The place where
it differs from list comps is that it has appropriate hooks for the
specific linq implementation to override the way results are gathered,
and such hooks in Python would be an interesting idea. I've taken a
shot and figuring out where and how they should be implemented, but I
haven't come up with anything I like. Maybe you should try it?

LINQs "native" object API (not the syntax sugar available in C#) is
not very different from Python ORMs.
Sep 24 '08 #9

P: n/a
r0g
be************@lycos.com wrote:
sturlamolden:
>No, because Python already has list comprehensions and we don't need the XML buzzword.<

LINQ is more than buzzwords. Python misses several of those features.
So maybe for once the Python crowd may recognize such C# feature as
much better than things present in Python.
Said that, I presume Python will go on as usual, and LINQ-like
capabilities will not be integrated in Python. In the meantime where I
live lot of people will keep using C# instead of Python and CLisp,
natural selection at work indeed.

Bye,
bearophile
LOL, I just read that and thought - Ok this sounds serious I'd better go
find out what this LINQ business is all about so I googled.. and ended
up on MSDN where there's impressive sounding talk about how we need a
way to query databases and XML files with a unified syntax like we do
for for standard datatypes like files and arrays. Well yes,that makes
sense I think and proceed to look at their example code, curious as to
what this new paradigm looks like:

using System;
using System.Linq;
using System.Collections.Generic;

class app {
static void Main() {
string[] names = { "Burke", "Connor", "Frank",
"Everett", "Albert", "George",
"Harris", "David" };

IEnumerable<stringquery = from s in names
where s.Length == 5
orderby s
select s.ToUpper();

foreach (string item in query)
Console.WriteLine(item);
}
}

ROTFLMAO! Wow, what progress they're making! Quick guys let's jump on
before we get left behind - we dont want to miss out on this exciting
and mysterious 'foreach' construct or this strange and exotic sounding
'IEnumerable query' thing. To think that python might someday reach such
lofty heights where we'll be able to simply write...

names = ["Burke", "Connor", "Frank", "Everett",
"Albert", "George", "Harris", "David"]

result = [each.upper() for each in names if len(each) == 5]

result.sort()

for each in result: print each
Yes clearly 'the Python crowd' must admit LINQ is 'much better', I'm
sold, in fact off to download my "Free, but limited editions of Visual
Studio 2005 for a single programming language supported by .NET" right away!

OK so maybe I'm being naive here but it looks to me like this new
paradigm's big idea is to use a python + SQL type syntax to access data
in random objects. Big whoop. It's not that difficult to write a
generators that wraps XML files and databases is it?

What am I missing here?
Roger Heathcote.
Sep 24 '08 #10

P: n/a
On Sep 24, 9:11*pm, bearophileH...@lycos.com wrote:
In the meantime where I
live lot of people will keep using C# instead of Python and CLisp,
natural selection at work indeed.
Please explain to me what Linq can do that Python does not. Put you
emphasis on why this can't be done with a library, and thus will
require addition of new syntax to the language.
Sep 24 '08 #11

P: n/a
r0g <ai******@technicalbloke.comwrote:
OK so maybe I'm being naive here but it looks to me like this new
paradigm's big idea is to use a python + SQL type syntax to access data
in random objects. Big whoop. It's not that difficult to write a
generators that wraps XML files and databases is it?

What am I missing here?
Simple LINQ expressions like the one you gave map easily to Python list
comprehensions. What Microsoft have done though is provide a consistent
implementation which allows you to write complex SQL like expressions which
will work identically on databases or most other sequence types.

Here's another LINQ example:

List<Customercustomers = GetCustomerList();

var customerOrderGroups =
from c in customers
select
new {c.CompanyName,
YearGroups =
from o in c.Orders
group o by o.OrderDate.Year into yg
select
new {Year = yg.Key,
MonthGroups =
from o in yg
group o by o.OrderDate.Month into mg
select new { Month = mg.Key, Orders = mg }
}
};

ObjectDumper.Write(customerOrderGroups, 3);

I'm not overly keen on SQL syntax: I think this sort of complex expression
is hard to read. LINQ has a pretty straightforward conversion to method
calls, and for me this consistent set of methods is the important thing
rather than the SQL syntax. e.g. 'group by' maps directly to a call to a
method GroupBy(), so another of Microsoft's LINQ examples is:

public void Linq45() {
string[] anagrams = {"from ", " salt", " earn ", " last ",
" near ", " form "};

var orderGroups = anagrams.GroupBy(
w =w.Trim(),
a =a.ToUpper(),
new AnagramEqualityComparer()
);

ObjectDumper.Write(orderGroups, 1);
}

public class AnagramEqualityComparer : IEqualityComparer<string>
{
public bool Equals(string x, string y) {
return getCanonicalString(x) == getCanonicalString(y);
}

public int GetHashCode(string obj) {
return getCanonicalString(obj).GetHashCode();
}

private string getCanonicalString(string word) {
char[] wordChars = word.ToCharArray();
Array.Sort<char>(wordChars);
return new string(wordChars);
}
}

Python still wins hands down on this example both in verbosity and
readability:
>>anagrams = ["from ", " salt", " earn ", " last ",
" near ", " form "]
>>from itertools import groupby
def AnagramKey(w):
return sorted(w.strip().upper())
>>for k,words in groupby(sorted(anagrams, key=AnagramKey),
key=AnagramKey):
for w in words:
print w.strip().upper()
print "..."
EARN
NEAR
....
SALT
LAST
....
FROM
FORM
....

I haven't yet had occasion to use LINQ in anger yet, so I have no idea
whether its an idea to love or to hate. I do think it is good that C# has
effectively sprouted list comprehensions (not to mention anonymous types
and type inferencing) and I expect there may be some aspects worth looking
at for Python but I think they are more likely to lead to itertools
functions than extensions to syntax.
Sep 24 '08 #12

P: n/a
On Sep 24, 10:59*pm, Duncan Booth <duncan.bo...@invalid.invalid>
wrote:
Simple LINQ expressions like the one you gave map easily to Python list
comprehensions. What Microsoft have done though is provide a consistent
implementation which allows you to write complex SQL like expressions which
will work identically on databases or most other sequence types.
han extensions to syntax.
List comprehensions work with any iterable sequence. You can nest them
to make more complex statements. You can also use a generator to
iterate through a database or an XML document. Here is approximately
where linq stops being a marvelous addition to Python.

And I can honestly do without the SQL syntax.

Sep 24 '08 #13

P: n/a
On Sep 24, 4:59*pm, Duncan Booth <duncan.bo...@invalid.invalidwrote:
....
I haven't yet had occasion to use LINQ in anger yet, so I have no idea
whether its an idea to love or to hate. I do think it is good that C# has
effectively sprouted list comprehensions (not to mention anonymous types
and type inferencing) and I expect there may be some aspects worth looking
at for Python but I think they are more likely to lead to itertools
functions than extensions to syntax.
My thoughts exactly when I first looked at it. "Hey C# NOW has list
comprehensions!!! SWEET!!!"

As I understand it LINQ is libraries(System.Data.Linq.dll,
System.XML.Linq.dll) + syntactic sugar, i wouldn't call that a change
to the language. The addition of lambdas and functions as first class
objects however, that was indeed a language change, and it was a
change for the better that makes LINQ possible, (also makes writing
delegates less cumbersome). To someone other than a C# person, these
features are not new or revolutionary, it's just C# catching up. Kudos
to them for getting there before java.

After some more thought on what might conceivably be missing from
python that LINQ has, that someone might want is the equivalent of
System.XML.Linq.dll, and I can't see any reason why someone couldn't
write it with the exception that there doesn't seem to be a definitive
XML lib for python, meaning generally regarded as the best. That in my
mind would be a prerequisite. But if someone wrote both the XML lib
with the LINQ-ish syntax, maybe it would become the definitive XML
lib.

Again, I personally don't see myself using XML and thus needing that
unless my hand was forced by some business requirement that dictated I
use XML, I would not use it if I had a choice. The SQL LiNQ-ish
though, I think every python ORM has that covered to a degree, and I
use that quite a bit.

Sep 24 '08 #14

P: n/a
r0g
Duncan Booth wrote:
r0g <ai******@technicalbloke.comwrote:
>OK so maybe I'm being naive here but it looks to me like this new
paradigm's big idea is to use a python + SQL type syntax to access data
in random objects. Big whoop. It's not that difficult to write a
generators that wraps XML files and databases is it?

What am I missing here?

Simple LINQ expressions like the one you gave map easily to Python list
comprehensions. What Microsoft have done though is provide a consistent
implementation which allows you to write complex SQL like expressions which
will work identically on databases or most other sequence types.
Hmm, that's a nice idea in theory but I don't think it's the python
killer Mr/Ms bearophile thinks it is. I can't think of any use cases
where this would save me a great deal of time (certainly not enough to
offset the relative slowness of working in C#) but supposing they do
exist this stuff definitely belongs in a module. Seems to me C# is
playing catchup in most ways and this nugget of 'innovation' is just the
exception that proves the rule.

Roger.
Sep 25 '08 #15

P: n/a
sturlamolden <st**********@yahoo.nowrote:
On Sep 24, 10:59*pm, Duncan Booth <duncan.bo...@invalid.invalid>
wrote:
>Simple LINQ expressions like the one you gave map easily to Python
list comprehensions. What Microsoft have done though is provide a
consistent implementation which allows you to write complex SQL like
expressions whi
ch
>will work identically on databases or most other sequence types.
han extensions to syntax.

List comprehensions work with any iterable sequence. You can nest them
to make more complex statements. You can also use a generator to
iterate through a database or an XML document. Here is approximately
where linq stops being a marvelous addition to Python.
A lot of what LINQ does is already easy to do in Python, and most of the
rest can probably be added fairly easily, but it does provide a consistent
framework which may make it easier to do complex LINQ statements than
complex list comprehensions.

BTW, a minor correction: LINQ statements are closer to generators, not list
comprehensions. They don't actually evaluate their results until you
iterate over them and you can re-used the same LINQ statement multiple
times getting different results if the data has changed.
And I can honestly do without the SQL syntax.
snap :)

--
Duncan Booth http://kupuguy.blogspot.com
Sep 25 '08 #16

P: n/a
On 25 Sep, 10:08, Duncan Booth <duncan.bo...@invalid.invalidwrote:
A lot of what LINQ does is already easy to do in Python, and most of the
rest can probably be added fairly easily, but it does provide a consistent
framework which may make it easier to do complex LINQ statements than
complex list comprehensions.
Yes, that's the word, "consistent framework". I wonder what that
means? Do you mean Python syntax is inconsitent?

BTW, a minor correction: LINQ statements are closer to generators, not list
comprehensions. They don't actually evaluate their results until you
iterate over them and you can re-used the same LINQ statement multiple
times getting different results if the data has changed.
Python has generator expressions with the same syntax as list
comprehensions, except you use () instead of [].



Sep 25 '08 #17

P: n/a
sturlamolden <st**********@yahoo.nowrote:
On 25 Sep, 10:08, Duncan Booth <duncan.bo...@invalid.invalidwrote:
>A lot of what LINQ does is already easy to do in Python, and most of
the rest can probably be added fairly easily, but it does provide a
consistent framework which may make it easier to do complex LINQ
statements than complex list comprehensions.

Yes, that's the word, "consistent framework". I wonder what that
means? Do you mean Python syntax is inconsitent?
No Python's syntax is fine. The api's aren't as consistent though:
Microsoft added a common set of extension methods which work on
databases, xml, builtin sequences and can be easily extended to include
other custom sequences.

As an example to filter a list in Python you'd use a list comprehension
with an 'if', but for a database you'd probably prefer a select with a
'where' clause so as to avoid retrieving and discarding 999999 or your 1
million rows. The apis defined by LINQ allow that sort of optimisation
to happen transparently, the simple list would just test each element
but the database would run an appropriate query.

So what the 'can we have LINQ in Python' people are asking is to be able
to write things like:

x = (c for c in customers if c.id=='123')

and know that they aren't doing a linear search unless that is the best
that can be done. The LINQ equivalent would be something like:

var x = from c in customers where c.id=='123'
select new { c.name, c.id };

which is compiled to:

var x = customers.Where(c =c.id=='123');

and depending on the type of 'customers' the Where method can either get
a callable function to test the condition or an expression tree which it
can compile into another language such as SQL (C# lambdas can compile
either to executable code or to Expression<objects that you can
further process).

There's an article at
http://www.interact-sw.co.uk/iangblo...xpressiontrees which
has a real example showing how:

DataContext db = new DataContext("server=.;initial catalog=northwind");
Table<Ordersorders = db.GetTable<Orders>();
Table<Customerscustomers = db.GetTable<Customers>();

var q = from o in orders, c in customers
where o.ShipCity == "London" && (o.CustomerID == c.CustomerID)
select new { o.OrderDate, c.CompanyName, c.ContactTitle,
c.ContactName };

actually ends up as a single SQL query:

exec sp_executesql N'SELECT [t1].[CompanyName], [t1].[ContactName],
[t1].[ContactTitle], [t0].[OrderDate]
FROM [Orders] AS [t0], [Customers] AS [t1]
WHERE ([t0].[ShipCity] = @p0) AND ([t0].[CustomerID] = [t1].
[CustomerID])', N'@p0 nvarchar(6)', @p0 = N'London'

>BTW, a minor correction: LINQ statements are closer to generators,
not list comprehensions. They don't actually evaluate their results
until you iterate over them and you can re-used the same LINQ
statement multiple times getting different results if the data has
changed.

Python has generator expressions with the same syntax as list
comprehensions, except you use () instead of [].
It might surprise you to know that I have actually come across generator
expressions. :^)

My wording was deliberate: LINQ queries are re-usable, so are Python's
generators, but generator expressions are not. The comparison isn't
exact, you have to call the generator to get an iterator whereas a LINQ
expression gives you something which is directly iterable.

--
Duncan Booth http://kupuguy.blogspot.com
Sep 25 '08 #18

P: n/a
On 25 Sep, 14:22, Duncan Booth <duncan.bo...@invalid.invalidwrote:
No Python's syntax is fine. The api's aren't as consistent though:
Microsoft added a common set of extension methods which work on
databases, xml, builtin sequences and can be easily extended to include
other custom sequences.
That is correct, but it is a library issue and cannot be solved by
adding new syntax.
Sep 25 '08 #19

P: n/a
Duncan Booth:
Microsoft added a common set of extension methods which work on
databases, xml, builtin sequences and can be easily extended to include
other custom sequences.
When the processing is done in memory, LINQ may also work well with
multi-core CPUs, see PLINQ.

Bye,
bearophile
Sep 25 '08 #20

P: n/a
2008/9/24 Duncan Booth <du**********@invalid.invalid>:

Python still wins hands down on this example both in verbosity and
readability:
But AFAICS, the Python version you give creates a temporary. One of
the advantages cited for LINQs functional programming paradigm is that
it specifies what is wanted at a higher level, so the compiler can
decide whether to create temporaries, and can also decide whether to
farm the thing off to multiple processors -- harder if you've
specified in detail /how/ to do the job. Not an issue for little jobs,
but certainly an issue for, for example, a friend who had a daily
database job to do that took over 24 hours to run.
I haven't yet had occasion to use LINQ in anger yet, so I have no idea
whether its an idea to love or to hate. I do think it is good that C# has
effectively sprouted list comprehensions (not to mention anonymous types
and type inferencing) and I expect there may be some aspects worth looking
at for Python but I think they are more likely to lead to itertools
functions than extensions to syntax.
Yes, looking at what LINQ adds to C# (according to
http://msdn.microsoft.com/en-gb/library/bb397909.aspx):
- Implicitly typed variables: Python already has.
- Object and collection initialisers: Not sure whether Python can do
this directly, but it can certainly emulate it with a dictionary.
- Anonymous types: Not sure whether Python can do this directly, but
it can certainly emulate it with a dictionary.
- Extension methods: Python already has.
- Lambda expressions: Python already has.
- Auto-Implemented properties: No, but that's just syntactic sugar to
make declarations more compact.

So all of the language elements that are needed for LINQ are already
in Python; a library should do the trick.

--
Tim Rowe
Oct 27 '08 #21

P: n/a
"Tim Rowe" <di*****@gmail.comwrote:
>I haven't yet had occasion to use LINQ in anger yet, so I have no
idea whether its an idea to love or to hate. I do think it is good
that C# has effectively sprouted list comprehensions (not to mention
anonymous types and type inferencing) and I expect there may be some
aspects worth looking at for Python but I think they are more likely
to lead to itertools functions than extensions to syntax.

Yes, looking at what LINQ adds to C# (according to
http://msdn.microsoft.com/en-gb/library/bb397909.aspx):
- Implicitly typed variables: Python already has.
- Object and collection initialisers: Not sure whether Python can do
this directly, but it can certainly emulate it with a dictionary.
- Anonymous types: Not sure whether Python can do this directly, but
it can certainly emulate it with a dictionary.
- Extension methods: Python already has.
- Lambda expressions: Python already has.
- Auto-Implemented properties: No, but that's just syntactic sugar to
make declarations more compact.

So all of the language elements that are needed for LINQ are already
in Python; a library should do the trick.
Not quite. The C# implementation of lambda expressions has a neat trick
that is central to LINQ. A lambda expression compiles either to executable
code, or to an expression tree. If you are filtering some C# sequence
object then (as with Python lambdas) the LINQ code simply calls the
compiled lambda expression like any other delegate, but if you are
filtering a SQL lookup the LINQ code gets an expression syntax tree and
further compiles it (at runtime) into SQL code to perform the filtering at
the database level.

Python can't do the compile time matching to vary what it produces
according to how the lambda is used, but it would be perfectly possible for
function objects which are compiled from a lambda expression to grow an
extra attribute that would hold an AST for the expression. That way you
could write code that took a filter function and, if it had the AST
attribute available, compiled it to SQL or xpath or whatever else you
fancied.
Oct 27 '08 #22

This discussion thread is closed

Replies have been disabled for this discussion.